Some workflows don't fit inside one model. We design composite systems — specialist models coordinated by a purpose-built harness that routes each step, validates the output, and hands off when one model isn't the right tool for the job.
A focused model is the right call for a narrow, repeatable workflow. Once a task spans several kinds of work, the calculus changes — and published research backs up why.
No single model is best at everything. Routing research shows that sending each query to the model best suited for it — rather than the biggest model for every query — can cut cost by over 85% with no drop in response quality.
Layered ensembles where models see and build on each other's outputs have outperformed the single strongest model available on standard benchmarks — a documented "collaborativeness" effect between LLMs.
Even holding the model constant, different agent scaffolds disagree on close to a third of individual tasks despite similar headline accuracy. An oracle that just picked the best scaffold per task reached 100% — the setups are complementary, not redundant.
Not every sub-task deserves the most expensive model. A composite system can route routine steps to a small, fast model and escalate only the steps that need more capability.
How you wire a model up — routing, validation, retries, hand-offs — is not a minor implementation detail. It's frequently the biggest lever you have, as the next section shows.
Three independent, published studies held the model fixed and varied only the harness. Each found harness choice swung benchmark scores by double digits — in one case, more than switching to a better model did.
| Benchmark | Model (held fixed) | Harness | Score |
|---|---|---|---|
| Harness-Bench 6 harnesses, 106 tasks |
Same backend model | Best harness | 76.2% |
| Worst harness | 52.4% | ||
| CORE-Bench Hard | GPT-5.4 | Codex CLI scaffold | 97.4% |
| CORE-Agent scaffold | 51.3% | ||
| SpatialBench | Opus-4.5 | Latch harness | 61.7% |
| Claude Code harness | 48.1% | ||
| No harness (base) | 38.4% |
On SpatialBench, the 23.3-point gap between the best and worst harness for Opus-4.5 exceeded the 2.9-point gap between Opus-4.5 and Sonnet under the same harness — harness choice moved the score more than a model upgrade did. Sources: Harness-Bench (Yao et al.), Life After Benchmark Saturation: A Case Study of CORE-Bench (Nadgir, Kapoor, Narayanan et al.), SpatialBench (Workman et al.).
Same discipline as a single focused model, applied across the whole workflow — every routing and hand-off decision maps to an observable difference in outcome.
Break the end-to-end job into the distinct kinds of work it actually requires — retrieval, reasoning, tool use, generation — and where they genuinely differ in what a model needs to be good at.
Some steps call for a general frontier model, others for a small model fine-tuned on your domain. We pick or train the right model for each piece rather than forcing one model to cover all of it.
Routing logic, output validation between steps, and hand-off rules for when one model's result isn't good enough — the layer that the research above shows matters as much as model choice.
We measure the composite system end-to-end on real tasks. A model that scores well alone doesn't guarantee the system scores well — the harness and hand-offs are evaluated together with the models.
More models and more moving parts is not automatically better — mixing different models has to earn its keep over a single well-tuned one. Here's how we think about the choice.
One narrow, repeatable domain — DevOps commands, a support queue, a single document type. A single fine-tuned model is simpler to run, cheaper, and faster than adding orchestration overhead you don't need.
The workflow spans genuinely different kinds of work, or task difficulty varies enough that routing the easy majority to a cheap model and escalating the rest pays for itself. Worth noting: research on mixing different models is mixed — some work finds no benefit over resampling one strong model, so this needs to be justified per workflow, not assumed.
Tell us what the end-to-end job looks like and where it currently breaks down. We'll scope whether it needs one focused model or a composite system, and what that looks like.
info@apinference.com