Orchestrating specialised agents — selection, synthesis, and showing the work

One agent with forty tools degrades. Several narrow agents behind an orchestrator do not — provided the orchestrator selects before it asks, reads structured findings rather than prose, and can say it does not know.

The first version of an agent is a single model with a handful of tools, and it works. The tenth tool is where it stops working: tool selection accuracy falls, the context fills with schemas the model will not use for this question, and when the answer is wrong there is no way to tell which capability failed.

The usual next move is a set of narrow agents, each good at one thing, behind an orchestrator. That move is right, and it introduces four problems worth naming before building.

The loop

Figure 1 — Selection happens before invocation, not after. Agent D is not asked, which is the point: the cost of an orchestrator is the number of agents it wakes, and the quality of its answer falls as irrelevant findings accumulate.

1 · Select before you invoke

The naive orchestrator asks every agent and lets the synthesis step sort it out. It is simple, and it is wrong on three axes at once: latency is the slowest agent, cost is the sum of all agents, and answer quality decreases as irrelevant findings dilute the relevant ones.

Selection does not need a model. In most systems the question carries enough signal — the entities it names, the vocabulary it uses, the surface it came from — for a cheap classifier or a set of explicit rules. Where a model is warranted, it should return a ranked shortlist, not a yes/no per agent, so the orchestrator can apply its own budget.

Two properties are worth designing in from the start:

Selection must be inspectable. Log which agents were considered, which were selected, and why. "The orchestrator did not ask the agent that knew the answer" is a common failure, and undiagnosable without this.

Selection must be overridable. A caller who knows which agent they want should be able to say so. Routing is a heuristic, and heuristics need an escape hatch.

2 · Agents return findings, not prose

The single decision that determines whether this architecture holds up: an agent's output is a structured finding, not a paragraph.

from dataclasses import dataclass
from typing import Literal

@dataclass(frozen=True)
class Finding:
    agent: str                     # who produced it
    kind: Literal["fact", "anomaly", "absence"]
    statement: str                 # one claim, in natural language
    confidence: float              # class="tok-num">0..class="tok-num">1, calibrated per agent, not per model
    evidence: list[str]            # ids resolvable to a source, never free text
    computed_at: str               # ISO-class="tok-num">8601 — the synthesis step needs the age
    inputs_digest: str             # what it looked at, for reproducibility

Three consequences follow, and each one is the reason for a field above:

kind="absence" is a first-class result. "I looked and there is nothing" is different from "I was not asked" and from "I failed". Without the distinction, the synthesis step cannot tell silence from a negative, and it will guess.

evidence holds identifiers, not quotations. An id can be resolved, checked and shown to the user. A quotation copied into a payload is a claim about a source, not the source, and it survives the source being corrected.

confidence is per agent, not per model. A model's self-reported confidence is close to meaningless. A confidence that comes from the agent's own logic — how many corroborating rows, whether a threshold was crossed, how much data was missing — is usable. Where an agent genuinely cannot calibrate, omit the field rather than fabricate it.

3 · Synthesis is a separate job

Synthesis is not concatenation, and it is not "ask the model to summarise these". It has its own failure modes:

Conflicting findings must be surfaced, not averaged. Two agents disagreeing is information, often the most valuable output of the whole run. An orchestrator that quietly picks the higher confidence has destroyed it.

Age must be carried through. A finding computed six hours ago and one computed thirty seconds ago do not deserve equal weight, and the user should see the age of what they are being told.

Abstention must be reachable. If the selected agents returned nothing usable, the correct output is "I do not know, here is what I looked at" — with the list. This is the hardest behaviour to preserve, because every fluent-sounding alternative scores better in a demo.

The synthesis prompt must not be able to invent evidence. Ground it strictly in the findings passed to it, and validate afterwards that every evidence id it cites was in the input. That check is a few lines and it catches the failure that erodes trust fastest.

4 · Freshness, and when to re-run

Agents produce findings on a schedule; a question arrives at an arbitrary moment. The orchestrator therefore has to decide, per agent, whether an existing finding is still usable or whether the agent should run again.

A workable rule is a staleness budget per agent, derived from how fast its underlying data actually changes — not from a global default. Something derived from a nightly batch has a budget of a day; something derived from a live stream has minutes.

Re-running is where an orchestrator becomes dangerous, so it is worth constraining explicitly:

Opt-in, not default. Answering with stale data and saying so is safer than triggering work the caller did not ask for.
Bounded. A hard cap on how many agents may be re-run for one question. Without it, a single query can fan out into a workload nobody sized.
Asynchronous, with the stale answer returned now. The user gets an answer immediately, marked as stale; the refresh lands for the next question.
Authorised. Triggering computation is a privileged action, and it belongs behind the same authorisation as any other write.

5 · Make the state legible

A system that wakes several agents, waits on some, skips others and occasionally declines to answer is opaque from the outside. Two seconds of silence and a paragraph of text give the user no way to distinguish "thinking hard" from "stuck" from "had nothing".

Some form of live state display earns its place here — a progress indicator, a status line, an animation whose behaviour is bound to what the system is doing. The engineering constraint matters more than the form it takes:

Never render a state you do not measure. An indicator driven by a timer rather than by the orchestrator is a decoration that lies under load, and it lies exactly when the user most needs the truth. If a state cannot be observed, it should not appear in the display.

Degraded and offline are states too. The display has to have a rendering for "an agent is unreachable" and for "this ran with partial data". A visualisation that only knows how to look healthy is worse than no visualisation, because it converts an outage into confusion.

What it costs

This architecture is not free, and the costs are predictable:

More moving parts. Every agent is a deployment, a schedule, a failure mode and a permission.
Selection errors become the dominant failure. In a mature system, wrong answers are more often "asked the wrong agents" than "the agent was wrong" — which is why selection has to be logged.
Latency is the slowest selected agent, so bounded parallelism and per-agent timeouts are not optional.
Findings need versioning. A schema change on a finding type ripples into the synthesis step and into anything that stored one.

It is worth it when the domains are genuinely distinct and each needs its own tools, its own data access and its own permissions. It is not worth it for three tools and one data source — there, the single agent is still the right answer, and the interesting problem is elsewhere.