Supervisor vs hierarchical agent orchestration: how to choose

Three orchestration patterns, what each costs you, and a simple rule for picking the smallest one that works.

Multi-agent systems are popular because splitting a big prompt into specialists feels like good engineering. It can be. It can also multiply cost, latency and failure modes. The pattern you choose should follow the shape of the work.

The three patterns

1. Sequential handoff

Agent A finishes and passes its result to agent B. It is easy to reason about and trace, and it fits pipelines such as extract, validate, draft. Its weakness is rigidity: if step three needs something from step one, the chain has to carry it.

2. Supervisor and specialists

One agent receives the request, decides which specialist to call, reads the result and decides what happens next. Specialists have narrow prompts and tools. This works well for a handful of distinct capabilities, such as a billing specialist, a technical specialist and a returns specialist behind one support entry point.

3. Hierarchy

Supervisors manage supervisors. A top-level planner splits a large goal into workstreams, each run by its own supervisor with specialists beneath it. It scales to big tasks like research reports or multi-system migrations, at the price of more tokens, more latency and more places to fail.

A rule for choosing

Pick the smallest topology that meets the quality bar on your evaluation set. In practice that usually means:

• Single agent with tools when the task fits in one context and one set of permissions.
• Sequential handoff when the work is a fixed pipeline.
• Supervisor when requests vary and you have distinct capabilities to route among.
• Hierarchy only when a supervisor's context overflows or the work splits into genuinely independent workstreams.

What matters more than the pattern

Typed shared state. Agents that pass free text lose detail at every handoff. Agents that read and write a typed state object, with fields such as customer, order, plan and open questions, stay consistent and are easy to log.

Budgets. Every run needs limits on steps, tokens and wall-clock time. Without them a supervisor can bounce a request between specialists indefinitely.

Traces. Record every message, tool call and routing decision. When a run goes wrong you should be able to replay it and see which decision was the first bad one.

Deterministic routing where possible. If a rule can decide, such as the ticket's product field, use the rule. Reserve the model for decisions that need judgement.

A common failure

A team splits one agent into six specialists because the original prompt was long. Quality drops, because routing errors now compound, and cost rises because every request pays for a supervisor call. The better fix was usually to shorten the prompt, add retrieval for reference material and give the single agent clearer tools.

Where to go next

If you are planning an agent system, our multi-agent orchestration page describes how we design and test these architectures, and the agentic AI development overview covers the rest of the stack.