Multi-agent systems in practice.

What counts as a multi-agent system

A multi-agent system is any design where several model instances, each with its own instructions, tools and context window, cooperate on one task. The label covers four quite different patterns, and the right test depends on which one you mean.

Anthropic’s guidance on building effective agents recommends the simplest solution that works, which may mean no agent at all. Our lesson on when a workflow does not need an agent applies doubly to multiple agents.

When multiple agents help, and when they hurt

The evidence points one way: multi-agent designs help on wide, parallelisable work and hurt on tightly coupled work.

Anthropic reported in How we built our multi-agent research system (June 2025) that a lead agent with parallel sub-agents outperformed a single agent by 90.2% on its internal research evaluation. The same post reports that agents use about 4 times the tokens of a chat and multi-agent systems about 15 times, and that token usage alone explained 80% of performance variance in one browsing evaluation. It also says domains where agents must share the same context or have many dependencies, including most coding, are not a good fit today.

Cognition’s Don’t Build Multi-Agents argues the opposite case from the same facts: share full context and traces, because actions carry implicit decisions and conflicting decisions produce bad results.

Read together, the rule is practical. If sub-tasks can run independently and return evidence, parallel agents can buy breadth. If each step depends on decisions made in the last one, a single agent with a well-managed context usually wins.

Design a controlled comparison

Decide with an experiment, not an architecture diagram. The comparison has to hold everything constant except the thing being tested.

1. Fix the task set. Use 50 or more tasks drawn from real work, each with written success criteria and gold evidence, stratified by task type. Keep a share unseen until the final run.
2. Hold the stack constant. Same models, tools, retrieval, data access and shared instructions for both systems.
3. Give the single agent a fair budget. Run it at the multi-agent system’s average token spend as well as at its own default. Since spend explains so much of the gap in published results, a comparison at unequal budgets mostly measures spend.
4. Repeat runs. Three or more runs per task per system, because outputs vary between runs.
5. Grade blind. Graders, human or model, do not know which system produced an output. Check any model judge against a human-graded sample first.
6. Analyse paired differences. Compare the two systems task by task and report a confidence interval, not two averages.

import random
from statistics import mean

def paired_bootstrap(single: dict[str, float], multi: dict[str, float],
                     n_boot: int = 10_000, seed: int = 7):
    """Scores per task id (0 to 1, averaged over repeat runs).
    Returns the mean improvement of multi over single and a 95% interval."""
    tasks = sorted(set(single) & set(multi))
    diffs = [multi[t] - single[t] for t in tasks]
    rng = random.Random(seed)
    boots = sorted(mean(rng.choices(diffs, k=len(diffs))) for _ in range(n_boot))
    return mean(diffs), (boots[int(0.025 * n_boot)], boots[int(0.975 * n_boot)])

def cost_per_success(runs: list[dict]) -> float:
    successes = sum(r["passed"] for r in runs)
    return sum(r["cost_usd"] for r in runs) / successes if successes else float("inf")

If the interval for the improvement crosses zero, you have not shown a difference, whatever the averages say. The broader method for task suites is in AI agent evaluations.

What to measure

Scoring should punish noise as well as reward hits. GTM-Bench, a 72-task benchmark for go-to-market research agents published by Blackpearl Group and co-authored by Levy Street team members, scores each returned lead +1, 0 or −1, so a system cannot win by returning more rows. Volume-inflating systems are a classic multi-agent failure, because each sub-agent adds its own findings.

Design rules if the multi-agent system wins

When the comparison favours multiple agents, these rules keep the advantage in production:

• Delegate with a full task specification. Objective, output schema, sources to use and avoid, stopping rule and budget. Vague delegation is the most common cause of duplicated or misdirected work.
• Sub-agents gather; one agent acts. Keep writes, approvals and final decisions with a single orchestrator, so two agents never make conflicting changes. Our guide to designing reliable AI agents covers that control boundary.
• Return evidence, not conclusions. Each sub-agent returns findings with citations the orchestrator can check.
• Cap fan-out and depth. A maximum number of sub-agents per task and no sub-agents spawning sub-agents without an explicit limit.
• Trace everything under one ID. Every sub-agent’s full trace links to the parent run, so failures can be reconstructed.
• Verify before synthesis. A check step confirms each cited claim against its source before the final answer is written.

Budgets per run and per sub-agent belong in the same design; see managing LLM costs.

Where multi-agent systems go wrong

The MAST taxonomy in Why Do Multi-Agent LLM Systems Fail? identifies 14 failure modes in three categories: system design issues, inter-agent misalignment and task verification. The table maps them to symptoms you can detect.

Retry and recovery rules for each agent still apply; see agent retry and failure recovery. If you are weighing a multi-agent design for a real workflow, our AI agent development work starts with this comparison.

• The pattern (orchestrator, handoff, critic or vote) is named, and a single-agent baseline exists
• The task set has 50 or more real tasks with written success criteria and gold evidence
• Models, tools, data and shared instructions are identical across both systems
• The single agent is also run at the multi-agent system’s token budget
• Each task runs at least three times per system, and grading is blind
• Results are reported as paired differences with a confidence interval
• Cost per successful task, latency and coordination overhead are reported with quality
• Sub-agents receive a full task specification and return cited evidence
• Writes and final decisions sit with one agent; fan-out and depth are capped
• Every sub-agent trace links to the parent run ID

Written by the Levy Street team in Wellington. Last updated 19 October 2026.