Multi-Agent Systems: The 3 Architectures That Work in Production

There is a predictable moment in every agent project where the single-agent approach stops working. You started with one agent, gave it a few tools, and it was brilliant. Then you added a few more tools and a few more responsibilities, and it got worse. Not slightly worse, noticeably worse: it picks the wrong tool, forgets the thread, and reasons less clearly than it did when it knew less. You made it more capable and it became less competent.

That is the context-switching problem, and it is why multi-agent systems exist. Instead of one generalist juggling every responsibility, you assign specialized roles to separate agents, each with a narrow job, a small toolset, and a clean context. This covers the three architectures that actually work in production, the router, the network, and the hierarchy, plus the genuinely hard part nobody warns you about (shared state), and the handoff protocols that hold the whole thing together. It builds directly on the loop described in building agentic workflows.

Why single agents break down

A single agent degrades as you add responsibilities for two compounding reasons. The first is tool confusion: every tool you add expands the space of choices, and past a handful of similar-sounding options the model starts selecting valid-but-wrong tools and executing them flawlessly. The second is context dilution: a generalist's working context has to hold instructions for every job it might do, which crowds out the details of the job it is actually doing right now.

Put together, these mean capability and reliability pull in opposite directions past a certain point. The instinct is to fix it with a better model or a longer system prompt, and it does not work, because the problem is structural rather than intellectual. A human analogy is exact: you would not hire one person to be your lawyer, your accountant, and your plumber, not because no one is smart enough, but because expertise is narrow and attention is finite. Specialization is the fix, and multi-agent architecture is how you implement it.

Architecture 1: the router (hub and spoke)

The simplest multi-agent pattern is a router. One lightweight agent sits at the front, classifies the incoming request, and dispatches it to the right specialist, a billing agent, a technical-support agent, a research agent, each of which has only the tools and instructions for its own domain. The router itself does no real work; its entire job is to decide who should handle this.

This is the pattern to reach for first, because it is easy to reason about, easy to test, and easy to debug. Each specialist can be evaluated in isolation, and when something goes wrong you usually know immediately whether the router misclassified or the specialist misbehaved. Its limitation is that it assumes requests fall cleanly into categories and that one specialist can finish the job alone. When a task genuinely needs two specialists to collaborate, the router has no way to express that, and you need one of the next two patterns.

Architecture 2: the network (handoffs)

In a network, agents are peers that can pass control to one another. A research agent gathers material and hands off to a writer agent, which hands off to an editor agent, which may hand back to the researcher if something is missing. There is no central coordinator; each agent knows which colleagues exist and when to involve them.

Networks are powerful because they mirror how a real team collaborates, and they handle tasks whose shape is not known in advance. They are also where multi-agent systems most often go wrong. Without strict rules, agents ping-pong work back and forth, each politely deferring to the other, and you get an expensive infinite loop with excellent manners. Tracing a failure through a network is genuinely difficult, because the responsibility is distributed and no single agent has the whole picture. Use a network when collaboration is truly required, and instrument it heavily, because the engineering mistakes that bite hardest here are loops and lost context.

Architecture 3: the hierarchy (manager and workers)

The hierarchy is the pattern that scales furthest. A manager agent receives the goal, decomposes it into subtasks, delegates each to a specialized worker, collects the results, and assembles the final output. Workers do not talk to each other; they report up. This is a deliberate constraint, and it is the source of the pattern's strength.

Because communication flows through one coordinator, the system stays traceable: there is always a single place that knows the plan and the current state of every subtask. Each worker stays small, cheap, and independently testable, which keeps error rates low and lets you route simple subtasks to cheaper models. The trade is that the manager becomes a bottleneck and a single point of failure, and a bad plan at the top wastes every worker's effort below. Even so, for complex decomposable jobs at volume this is usually the right answer, which is why it anchors the strategies in scaling AI agents.

The hard part: shared state

Every multi-agent tutorial shows you the boxes and arrows. Almost none of them tell you that the hard problem is not the topology, it is the state. The moment two agents work on the same task, you have to answer questions that sound trivial and are not: what does each agent know, when does it learn it, and what happens when two agents believe different things about the same fact?

Get this wrong and you get failures that are maddening to debug. One agent updates a record while another is still reasoning about the old version and confidently acts on stale information. Or you naively share everything with everyone, which recreates the exact context-dilution problem that made you split the agents in the first place. The discipline is to be explicit and stingy: give each agent the minimum context it needs for its job, define a single source of truth for shared facts rather than letting each agent keep its own copy, and version that state so you can see what each agent knew at each step. This is the same state-isolation principle that runs through the agentic AI tech stack, and in multi-agent systems it is not optional, it is the whole ballgame.

Handoff protocols: the contract between agents

A handoff is not just passing a message; it is a transfer of responsibility, and it needs a contract as strict as any API. Sloppy handoffs are the single largest source of multi-agent failure, because the receiving agent gets an ambiguous blob of text and has to guess what it is meant to do with it.

• State what is being handed off, explicitly. The receiving agent needs the task, the relevant context, and the definition of done, not a transcript to interpret.
• Transfer responsibility unambiguously. Exactly one agent owns the task at any moment. If two think they own it, you get duplicated work; if none do, the task silently dies.
• Define legal handoffs up front. Not every agent should be able to hand off to every other. Constraining the graph prevents most loops before they can happen.
• Cap the hops. A hard limit on handoffs per task turns a potential infinite loop into a bounded failure you can catch and escalate.
• Log every transfer. When something goes wrong, the handoff trail is the only way to reconstruct what happened and where the reasoning went off course.

How to choose an architecture

Most teams overthink this. The choice follows from the shape of the work, and three questions settle it almost every time.

1. Does one specialist finish the job? If a request belongs to exactly one domain and that domain can complete it end to end, use a router. Do not build collaboration machinery for work that never needs collaborating.
2. Can you decompose the goal up front? If a coordinator can look at the goal and break it into subtasks, use a hierarchy. It stays traceable, keeps workers cheap and small, and scales further than anything else.
3. Is the path genuinely emergent? If you cannot know which specialist is needed next until the previous one reports back, and the sequence differs every time, use a network, and instrument it heavily.

When two answers seem to fit, take the simpler architecture. A router you outgrow is a cheap mistake to correct; a network you did not need is an expensive one to debug. It is also perfectly reasonable to mix them, a router at the front dispatching into a small hierarchy is a common and effective production shape, because it keeps the easy cases cheap and reserves the coordination machinery for the hard ones.

When not to use multiple agents

Multi-agent systems are fashionable, which means they get used where a single agent would do the job better. The honest guidance is to stay with one agent as long as you can. A single agent with a handful of well-chosen tools is simpler, cheaper, faster, and vastly easier to debug than any distributed alternative, and every agent you add multiplies the coordination surface, the token cost, and the number of ways things can fail.

The signal to split is concrete rather than aesthetic: you are splitting when one agent has grown so many tools that it reliably picks the wrong one, or when its instructions have grown so broad that its context is diluted, or when genuinely distinct expertise is required. Split at those seams, and split into the fewest agents that resolve the problem. Adding agents because the architecture diagram looks more impressive is how teams turn one manageable problem into five interacting ones.

Frequently asked questions

The bottom line

Multi-agent systems solve a real problem: a single agent gets less competent as you make it more capable, because tools confuse it and context dilutes it. The fix is specialization, implemented as a router for clean categories, a network for genuine collaboration, or a hierarchy for complex decomposable work at scale. But the architecture diagram is the easy part. The real work is shared state, deciding exactly what each agent knows and keeping one source of truth, and strict handoff protocols that transfer responsibility unambiguously and cap the hops. Start with one agent, split only when it demonstrably breaks, split into the fewest agents that fix it, and treat every handoff like an API contract rather than a conversation. For a picture of what this looks like when it works, see the multi-agent operations example.