One agent is novel. Multiple agents working together is a production problem.
The moment you deploy more than one agent, you hit real architecture challenges:
- Agent A makes a decision; agent B sees stale state and makes a conflicting decision.
- Agent C fails; which agents does it cascade to?
- Who owns the outcome when multiple agents contributed to a decision?
- How do you debug a failure that spans three agents and took 15 minutes to resolve?
Most teams are not ready for this. Yet agentic systems are already multi-agent by design.
The root cause is consistent: teams treat multi-agent systems like chained prompts. They are not. They are distributed systems, and they fail like distributed systems.
Why this matters
Single-agent systems hit limits when tasks require genuine parallelism, domain specialization, or fault isolation across components.
Real-world problems require agents to:
- Divide work (some agents handle validation, others handle execution).
- Share state (one agent’s output is another’s input).
- Handle failures gracefully (one agent fails, others continue or escalate).
- Chain decisions (agent feedback updates agent behavior).
Without a coordination layer, multi-agent systems do not just slow down. They produce wrong outputs quietly. Each agent is fast, but the system as a whole is unreliable.
What changed
Frameworks like Semantic Kernel and AutoGen now make multi-agent composition easy to write.
They did not make multi-agent systems reliable.
The gap most teams fall into: they deploy orchestration code without designing state management, failure modes, or decision ownership. The framework compiles. Production breaks them.
There is also a split in the system that most teams never name explicitly.
Multi-agent systems introduce two distinct layers:
- Control plane: coordination, routing, decisions across agents
- Data plane: task execution, outputs, tool calls within each agent
Most failures happen at the boundary between these two. An agent completes its task successfully (data plane works) while the coordination layer passes stale context to the next agent (control plane fails). The result looks like a logic error. It is an architecture error.
A layered design
Use this as your starting framework:
Layer 1: Agent Charter Define what each agent does, what state it owns, and what it must not touch. If two agents can both write to the same state, you have already introduced a conflict.
Layer 2: State Management If your state is not durable, your system is not multi-agent. It is just multiple prompts chained together. Use a persistent store (database, event log). Never pass state only through memory or prompt context.
Layer 3: Coordination Protocol Define how agents signal each other (queues, pub/sub, event streams). The protocol must be explicit and inspectable. An agent that silently assumes another finished is a failure waiting to happen.
Layer 4: Failure Boundaries For each agent, decide: if this agent fails, what should happen? Contained failure (retry, fallback) or cascade (pause dependent agents, escalate to human)? Design this before you need it.
Layer 5: Observability Logging decisions per agent is necessary but not sufficient. You need to reconstruct the full decision path across agents: what state each agent received, what it produced, and in what order. Per-agent logs alone will not let you debug cross-agent failures.
Layer 6: Testing Test each agent in isolation first. Then test coordination failure modes explicitly: what happens when agent B reads stale state from agent A? What happens when agent C partially completes before failing? These are not edge cases. They are the normal failure modes of distributed systems.
Common failure modes
Most multi-agent failures fall into one of these patterns:
Stale read failure. Agent B reads state that agent A has not yet committed. Agent B produces an output based on incomplete or outdated information. Both agents report success.
Conflicting write. Two agents write to shared state without coordination. The last write wins. Neither agent is aware of the conflict. The downstream agent sees a result neither one intended.
Partial completion with no rollback. Agent A completes. Agent B starts, then fails mid-execution. The system is in an inconsistent state. No agent owns the recovery.
Hallucination propagation. Agent A produces a hallucinated intermediate result, such as a fabricated requirement, an incorrect API response, or a wrong field name. Agent B receives that output as ground truth and builds on it. Schema validation does not catch semantic errors. The failure compounds across every subsequent agent in the chain.
Silent failure. An agent fails in a way that produces no error signal. It returns an empty result, a malformed output, or a plausible-looking wrong answer. No agent in the system claims ownership of the failure. The final output is wrong and no one knows why.
One useful calibration: end-to-end reliability degrades quickly in chained workflows. If each step has independent success probability p, then a chain of n steps has expected success p^n. Each additional agent can multiply the failure surface.
Practical implementation
Start with two agents and one handoff.
Example: Agent A gathers requirements; Agent B generates a recommendation.
- Define Agent A scope: owns input, produces JSON requirements.
- Define Agent B scope: reads requirements, owns recommendation, must not modify requirements.
- Use a database table for requirements state, not prompt memory.
- Write the requirements to the database when A finishes; B reads from the database (not from A directly).
- Log every state transition: what was written, when, and by which agent.
- Test: what happens if B reads requirements before A commits? If B fails mid-recommendation?
- Deploy with circuit breaker: if B fails three times, pause and escalate to a human.
After one month running in production, extend to three agents. Repeat the charter-state-protocol definition for each new agent before writing code.
One thing that does not appear in architecture diagrams: cost. Three agents means three inference calls. If each agent calls a frontier model, the cost per workflow is three times the per-call cost, not counting retries. Budget for this before you deploy.
Risks and trade-offs
Multi-agent systems add real complexity: state management, coordination, observability infrastructure, and a harder testing surface.
The payoff is that problems scale horizontally, with more specialized agents on the same architecture, rather than requiring a single monolithic agent that must do everything. A monolithic agent is harder to debug, slower to improve, and has no clear failure boundaries.
The warning is this: do not add multi-agent complexity unless the task genuinely requires it. Sequential workflows and single agents with tool calls handle a larger share of production problems than the current hype suggests. Add agents when you need specialization, parallelism, or fault isolation, not because a framework makes it easy.
What to do this week
If you are designing a multi-agent system:
- Draw a diagram: boxes are agents, arrows are handoffs.
- Label each agent with its inputs, outputs, and state ownership.
- For each arrow, answer: what state is shared? How is it passed? What if the handoff fails?
- Identify failure cascade points: if this agent fails, what agents depend on it?
- Add a “failure modes” column to your diagram for each agent: stale read, conflicting write, partial completion, silent failure.
That is enough to start.
The real reason multi-agent systems fail
Multi-agent systems are not complex because of AI.
They are complex because they are distributed systems.
The failure modes, stale state, split ownership, cascade failure, and partial writes, predate LLMs by decades. What is new is that the failures are harder to reproduce (non-deterministic execution), harder to observe (reasoning is not logged by default), and harder to test (you cannot replay an LLM call and get the same output).
If you design a multi-agent system like a prompt pipeline, it will fail like a prompt pipeline. Design it like a distributed system, and you have a foundation that actually holds.
I work at Microsoft. The views expressed here are my own and based solely on publicly available information. This content is for educational purposes and does not represent official Microsoft guidance or commitments.