I expected specialist agents to help on complex tasks.

On the architecture diagram, the split looked natural. I built an orchestrator that delegated product discovery—search and recommendations—to one specialist, and support FAQs to another. Each specialist would own a smaller problem and use a model suited to its role. The orchestrator would bring their answers together. The result looked cleaner and, at least in theory, more capable.

I expected the distributed version to win. Then I ran the evaluation.

It did not reward that intuition.

Across 1,500 e-commerce cases, a distributed agent-to-agent architecture and a single agent with direct access to the same tools produced comparable aggregate quality. The distributed path scored similarly to the monolithic path, but took longer on average than the direct-tool agent.

In other words, delegation did not create a meaningful quality advantage in this setting. It did create another 21.7 seconds of latency.

That does not mean distributed agents are a bad architecture. The evaluation covered two specialist roles within one e-commerce domain and one shared tool surface. Both specialists ran in the same environment. It did not test the independent teams, trust boundaries, deployments, or state ownership that can make distribution valuable in production.

It did clarify the question. The choice is not about whether several specialists look cleaner on a diagram. It is about whether the boundary between them earns the cost of coordination.

An old architecture question in new clothes

The choice between one agent and several resembles the choice between a monolith and microservices.

A software monolith is not necessarily one large, tangled block of code. It can have clear modules, typed interfaces, isolated domains, and strong internal boundaries. What makes it monolithic is primarily its runtime and deployment topology: its parts operate as one system.

The same distinction matters for agents. A monolithic agent does not have to mean one giant prompt handling everything. It can use modular instructions, policies, evaluators, and many well-designed tools. One coordinating agent still owns the interaction and calls those tools directly. Splitting its instructions across several files does not change that.

A distributed agent architecture introduces independently executing specialists. A front-door agent decides where work should go, delegates it, waits for another agent to plan or act, then interprets and synthesizes the result.

That boundary creates options, but it also creates work:

• another routing decision;
• another prompt and context handoff;
• another process or network hop;
• another contract to version;
• another component to observe and debug;
• another place where a failed subtask can hide behind a plausible final answer.

This is where the microservices analogy helps. Both choices trade lower local coupling for higher system-level coordination. Distribution makes it possible for domains to be owned, deployed, secured, and scaled independently.

The analogy has its limits though. A conventional service call normally follows a deterministic contract and often fails explicitly. Agent delegation adds probabilistic routing, lossy context transfer, and synthesis. A specialist can misunderstand the task or retrieve the wrong evidence while still returning a fluent answer. The error may never reach the orchestrator as an error; it arrives as a plausible result. That is why the delegation contract should preserve observable evidence, relevant actions, and uncertainty rather than returning only a polished answer.

Distributed agents therefore need at least as strong a reason to exist as distributed services, arguably a stronger one.

Maybe you don’t need another agent. Maybe you need a tool.

Before introducing another agent, ask whether the other side of the boundary needs to make decisions or simply execute a capability.

A tool executes a capability on request. It searches a catalog, queries inventory, retrieves a document, calculates a price, updates a record, or calls an API. The main agent remains responsible for the workflow and decides how to use the result.

A specialist agent owns a meaningful part of the workflow. It may plan several steps, choose among its own tools, maintain state, enforce domain-specific policy, or complete a long-running task. It returns a domain result rather than exposing every low-level operation to the orchestrator.

A tool is the hammer. An agent is the craftsperson: it decides which tool to use, in what order, and whether the work is complete. Sometimes the architecture does not need another craftsperson. It needs a better hammer.

The distinction is not whether the component uses an LLM. The distinction is responsibility.

Imagine removing the model from the proposed specialist. If what remains is a thin wrapper around an API or data source, it was probably a tool all along. Calling it an agent adds orchestration without creating a useful layer. Exposing the capability as an MCP tool would have been more efficient.

A genuine agent boundary is easier to recognize. “Check fulfillment feasibility” can be a durable capability owned by a logistics domain. It may combine inventory, warehouse constraints, shipping restrictions, and delivery promises without making the front-door agent understand each system. “Search the product index,” by contrast, is usually a tool call.

What the evaluation showed

The evaluation compared two paths over the same e-commerce tool surface: search, recommendations, and support FAQs.

In the monolithic path, one agent selected and called those tools directly. In the distributed path, an orchestrator delegated product discovery—search and recommendations—to one specialist and support FAQs to another, then synthesized their responses. The cases and underlying capabilities remained the same. What changed was who called the tools and how many coordination steps stood between the request and the final answer.

Both paths expose the same capabilities. Only the caller topology changes: direct tool access on the left, delegated specialist calls on the right.

The original evaluation leaderboard, reduced to the two architectures discussed here. It reports its own weighted overall score and median latency.

The comparison is easier to read across three axes: quality, speed, and price. Quality barely moved. Latency more than doubled. Cost went down, even though the distributed path used more tokens.

The distributed path took about 2.5 times as long. The reason was structural: the orchestrator had to decide to delegate, hand work to a specialist, wait for it to search, receive its answer, and synthesize the result. The direct-tool agent could move from tool output to final response without that relay.

The distributed path also showed a wider spread of outcomes. Multiple specialist searches sometimes produced broader answers that the direct path missed. But each additional handoff created another failure surface. When the distributed system failed for genuine architectural reasons, the problem was generally not the orchestrator's final reasoning. A specialist had retrieved the wrong evidence or returned no useful evidence at all.

This matters because a fluent synthesis cannot recover information that a specialist failed to retrieve.

The cost result adds an important nuance. The distributed run processed about 1.7 times as many tokens, yet cost $41 less because retrieval-heavy specialist work ran on a less expensive model while the stronger model remained responsible for orchestration and synthesis. Distribution made that model-tier split natural. Another evaluation confirmed that using the smaller model for the monolithic agent reduced quality. The smaller model was useful in the distributed architecture because the orchestrator gave it a narrower, well-defined task.

But topology alone did not create the saving. The saving came from the particular assignment of models to roles. When evaluating cost, we should compare complete configurations rather than assume that more agents are inherently cheaper or more expensive.

A faster distributed configuration reduced cost and latency further, but it also retrieved fewer of the right products. Its final answers were concise and grounded in what it found; the problem was that it had not found enough. The optimization improved visible efficiency while weakening upstream retrieval.

The lesson is not that the monolith won a universal contest. It is narrower and more useful: when one team owns one domain and both architectures expose the same tools, delegation must prove that it adds more than latency and failure surfaces.

When a boundary earns its overhead

Domain separation alone is not enough. Most systems can be described as a collection of domains. The useful question is what separating a domain allows it to do independently.

Independent ownership

Distribution becomes valuable when different teams genuinely own different business capabilities.

A billing team may need to evolve refund policy without coordinating every change with the team responsible for the customer-facing agent. An inventory team may need its own release cadence, evaluation suite, and incident response. In that situation, the agent boundary reflects an existing organizational boundary rather than inventing one.

The specialist contract lets the front-door agent ask for an outcome while the owning team remains free to change the implementation.

Deployment, security, or trust boundaries

A specialist may live in another environment, use different credentials, or operate over data that should not be copied into a central agent's context.

For example, a human-resources specialist could keep employee data and privileged tools inside its own security boundary. The orchestrator receives only the approved result. Here, distribution is not primarily about better reasoning. It preserves a trust boundary that the system already needs.

Context is precious and expensive

There is another reason to delegate work, even when the subagent is not owned by another team: context isolation.

An agent's context is its working memory, not an unlimited database. Every search result, tool response, log, document, and intermediate finding competes with the original goal, earlier decisions, and user constraints. During a long-running task, that context accumulates. Once the window fills, older information may need to be compressed or dropped. Even before reaching that limit, a growing volume of irrelevant detail can make the important information harder to use consistently.

A subagent can absorb that detail inside a separate context.

Suppose the main agent is coordinating a long investigation. A subagent can analyze a large volume of logs, test several hypotheses, and trace the most likely cause. The main agent does not need every log line or every dead end. It needs the finding, the evidence that supports it, and what remains uncertain.

That is the useful pattern: keep the goal, user constraints, and cross-task decisions in the main agent's context; move bounded, evidence-heavy investigations into subagents; then return a compact result. Used this way, delegation is not only domain decomposition. It is context management. It prevents one noisy subtask from consuming the working memory needed to complete the larger task, and it helps preserve earlier information over longer runs.

The benefit is of course not automatic. A handoff can omit a constraint, and a summary can erase a weak signal that later becomes important. The main agent must give the subagent enough context to perform the task, and the subagent should return evidence and uncertainty rather than a conclusion with no traceability.

The common thread in all those examples is independence. Distribute when a domain needs independent ownership, deployment, security, state, context, or operations—not automatically. Until then, the monolithic path is not the unsophisticated choice. It is the architecture with the lowest coordination cost.

A compact decision guide

Conclusion

Start with a monolithic agent. Move to distributed agents only when the domain boundary is valuable enough to pay for delegation.

This is the same architectural trade-off we already know from monoliths and microservices. Splitting a system into smaller pieces does not create value by itself. It creates coordination costs. The value is not in the boxes; it is in the independence the separation preserves: ownership, deployment, contracts, security boundaries, state, and operating models.

For agents, the question is therefore not “Would several specialists look cleaner?” It is:

What valuable separation does another agent preserve?

If there is no convincing answer, keep one agent and give it the tools it needs.