The Failure of Multi-Agent Mimicry

ZS Associates initially built a multi-agent system designed to mirror the four-step workflow of a human pharma analyst: signal detection, source localization, driver attribution, and synthesis. The system failed because it lacked a unified "owner" of the reasoning process. While individual agents could identify correct facts, the handoffs between them resulted in incoherent outputs—such as identifying a payer-related issue but recommending a sales-rep-focused action. The core issues were context loss during handoffs, the misuse of LLMs for deterministic data tasks, and a lack of shared domain knowledge.

Rebuilding via Observational Engineering

Instead of theoretical redesign, the team observed how Claude Code operated in an empty directory with only bash and database access. This revealed that the system should be smaller, not larger. They implemented three critical architectural shifts:

  • Deterministic Pre-processing: Signal detection is now handled by a deterministic pipeline using statistical methods, thresholds, and guardrails. The agent only wakes up once a signal is queued, shifting its role from "guessing" to "investigating."
  • Consolidated Reasoning: The team replaced the multi-agent orchestrator with a single agent that owns the end-to-end reasoning. This agent dynamically spawns sub-agents only for focused, isolated tasks (e.g., specific data lookups) while retaining control over the final judgment.
  • Knowledge Graph as a Control Plane: Rather than using a knowledge graph as a passive lookup table, it serves as a control plane that bounds the agent's search space. Every edge in the graph represents a hypothesis. The agent traverses the graph, evaluates hypotheses against raw data, and iterates until the root cause is identified.

Key Takeaways for AI Engineering

  • Don't mimic human org charts: Avoid mapping agent topologies to human workflows. Let the architecture be derived from the actual requirements of the data and reasoning tasks.
  • Separate deterministic and agentic work: Never let an agent perform tasks that can be solved with deterministic code or statistical methods.
  • Centralize reasoning: A single agent must own the end-to-end reasoning to maintain coherence. Sub-agents should be used only for delegated, focused execution.
  • Control planes over lookup tables: Use structured knowledge to guide agent navigation, ensuring the model stays within the bounds of domain-valid hypotheses.