The Failure of Naive Routing

Model routing is currently in a nascent, experimental phase. The panelists agree that the industry is moving toward a multi-model world, but warn against "naive" routing—the practice of simply assigning tasks to models based on benchmark performance. Alex Atallah (OpenRouter) highlights that small models often "thrash" when pushed outside their training distribution, leading to infinite loops and tool-calling errors that eventually cost more than a single, high-quality inference from a frontier model. Walden Yan (Cognition) notes that agentic tasks are inherently dynamic; a session might begin as a simple codebase query and evolve into complex live debugging. A model chosen for the initial task may become a bottleneck, forcing the user to switch models mid-stream and losing the efficiency gains.

The "Sidekick" Architecture vs. Sub-Agents

Cognition’s approach, implemented in their "Fusion" system, avoids the traditional sub-agent pattern. Instead of spinning up multiple ephemeral agents, they utilize a "sidekick" model that maintains a continuous running context. This architectural choice is critical for cost-efficiency: by keeping the KV cache warm, the system avoids the redundant token costs associated with re-prompting or re-loading context. Yan argues that context compaction should be viewed as an intelligence-preserving strategy rather than just a cost-saving one, as model quality degrades significantly when the context window is forced to truncate or miss cache hits.

Orchestration and In-Distribution Performance

There is a consensus that the "orchestrator" model—the one responsible for planning and delegation—should ideally be a frontier model. While smaller models are highly efficient for in-distribution tasks (e.g., simple text classification), they lack the general intelligence required to recognize when a task has exceeded their capability. A frontier model acting as the "wrapper" or "planner" can monitor the implementation process, recognize failure modes, and intervene. The panelists suggest that future research should focus on training models specifically for collaboration, using Reinforcement Learning (RL) to teach models how to delegate effectively rather than just how to solve tasks end-to-end.

The Economics of Cache and Collaboration

OpenRouter’s experience with their auto-router reveals that routing systems often sit idle until a specific application pattern emerges—such as a heartbeat-driven agent that requires two distinct intelligence profiles. The panel emphasizes that the future of routing lies in co-designing models with orchestration in mind. By sharing context intelligently and utilizing cache-efficient architectures, developers can achieve "Fable-level" intelligence at a fraction of the cost, often by allowing cheaper models to explore codebases with greater intensity than a single, expensive model could afford to do.