The Architecture-Eval Feedback Loop

AI development is currently defined by "replatforming" rather than simple iteration. As models undergo step-function improvements in tool-calling, context handling, and reasoning, the systems built to work around their previous limitations become technical debt.

Evaluation strategies must remain congruent with the current architecture:

  • Single Prompt Era: Focused on final answer quality (accuracy/hallucination) using golden datasets.
  • Chain Era: Introduced multiple failure points (parsing, retrieval, context stuffing), requiring node-level evaluation.
  • Graph Era: Used rigid orchestration or state machines to force reliability when models were brittle. This created new failure surfaces, including branch logic errors and node-to-node contract failures.
  • Modern Loop Era: With models now capable of reliable planning and self-correction, systems are returning to flexible loops. However, this introduces high variance, where the same input yields different valid trajectories.

Shifting from Samples to Distributions

When a system is reliable enough to run in a loop, a single evaluation result is no longer sufficient. You must measure the distribution of outcomes to distinguish between capability and reliability:

  • Pass at k (Capability): Measures if the system succeeds at least once across k attempts.
  • Pass rate at k (Reliability): Measures how many of those k attempts succeed.

A system might show high capability but low reliability. By running the distribution, you identify whether the system is failing due to inherent model limitations or brittle orchestration logic.

The Production Flywheel

Static evaluations eventually go stale. To maintain a production-grade system, you must implement a flywheel that harvests production data to inform new test cases. This is critical for two reasons:

  1. Regression Testing: Ensuring that as you adopt new models, you do not break existing functionality.
  2. Discovery: Using cluster analysis on production logs to identify novel failure modes you did not anticipate.

Teams that treat evals as a durable asset—rather than a one-time setup—are better positioned to perform the "complex surgery" required to integrate new model capabilities without sacrificing system stability.