The Failure of Static Evaluation
In high-stakes domains like healthcare, AI ambient scribes often produce notes that are syntactically correct but contextually dangerous. Errors fall into three categories: additions, changes, and omissions. While blatant hallucinations are easy to catch, the most dangerous errors are subtle—such as omitting a red-flag symptom (e.g., jaw pain in a patient with a new headache, indicating giant cell arthritis) or misinterpreting a patient's intent.
Standard evaluation approaches—using frontier models with static rubrics or deterministic concept counting—fail because they lack "taste." They cannot distinguish between noise (e.g., a holiday in France) and a critical diagnosis (e.g., a trip to Lake Malawi). Because the definition of a "good" note is tacit, contextual, and constantly evolving, it cannot be fully captured in a static, pre-written rubric.
The Continuous Evaluation Loop
To move beyond the limitations of static checks, builders must treat evaluation as a continuous, dynamic process rather than a one-time engineering task. This involves a three-step loop:
- Discover: Do not guess failure modes on a whiteboard. Deploy the system, observe real-world outputs, and cluster the errors that actually occur. This creates a failure mode ontology derived from reality rather than imagination.
- Capture: Collect expert judgment on these real outputs. Instead of simple pass/fail scores, capture the reasoning and corrections from domain experts. This raw data is the "gold standard" that cannot be written down in advance.
- Calibrate: Instead of running a generic rubric, assemble a case-specific standard for every output. Use context engineering to pull in the most similar past cases, expert corrections, and relevant guidelines. This allows the judge to evaluate the current note against the specific, evolving standard of what matters for that context.
Why This Works
This approach shifts the burden from trying to write a perfect, all-encompassing prompt to building a system that learns from expert feedback. By using past judgments as context, the evaluation system becomes more accurate over time. This methodology is applicable beyond healthcare to any domain where being "confidently wrong" carries high costs, such as legal contract review or customer support automation. The goal is not to build a static safety net, but to create a system that evolves alongside the domain's shifting standards.