The Failure of the 'Bolt-On' Approach

Most enterprise AI proofs-of-concept (POCs) succeed in isolation but fail during productionization because they treat compliance, auditability, and security as secondary features to be added later. This results in brittle systems that cannot meet the rigorous standards of regulated industries like healthcare or finance. The authors argue that engineers must prioritize these constraints as the foundational architectural principles rather than attempting to retrofit them onto a high-performing demo.

Core Architectural Primitives

To build production-ready agents, the authors propose three foundational primitives that shift compliance from a manual burden to an inherent system property:

• Immutable Append-Only Event Logs: Instead of relying on standard developer logs, implement a transaction-style event log that records every agent action, data access, and authorization. This creates a durable, timestamped chain of evidence. While this makes reads more complex (requiring snapshots or projections), it makes auditability trivial and allows for perfect system state reconstruction.
• Schema-Driven Object Storage: Keep sensitive data (like PHI) outside the event log. The log should only contain references to immutable blobs in object storage. This separation allows developers to debug agent logic and trace execution paths without ever being exposed to the underlying sensitive data.
• Human-Agent Parity: Treat LLMs and humans as the same class of 'agent.' By defining a shared interface for actions and context, the system can seamlessly escalate tasks from an AI to a human. Because the downstream processes do not distinguish between the two, the system remains robust regardless of who (or what) performs the task.

Evaluation as a Byproduct

When these primitives are implemented, evaluation ceases to be a separate, difficult task and becomes a natural byproduct of the architecture.

• Replayability: Because the system is built on an immutable ledger, engineers can replay production events with modified prompts or models to see the exact impact of changes.
• Comparative Evals: Human-agent parity allows for direct performance comparisons between human decisions and AI outputs, providing a high-quality baseline for scoring.
• Privacy-Preserving Production Evals: By running evaluations against the object storage within the customer’s own environment, teams can validate agent performance on real production data without violating data residency or privacy requirements.