The Provenance Problem in LLM Pipelines

LLMs are effective at synthesizing information from multiple sources, but this process is inherently lossy and non-deterministic. When an agent generates a "fact"—such as a patient's allergy—it often merges data from disparate sources (e.g., EHR records, lab reports, and chat logs). Because the LLM synthesizes this output, the direct link to the original source is lost. Standard approaches like appending a source ID to a record fail because LLMs mutate, merge, and invalidate facts over time, rendering static pointers obsolete.

Provenance as a Graph Walk

To solve this, provenance must be engineered into the data structure itself rather than logged as an afterthought. The Graphiti framework models provenance as a temporal knowledge graph.

  • Nodes and Edges: Source data are stored as "episodes" (nodes). Derived facts are represented as triples (subject-verb-object) connected to these episodes via edges. Tracing a fact to its origin becomes a simple graph traversal.
  • Metadata Projection: By tagging a source episode once, all derived facts and entities inherit that tag. This allows agents to filter for veracity—for example, only retrieving facts derived from verified clinical sources.
  • Handling Contradictions: When new data invalidates an existing fact, the system does not simply delete the old record. Instead, it marks the edge as invalid and notes the new source that triggered the change, preserving the audit trail.

Compliance and Debuggability

Modeling lineage as a graph provides three critical engineering advantages:

  1. Granular Deletion: In compliance scenarios (e.g., GDPR "right to be forgotten"), the graph structure allows for precise deletion. If a fact is supported by three sources and one is deleted, the fact remains if the other two sources still support it. If no sources remain, the fact is automatically pruned.
  2. Veracity Filtering: Agents can apply business logic to verify facts based on their parentage. For high-stakes decisions like medical prescriptions, an agent can be programmed to require verification from every parent source, whereas less critical data might only require a single verified parent.
  3. Debuggability: Because every fact is linked to its source episodes, developers can trace exactly why an agent reached a specific conclusion, transforming the LLM from a "black box" into an auditable system.

While graph construction is computationally expensive, the team behind Graphiti uses a mix of LLMs for extraction and traditional NLP/information retrieval techniques (like sim-hash and entropy analysis) for deduplication to keep costs and latency manageable.