Maintaining LLM Knowledge Graph Provenance via Temporal Graphs
AI Engineergo watch the original →
the gist
To solve the loss of source lineage during LLM synthesis, model provenance as a graph where facts are nodes linked to source 'episodes', allowing for verifiable audits, selective deletion, and lineage-aware fact invalidation.
Provenance as a Graph Structure
LLM-driven synthesis is inherently lossy and non-deterministic, which destroys the paper trail required for compliance and debugging. Storing a simple source ID on a fact is insufficient because LLMs frequently merge entities and synthesize facts from multiple, shifting sources. Instead of logging provenance after the fact, it must be engineered into the data structure itself. By modeling provenance as a knowledge graph, every derived fact is linked to its source 'episodes'. Tracing a fact to its origin becomes a simple graph walk, and metadata projection allows tags applied to a source to propagate automatically to all derived nodes and edges.
Lineage-Aware Mutation and Deletion
Maintaining graph integrity requires handling data evolution, such as entity merging and fact invalidation. When new data contradicts existing facts, the system adds an invalidation date to the mutated edge while retaining the original lineage. This structure also enables precise GDPR compliance and retention management. If a source episode is deleted, the system checks the graph to see if any other sources support the derived fact. If at least one supporting source remains, the fact persists; otherwise, it is removed. This ensures that agents only operate on facts with verified, existing provenance.
Engineering Implementation
Graphiti, the framework behind Zep, handles these requirements by treating episodes as first-class nodes in the graph. The ingestion pipeline performs structured extraction of entities, relationships, and candidate facts. To manage costs and latency, the system prioritizes traditional NLP and information retrieval techniques—such as sim-hash and entropy analysis—for deduplication and deconfliction, reserving LLM calls for the initial extraction and reflection steps. This approach allows agents to filter context based on specific veracity tags, such as ensuring a medical prescription is only issued if all source episodes for an allergy flag are verified.