Causal Knowledge Graphs vs. Traditional RAG

Exploring the limitations of vector search and why the future of AI memory lies in structured causal graphs.

Retrieval-Augmented Generation is, at this point, the standard approach for giving AI systems access to external knowledge. You store documents as vector embeddings. When a query arrives, you find the most semantically similar chunks. You inject them into the prompt. The model answers.

It works. For document retrieval, for question answering over large corpora, for chatbots that need to reference a knowledge base — RAG is a reasonable solution to a real problem.

It is not, however, a memory system. And the difference matters more than most people building agents currently appreciate.

Vector search finds similarity. That is its strength and its fundamental limitation.

When you embed a sentence and store it, you are compressing its meaning into a point in high-dimensional space. Sentences with similar meanings end up near each other. When a query arrives, you find the nearest points and retrieve the associated text.

This works well when the query and the relevant information are semantically similar. It works poorly when the relationship between query and relevant information is logical rather than semantic.

Consider: you are building an agent for a software developer. Over months of use, the agent has learned the following:

Now the user asks: "Should I use a closure or a class for this state management problem?"

A RAG system will retrieve the chunks most semantically similar to "closure class state management." That might return the facts about functional programming preferences and the switch to Rust. Or it might not, depending on how the embeddings clustered. The retrieval is probabilistic and depends on surface similarity.

A causal knowledge graph asks a different question: given what I know about this person's preferences, what does the graph imply about this decision? Functional programming preference implies closures. Performance sensitivity implies avoiding unnecessary allocation. The Rust context implies familiarity with ownership semantics that makes closures natural. These facts are not retrieved independently — they are traversed as a connected structure that produces a conclusion.

The difference is between retrieving relevant facts and reasoning from connected knowledge.

RAG systems have a provenance problem that becomes serious at scale.

When you retrieve five document chunks to answer a question, those chunks were written independently. They may agree. They may contradict each other. They may be true in different contexts that the retrieval system cannot distinguish. The model receives them as a flat list and has to reconcile them during generation — which it does imperfectly, because reconciliation during generation is much harder than reconciliation during storage.

In a knowledge graph, contradictions are first-class objects. When VYN's contradiction detector identifies that two stored observations are in tension — "user prefers minimal dependencies" versus "user has 47 packages in their requirements.txt" — it does not discard one silently. It stores both, marks them as contradictory, and includes the contradiction in context when relevant. The model receives not just the facts but the known tension between them.

This changes the quality of reasoning substantially. A model that knows "these two things are in tension" reasons differently than one that receives both facts as equally valid and has to figure out the contradiction on its own under generation pressure.

Standard RAG systems accumulate without consolidating.

Every document you add is a new chunk in the vector store. The store grows. Retrieval becomes noisier as the corpus expands because there are more approximately-relevant chunks competing for the top-k slots. Older information does not decay. Superseded information stays in the store even after more current information has been added.

For agent memory specifically, this is a serious problem. An agent used daily for a year might have tens of thousands of stored observations. A vector store of that size, without consolidation, will surface irrelevant old observations in almost every retrieval. The signal-to-noise ratio degrades over time.

VYN's memory system includes a consolidation cycle that runs weekly. It clusters semantically similar memories using the vector store, then uses the model to synthesize clusters into abstract principles. "User prefers React," "User uses Next.js," "User mentioned building a component library" get consolidated into "User works primarily in the React ecosystem with a preference for Next.js for full-stack applications." The three specific memories are archived. One abstract principle takes their place.

This is not a novel idea — it is roughly analogous to how memory consolidation is thought to work during human sleep. Specific episodic memories are abstracted into semantic knowledge. The specific experience fades; the lesson remains.

The difference is that in a vector store, nothing fades. Everything accumulates with equal weight. You need to build the consolidation mechanism explicitly, or your memory system eventually becomes a noise generator.

I am not arguing that knowledge graphs are strictly superior to RAG. They are not. The tradeoffs are real.

Knowledge graphs require structure. When information arrives, it has to be parsed into entities and relationships. That parsing is done by the model, which means it is inconsistent. The model will sometimes identify a causal relationship where none exists, or miss one that is obvious. The graph accumulates noise as well as signal, and the noise in a graph is harder to filter than the noise in a vector store, because graph noise creates false edges that produce false inferences.

Entity resolution is genuinely hard. If the user mentions "React" in one conversation and "React.js" in another, those should be the same node. Determining when two mentions refer to the same entity is a non-trivial problem that vector stores largely sidestep by working at the chunk level.

And knowledge graphs are expensive to maintain. Every new observation requires checking for contradictions with existing nodes, resolving entity references, placing the new node in the correct position in the graph topology, and potentially triggering hypothesis updates. This is much more computation than appending a vector to a store.

For most current applications, RAG is the correct choice. It is simpler, more scalable, and well-understood. The tooling is mature. The failure modes are known.

For agent memory specifically — systems that are supposed to learn about a person or domain over months and years, accumulate understanding rather than just information, and reason from what they know rather than just retrieve it — knowledge graphs address limitations that RAG cannot.

The architecture we are building in VYN is a hybrid. Vector search handles retrieval at the turn level — finding the semantically relevant memories for the current query. The knowledge graph handles reasoning at the session level — understanding what the agent knows as a connected structure rather than a collection of independent facts. Consolidation handles the long-term problem — keeping the store from becoming noise-dominated over time.

None of these components is sufficient alone. The vector store without the graph retrieves without reasoning. The graph without the vector store misses semantically similar but structurally disconnected memories. The consolidation without both produces abstract principles with no grounding.

The hard part of building intelligent memory is not picking the right data structure. It is understanding why each component fails and designing the combination that compensates for those failures.

We are still learning that. The system described here is better than what we had six months ago. It is not what we will have in another six months.

That is, I think, the right relationship to have with a hard problem.