What is GraphRAG?

GraphRAG (also written graph RAG) is retrieval-augmented generation where the thing being retrieved from is a graph of entities and relationships, not a collection of text chunks ranked by vector similarity. The term was popularised by a 2024 Microsoft Research project of the same name, and has since become the general label for the approach.

Ordinary RAG answers which passages resemble this question. GraphRAG answers which records are connected to the thing this question is about, and how.

Why the distinction matters

Vector search is excellent when the answer sits in one place, worded much like the question. What does our refund policy say about damaged goods is that shape, and embeddings handle it well.

Most business questions are not that shape. Is this customer worth escalating has no passage anywhere. The answer assembles from a support thread, a CRM record, an invoice and a renewal date — four systems that share no vocabulary and never mention each other. Nothing in the ledger is worded like the question.

Faced with that, similarity search returns the support thread, because a support thread sounds the most like a customer complaint. The model then answers from a quarter of the evidence and sounds completely certain, which is the worst failure mode available: partly grounded, fluent, and wrong.

The missing primitive is identity. Knowing that sam@bigco.com in the mailbox, "Sam Lee" on the ticket and contact 0012f in the CRM are one person is not something similarity can establish — two records about the same customer may share no words at all, and two records sharing many words may be about different customers entirely.

How it works

Extraction. Entities and relationships are pulled out of source records: people, companies, invoices, tickets, commitments, and the edges between them. This can be done by a model, by rules, or by reading the structure a system already has — a CRM already knows a contact belongs to an account.

Resolution. The same real-world thing, named differently in five systems, is collapsed to one node. This is the step that does the actual work, and the step most implementations underestimate.

Traversal. A question is answered by walking edges from the entities it mentions, gathering the connected records, and handing those to the model. Some systems also pre-compute summaries over clusters of the graph so that broad questions — what are the themes in our support volume — have something to read.

Generation. The model answers from the subgraph, ideally citing each record it used.

In practice most production systems are hybrid: the graph establishes which entities and records are in scope, and vector or keyword search ranks the text within that scope. Neither primitive is sufficient alone.

What it costs

GraphRAG is more expensive and more opinionated than dropping documents into a vector store.

• Extraction is work, and it is ongoing. Every new record has to be read and placed.
• Resolution can be wrong, and when it is, the errors are worse than retrieval errors. A merged pair of customers produces confidently wrong numbers across every downstream answer, and no spot-check catches it.
• Schema is a commitment. Deciding what counts as an entity, and which relationships matter, is a modelling decision you will live with.
• Freshness is harder. A changed record may invalidate edges elsewhere.

Those costs buy one thing: the ability to answer relational questions correctly. If your questions are genuinely document-shaped, you do not need a graph and should not build one.

GraphRAG and time

A graph records how things relate. It does not, by default, record when each relationship was believed — and for anything auditable that gap matters. A system that overwrites a renewal date can tell you today's date and nothing about what it told somebody in July.

Storing a validity interval on each edge, rather than just the current value, is what makes reading memory as of a date possible: following how a fact changed, and distinguishing "the model was wrong" from "the facts changed since". That is the difference between a graph you can query and a record you can defend.