GraphRAG explained: when knowledge graphs beat vector search

GraphRAG is retrieval-augmented generation where the retrieval step walks a knowledge graph of entities and their relationships instead of, or as well as, searching for similar passages. Vector search answers "find me text about X". GraphRAG answers "what is connected to X, through what, and what does that imply". For a business, the decision is not which is better in general but which questions your users actually ask. This article explains how GraphRAG works, the questions it answers that vector search cannot, what it costs to build and keep current, and how to decide whether you need it.

Why similarity search cannot answer relationship questions

Vector retrieval finds passages whose meaning resembles the question. Ask "which suppliers would be affected if component C-118 is discontinued?" and the retriever returns passages that mention C-118, or suppliers, or discontinuation. The answer, though, is not in any one passage. It requires knowing that C-118 is used in three products, that those products are built by two contract manufacturers, that one manufacturer sources C-118 through a distributor, and that the distributor's contract has a notice clause. That chain lives across a bill of materials, two contracts and a supplier register. No chunk contains it; the connections do. Similarity search retrieves fragments and leaves the model to guess the joins, which it does confidently and often wrongly.

How knowledge graph RAG works

GraphRAG versus vector search: which questions each wins

Vector search wins

• "What is our policy on X?" where the answer is in one passage
• "How do I configure Y?" from documentation
• Questions phrased differently from the source text
• Corpora that change constantly, where graph extraction cannot keep up

GraphRAG wins

• "Which contracts reference supplier S and what are their notice periods?"
• "Who reports to whom in the division after the reorganisation?"
• "What is the dependency chain from this component to finished goods?"
• "Summarise the main themes across this year's customer complaints", which community summaries answer and passage retrieval cannot
• "Which patients on drug D also have condition C?", where entities and relations are explicit

Most production systems use both

A hybrid of hybrid search and graph retrieval is the usual shape: passage retrieval for direct questions, graph traversal when the question mentions entities and relationships, with a router deciding per query. The passage side is described in Hybrid search; the graph side is added only for the question types that need it.

Entity-based retrieval: the fragile step

Graph retrieval starts by working out which entities the question refers to. "C-118" is easy; "the distributor we switched to last year" is not. Entity linking has to handle aliases (a company's legal name, brand name and abbreviation), typos, and references that only make sense with context. Extraction has the same problem in reverse: the model must recognise that "Acme Ltd", "ACME" and "Acme Limited" are one node. Every GraphRAG project spends real effort on entity resolution, and the domains where GraphRAG pays off are usually the ones where master data already exists (a supplier register, an org chart, a product catalogue) and can anchor the graph rather than leaving it entirely to extraction.

What GraphRAG costs that vector search does not

Extraction runs a model over every document, which costs more than embedding it and must be re-run when documents change. The schema of entity and relationship types needs design and governance. Extraction errors accumulate into a graph that confidently asserts wrong relationships, and those are harder to spot than a missed passage. Community summaries need regeneration as the graph grows. A graph database such as Neo4j or a graph layer over Postgres adds an operational component. None of this is prohibitive, but it means GraphRAG is a project rather than a feature flag, and the retrieval and knowledge engineering service scopes it separately from a standard RAG build.

Evaluating GraphRAG

The golden set has to include multi-hop questions, ones whose answer requires two or more relationships, with the expected entities and passages recorded. Score entity linking (were the right starting nodes found), traversal (were the right connected passages retrieved) and the answer's groundedness in that subgraph. A GraphRAG system that scores well on single-passage questions and poorly on multi-hop questions has a graph that is not being used, which happens when the router sends everything to vector search. The evaluation approach is the same one described in How to measure RAG quality, with the multi-hop set added. Microsoft's GraphRAG project documents the community-summary approach and is the primary reference for the technique.

Keeping the graph current

Documents change, and so do relationships. A contract is renewed with a new notice period; a person moves teams; a component is replaced. Incremental extraction on changed documents, with edges carrying the version and date of the passage they came from, keeps the graph honest. Answers should cite the passage and its date, exactly as in passage RAG, so a user can see that the notice period comes from the 2025 renewal and not the original agreement. Graphs that are built once and never refreshed produce authoritative-sounding answers about a world that no longer exists.

Permissions apply to graphs too. If a user may not see a contract, they may not see the edges extracted from it, and traversal must filter by entitlement at query time rather than leaking a relationship whose source they cannot open. The design is the same as for passages and is covered in permission-aware retrieval.

A worked example

A university modernising its administrative systems needed staff to answer questions like "which courses does this lecturer teach, which programmes do those courses belong to, and which accreditation requirements apply?" The data lived in course handbooks, an HR system, programme specifications and accreditation documents, and a vector-only assistant returned relevant-looking fragments without the chain. The rebuild anchored a graph on the existing course catalogue and staff directory as master data, extracted relationships from the handbooks and specifications with a small schema (teaches, belongs to, requires, accredited by), and routed relationship questions to graph traversal while keeping passage retrieval for "what does the policy say" questions. Answers cited both the traversal path and the source passages. The broader modernisation is described in the university ERP case study; the graph was one component that let the assistant answer questions the old search never could.

Team and timeline

GraphRAG needs a retrieval-focused AI engineer, a data engineer for extraction pipelines and refresh, and a domain expert on your side to define the schema and check extraction quality on a sample. Six to twelve weeks on top of, or alongside, a standard RAG build, depending on corpus size and the availability of master data to anchor the graph. It is scoped within the retrieval and knowledge engineering service, from $14,000 / ₹8.8L, and a three-week ProofRun from $6,250 is the right way to find out whether your question mix justifies it: build the golden set, run vector-only, and see how many questions fail for structural reasons. Where the graph feeds agents that act on the relationships, the multi-agent systems service takes over. Prices are on the pricing page.

Before you start: a checklist

• Collect real questions and mark which need more than one relationship to answer
• Identify master data (registers, catalogues, org charts) that can anchor entities
• Draft a schema of entity and relationship types, kept small
• Decide the graph store and how it links back to source passages
• Plan incremental extraction on document change, with dates on edges
• Build a multi-hop golden set alongside the single-passage one
• Decide how the router chooses between graph and passage retrieval
• Name the domain expert who reviews extraction samples

Glossary

• Knowledge graph: entities as nodes and relationships as edges, each linked to source text
• Entity extraction: pulling named things (people, products, clauses) out of documents
• Entity linking: matching a mention in a question or document to the right node, across aliases
• Multi-hop question: one whose answer requires following two or more relationships
• Community summary: a generated summary of a closely connected group of entities, used for broad questions
• Traversal: walking the graph from starting entities along edges to collect connected passages
• Router: the component that decides whether a query goes to graph or passage retrieval

Related reading

What is RAG? for the foundation, Why basic RAG fails in production for the fixes that come before a graph, and the pricing page.

Build the graph when your users' questions are about connections, anchor it on master data you already trust, and keep passage retrieval for everything else.