Most write-ups compare GraphRAG and vector RAG on answer quality. The question that actually decides whether you can ship it is cost, and the honest answer surprises people: GraphRAG's expense is not in the query. It's in the index — the step that runs before a single user asks anything, where an LLM reads every chunk of your corpus to build the graph. If you're deciding whether to turn GraphRAG on, this is the bill to understand first.

The takeaway in one screen:

This is the practical, numbers-first companion to GraphRAG vs Vector RAG: when a knowledge graph actually earns its cost and choosing a graph database for GraphRAG. Here's where every dollar goes, and how to cap each line before you commit.

Cost center #1: the index (this is the one that hurts)#

Vector RAG's index is boring and cheap: chunk the documents, run each chunk through an embedding model once, store the vectors. One pass, no LLM, linear in corpus size, and trivially incremental — when a document changes you re-embed only that document.

GraphRAG's index is a different animal. Following the pipeline in Microsoft's GraphRAG docs and the founding paper, building the graph means:

  1. Chunk the source documents.
  2. LLM-based extraction of entities, relationships, and claims — one or more LLM calls per chunk. This is the expensive step, and it runs across your entire corpus.
  3. Graph construction — consolidate the extracted elements into a single knowledge graph.
  4. Community detection with the Leiden algorithm, applied hierarchically to partition the graph into nested communities.
  5. Hierarchical community summarization — the LLM writes a "community report" for every community at every level, so you have summaries at multiple granularities.

Steps 2 and 5 are both LLM-heavy, and both scale with the amount of data. That is the whole cost story: GraphRAG converts a one-time embedding pass into thousands of LLM calls. The good news is that this makes the bill predictable. Your indexing cost is approximately:

index_cost ≈ (chunks × extraction_cost_per_chunk)
           + (communities × summary_cost_per_community)

Both terms are knowable before you run the full build. Take a representative sample of your corpus — say 1–2% — run just the extraction on it, measure the tokens, and multiply. You'll have your indexing bill to an order of magnitude before you spend the real money.

How to cap it:

The freshness tax people forget#

The index cost isn't a one-time charge if your data changes. Vector RAG stays fresh incrementally — new chunk, new embedding, upsert, done. A full GraphRAG re-index means re-extraction, re-clustering, and re-summarization, because a new entity can reshape the community structure the summaries were written against.

This is why the graph-database choice matters for changing data: real-time stores like Memgraph and FalkorDB exist partly to make updates cheap, and it's a core reason the graph-DB comparison is worth reading before you build. If your corpus updates hourly, budget for the re-index or pick an architecture that doesn't require one — batch your updates, or use a deferred/lazy approach that never builds the expensive static summaries in the first place.

Cost center #2: the query (cheap or expensive, your choice)#

GraphRAG's query cost is entirely a function of which retrieval mode you route to, per Microsoft's docs:

How to cap it: classify the incoming question and route it. Factoid and single-entity questions go to local search. Only true whole-corpus questions get global search, and those get a hard per-query token ceiling on the map-reduce step so one broad question can't run away with your budget. If you find yourself wanting global-quality answers on most queries, that's the signal to evaluate a lazy variant rather than paying full global cost every time.

The decision framework#

Don't ask "GraphRAG or vector RAG?" as a quality question. Ask it as a cost-justification question:

The mistake to avoid is turning full GraphRAG on for a workload whose questions are 90% local lookup — you pay the entire indexing bill to benefit the 10% of queries that need traversal, when hybrid retrieval would have served the 90% cheaply and reserved graph traversal for the rest. Price the two cost centers separately, size them against a real sample of your corpus, and only pay for the graph where the questions actually need it.