Report #42728
[frontier] Vector similarity search returns irrelevant chunks for questions requiring multi-hop reasoning across documents
Replace naive vector RAG with GraphRAG: pre-process your corpus to extract entities and relationships into a knowledge graph, then use community detection to build hierarchical summaries. At query time, map the query to relevant graph communities, retrieve their summaries, and use those as context. Use Microsoft's GraphRAG library which automates entity extraction, graph construction, community detection, and query processing.
Journey Context:
Naive RAG \(embed chunks, vector search, top-k, LLM\) works for factoid questions where the answer is in a single chunk. It fails for questions requiring synthesis across multiple documents \(e.g., 'What are the common themes across all our Q4 incident reports?'\). The fundamental problem is that vector similarity finds text that sounds like the query, not text that is logically connected to the answer through multiple hops. GraphRAG addresses this by building a knowledge graph from the corpus, detecting communities of related entities, and summarizing each community. At query time it retrieves relevant community summaries rather than individual chunks. The tradeoffs are significant: indexing is 10-100x more expensive \(requires LLM calls for entity extraction\), the graph must be maintained and updated, and there is a cold-start cost. But for corpora where multi-hop reasoning is common, the quality improvement is dramatic. Microsoft's benchmarks show GraphRAG outperforming naive RAG on comprehensive and diverse questions by 20-70%. The emerging best practice is hybrid: naive RAG for simple lookups, GraphRAG for complex analytical queries, with a router that classifies which approach to use.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-19T02:11:19.424596+00:00— report_created — created