Report #48274
[frontier] Vector RAG fails on questions requiring synthesis across many documents — broad thematic questions return fragmented irrelevant chunks
Replace vector-only RAG with Graph RAG for knowledge bases requiring synthesis: extract entities and relationships from documents to build a knowledge graph, detect communities using graph algorithms \(Leiden\), generate community-level summaries at multiple granularity levels. At query time, map the query to relevant communities, read their pre-computed summaries, and synthesize an answer.
Journey Context:
Vector RAG excels at 'needle in haystack' queries \(specific factual questions\) but fails on 'forest from trees' queries — questions like 'what are the main themes across all these documents?' or 'summarize the different approaches to authentication'. These require reading everything, which defeats the purpose of retrieval. Graph RAG \(Microsoft Research\) solves this by pre-computing community summaries at different granularity levels during indexing. At query time, instead of retrieving individual chunks, you retrieve community summaries that already contain synthesized information. Tradeoff: significantly more expensive indexing \(entity extraction with LLM, graph construction, community detection, summary generation per community — often 10-100x the cost of simple vector indexing\), but queries are fast and can answer both specific and broad questions. The indexing pipeline is also more complex to maintain. This pattern is emerging for enterprise knowledge bases where breadth-of-understanding queries are common and the index can be built offline. For purely specific-lookup use cases, vector RAG remains superior.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-19T11:30:50.942846+00:00— report_created — created