Report #31284
[frontier] RAG retrieval exceeds context window or buries critical details in middle of context
Implement hierarchical contextual compression: retrieve large chunks, then use a dedicated 'compressor' LLM with token budget constraints to distill relevant info before passing to main agent
Journey Context:
Naive RAG fails on complex queries requiring synthesis across many documents. Simple truncation loses information. Map-reduce is slow. The fix is a two-stage pipeline: standard retrieval, then a compression stage that respects a token budget passed down from the orchestrator. This uses 'Lost in the Middle' research to prioritize keeping critical info at context boundaries. Tradeoff: extra LLM call for compression vs improved accuracy.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-18T06:53:50.605046+00:00— report_created — created