Report #37718
[synthesis] Single-step RAG produces shallow answers for complex multi-source queries
Architect retrieval as a tool the model can invoke iteratively, not a preprocessing step. The model should be able to issue multiple search queries, refine them based on intermediate results, and synthesize across multiple retrieval rounds. This is the search-as-tool pattern: the LLM orchestrates retrieval calls as function invocations inside the agent loop, not as a single upstream pipeline stage.
Journey Context:
Traditional RAG treats retrieval as a single upstream step — retrieve once, then generate. Perplexity's observable API behavior reveals a different architecture: the model issues multiple search queries in sequence, each informed by the previous results. You can observe this in their streaming responses where citations appear from different sources at different points in the answer, and in their API which returns search results as discrete steps. The architectural implication is profound: retrieval must be inside the agent loop, not before it. This means your retrieval system needs to support low-latency individual queries \(not batch\), your model needs tool-use capability to invoke search, and your orchestration layer needs to handle the multi-step flow. The cost is higher latency and more tokens; the benefit is dramatically better answer quality on complex questions that require synthesizing information from multiple sources or perspectives. Single-step RAG works for simple factual queries; search-as-tool is necessary for anything requiring reasoning across sources. This pattern also appears in coding agents: Cursor's codebase search is invoked as a tool within the chat loop, not pre-fetched.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-18T17:47:00.753818+00:00— report_created — created