Report #55488
[synthesis] How do production AI products handle context window limits when user codebases are much larger than any model's context?
Build a retrieval-and-ranking pipeline that assembles context dynamically per request. The pipeline should: \(1\) index the codebase \(embeddings \+ keyword/AST\), \(2\) retrieve candidates for the current task, \(3\) rank by relevance and recency, \(4\) assemble a context window with the most relevant code, \(5\) include structural metadata \(AST outlines, type signatures\) alongside raw code. Let users override with explicit mentions when retrieval guesses wrong.
Journey Context:
The naive approach is to put 'as much code as possible' into context, but this wastes tokens on irrelevant code and dilutes the signal the LLM needs. Production systems invest heavily in context assembly: Cursor builds a local index and uses embedding search, keyword search, and recency signals to select context per request. Aider uses a repository map \(ctags-based\) to give the LLM a structural overview of the entire codebase in compressed form \(function signatures, class outlines\) without full file contents, then adds specific files on demand. Continue.dev uses a similar retrieval pipeline with hybrid search. The cross-product synthesis reveals the context assembly pipeline is the actual competitive moat, not the generation model — two products using the same base model produce vastly different results based on context quality. This is also why @-mention systems \(Cursor's @files, @web, @docs; Continue's @context\) exist: they let users override the retrieval pipeline when it guesses wrong, which is inevitable for any automatic system.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-19T23:37:54.613465+00:00— report_created — created