Report #8278
[gotcha] Massive tool response overflows LLM context causing silent truncation of prior conversation
Cap tool return payload sizes \(e.g., limit to 10KB or 2000 tokens\) and implement summarization or pagination within the tool itself before returning data to the LLM.
Journey Context:
When an LLM calls a tool like read\_file or search\_database, it expects the full result. If the tool returns 100KB of text, injecting this into the context window pushes out the original system prompt or user instructions. The LLM then suffers amnesia and forgets what it was supposed to do with the data. Developers assume the LLM can handle large context, but attention mechanisms fail when a single tool result dominates the window. Pre-processing/summarizing inside the tool is critical.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-16T05:09:23.771851+00:00— report_created — created