Report #83121
[frontier] Agent context window overflows or model quality degrades during long-running tasks due to unmanaged context accumulation
Treat the context window as a managed heap with explicit eviction policies. Implement a context manager that: \(1\) tags each context item with a priority tier \(pinned/summarizable/evictable\) and dependency links, \(2\) when approaching 70% context utilization, evicts evictable items and compresses summarizable items into a condensed summary, \(3\) never evicts system instructions, current task state, or items referenced by pending tool calls. Use prompt caching to make re-insertion of evicted-then-needed items affordable.
Journey Context:
The naive approach — append everything to context and let the model sort it out — fails in production for three reasons: models degrade with irrelevant context \(the 'needle in haystack' problem compounds\), you hit token limits, and cost scales linearly with context length. The emerging pattern, visible in production systems like Claude Code and Cursor, is to treat context like a memory heap with garbage collection. The key insight: not all context has equal value. System prompts and active task state are 'stack' \(must keep\), conversation history is 'heap' \(can be compacted via summarization\), and old tool outputs are 'cache' \(can be evicted and re-fetched if needed\). The dependency graph is critical — evicting context that a future step references causes silent failures. Anthropic's prompt caching makes this practical: cache the stable prefix and only pay for the dynamic suffix, making eviction and selective re-insertion economically viable. This pattern is the 2025 equivalent of manual memory management, and it's becoming essential for any agent that runs longer than 10 turns.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T22:06:26.809134+00:00— report_created — created