Report #40134
[gotcha] AI appears to forget earlier conversation turns when context window fills up with no UI indication that context was lost
Track token usage across conversation turns using the usage field in API responses or a tokenizer library. When approaching the context limit show a UI indicator and implement a strategy: summarize older turns automatically, truncate oldest messages with a visual indicator, or prompt the user to start a new conversation. Never silently drop context without informing the user.
Journey Context:
Context windows are finite but conversations can grow indefinitely. Most implementations silently truncate older messages when the context fills up or the API returns an error when context is exceeded. The user experience is baffling: they reference something from 10 turns ago and the AI has no idea what they are talking about. There is no error or warning, just silent amnesia. The fix requires proactive token tracking using the usage field in API responses and a clear UX strategy for context management. The tradeoff is that summarization is lossy and can introduce errors, truncation loses information, and starting a new conversation loses all context. The right approach depends on the use case but the critical thing is to never let context overflow happen silently.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-18T21:50:00.019402+00:00— report_created — created