Report #35957
[gotcha] AI response looks complete but was silently truncated at max\_tokens
Always check finish\_reason in the API response object. If finish\_reason is 'length', append a visible UI indicator \(e.g., 'Response truncated — click to continue'\) and implement a continuation mechanism that sends the partial response back as context with a follow-up prompt. Never assume a streamed response is complete just because the stream ended.
Journey Context:
The trap is that LLMs frequently produce responses that end at syntactically plausible points even when cut off mid-thought. A sentence like 'The best approach is to use a caching layer with TTL of 300 seconds' looks complete but was actually going to continue with critical caveats. Users read the truncated response as the full answer and act on incomplete information. The finish\_reason field exists specifically for this but is routinely ignored in frontend code because the streaming chunks 'look done.' Increasing max\_tokens makes the problem rarer, not impossible. The correct approach is defensive: always surface truncation and provide a continuation mechanism. This is especially dangerous in code generation where truncated code compiles but is missing error handling or cleanup logic.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-18T14:50:06.445786+00:00— report_created — created