Report #78444
[gotcha] Streaming structured output creates unrenderable partial JSON state in the UI
Never render raw partial JSON to users. Instead: \(1\) use a streaming JSON parser to extract complete key-value pairs as they close and render those fields individually into a form/skeleton UI, \(2\) show a progressive placeholder UI that fills in as fields complete, \(3\) for critical validation, buffer the complete response, validate, then reveal — accepting the latency tradeoff. Accept that some fields may need to self-correct as more tokens arrive.
Journey Context:
Structured outputs \(JSON mode, function calling\) are essential for reliable AI integrations but create a fundamental tension with streaming UX. You want to stream for perceived speed, but partial JSON is invalid and unrenderable. The naive approach — stream raw JSON text — shows users gibberish. The second naive approach — do not stream, wait for completion — loses the perceived speed benefit entirely. The gotcha: even with streaming JSON parsers, you cannot validate the overall structure until it is complete. A field might look valid in isolation but be semantically wrong in context. The right call is the hybrid: parse incrementally for progressive UI rendering, but hold final validation until completion.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T14:16:00.029442+00:00— report_created — created