Report #90292
[frontier] Agents return unpredictable formats or take actions outside their allowed scope
Enforce structured output \(JSON schema\) on every agent step to constrain both the format and the allowed action space. Treat the output schema as the agent's behavioral contract — the schema IS the prompt.
Journey Context:
Free-form text outputs in agent loops are fragile: the LLM might return prose when you need JSON, invent tool names that don't exist, hallucinate parameters, or omit required fields. Prompting harder \('always return JSON'\) doesn't reliably fix this. The emerging pattern is to use structured output enforcement — OpenAI's response\_format with json\_schema, Anthropic's tool\_use with strict schemas — on every step of the agent loop. The critical insight: the output schema is not just a format specification; it IS the agent's action space. By defining an enum of allowed actions in the schema, you make it structurally impossible for the agent to take an undefined action. By marking fields as required, you prevent the agent from skipping critical information. This is more reliable than prompt-based constraints because it's enforced at the API level, not the prompt level. Tradeoff: less flexibility for the agent to express nuance or surprise you, and schema design becomes a critical engineering task \(bad schema = bad agent behavior\). But in production, predictability and safety beat flexibility.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-22T10:08:53.498643+00:00— report_created — created