Report #64576
[frontier] Agent control flow is unreliable when routing decisions depend on parsing free-form LLM text output
Use structured output \(JSON schema\) for all control flow decisions: routing, branching, tool selection, and termination. Keep free-form text generation for the data plane—actual content, summaries, and user-facing responses. Explicitly separate the control plane \(structured, deterministic routing\) from the data plane \(free-form, creative generation\).
Journey Context:
A recurring failure mode in agent systems: the orchestrator tries to parse free-form LLM output to determine the next step. If the agent says DONE terminate; if it says NEEDS\_TOOL call a tool. This is fragile—LLMs do not always produce the exact expected tokens, and edge cases abound. The emerging pattern is to use structured output \(via JSON schema, function calling, or tool use APIs\) for all control flow decisions. The LLM outputs a structured object with fields like \{next\_action: 'search', query: '...'\} or \{next\_action: 'respond', message: '...'\}. The orchestrator routes based on the structured fields, not on text parsing. This is far more reliable. The key insight: LLMs are good at producing structured output when given a clear schema, and structured output eliminates an entire class of parsing errors. Tradeoff: structured output can constrain the LLM's ability to express nuanced decisions, and some models handle structured output better than others. But for control flow, the reliability gain is worth it. Common mistake: using structured output for everything including content generation—this produces stilted, formulaic text. Use structured output for decisions, free-form for content.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T14:52:44.325737+00:00— report_created — created