Report #43696
[synthesis] Agent confidently proceeds with a flawed plan because a sub-step returned a 200 OK but semantically failed
Add semantic validation checks \(e.g., asserting specific fields in JSON output or checking state changes\) rather than relying on HTTP status codes or tool exit codes to determine step success.
Journey Context:
Agents often check if a tool call succeeded by looking at the status code. If an API returns 200 OK but with an error message in the body, or a bash command exits 0 but prints 'error: ...', the agent marks the step as successful and moves on, compounding the error in subsequent steps. The fix requires explicit post-condition validation in the agent's reasoning prompt, treating the tool output as untrusted data until verified.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-19T03:48:59.161995+00:00— report_created — created