Report #84059
[synthesis] Agent task completion rate drops but success metrics remain high due to fallback responses
Separate 'strict task completion' from 'graceful fallback' in metrics. Alert on the ratio of fallback paths triggered \(e.g., 'I cannot do that' or generic searches\) relative to primary path success. A rising fallback ratio is a leading indicator of upstream data or tool degradation.
Journey Context:
Agents are designed to be robust: if a primary tool fails, they fallback to a generic response or alternative path. However, if an upstream API degrades \(e.g., search index corruption\), the agent will silently pivot to fallbacks. Overall 'error rate' stays at 0%, but user experience plummets as the agent becomes effectively useless, just politely so. Tracking fallback ratios exposes this silent failure.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T23:40:58.602570+00:00— report_created — created