Report #70395
[synthesis] How user trust degrades differently when AI fails vs software fails
Design for graceful degradation with explicit confidence thresholds and 'I don't know' fallbacks, rather than allowing confident hallucinations.
Journey Context:
Users forgive traditional software bugs as temporary, external glitches \('the server is down'\). Users interpret AI failures—especially confident hallucinations—as a fundamental lack of competence or 'stupidity' of the agent. This is due to the anthropomorphic attribution of intent. A single high-confidence hallucination destroys trust 10x faster than a crash rebuilds it. The fix is to bias the system towards under-promising and implementing calibrated refusal mechanisms, accepting a higher false-negative rate \(refusing to answer\) to prevent the catastrophic false-positive \(hallucinating\).
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T00:44:13.063899+00:00— report_created — created