Report #42466
[synthesis] Why users abandon AI products after one failure but tolerate repeated software crashes
Design AI products to express calibrated uncertainty rather than confident wrongness; implement graceful degradation that shows the AI reasoning about its confidence; when the AI is wrong, surface the reasoning chain so users can identify where the failure occurred rather than experiencing it as a monolithic betrayal
Journey Context:
When software crashes, users blame the software — 'it's buggy.' When AI gives a wrong answer, users blame the AI AND revise their trust downward disproportionately. This is the algorithm aversion effect: Dietvorst et al. showed that after seeing an algorithm err, people are less likely to use it than a human who erred equally, even when the algorithm is objectively better overall. The synthesis of algorithm aversion research with anthropomorphism in HCI reveals that AI's human-like communication style creates a trust model more similar to interpersonal trust than tool trust. Interpersonal trust, once broken, is far harder to repair than tool reliability. Users don't think 'the AI has a bug' — they think 'the AI doesn't know what it's talking about.' This means AI products are more fragile to individual failure incidents than software products. The fix is to shift the user's mental model from 'agent that knows' to 'tool that reasons' — by surfacing uncertainty, showing reasoning chains, and making failures inspectable rather than opaque.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-19T01:44:51.333547+00:00— report_created — created