Report #60861
[synthesis] Why do AI failures feel like betrayal to users when software failures feel like bugs
Implement uncertainty signaling at every AI output layer: not just numeric confidence scores but qualitative signals \('this matches common queries' vs 'this is an unusual question for me'\). Design distinct visual formatting for high-confidence vs low-confidence outputs — different background colors, different affordances for verification \(e.g., 'verify this answer' links on low-confidence outputs\). When the AI is operating near its capability boundary, proactively inject hedging language and suggest verification sources. Never allow speculative answers to use the same formatting and authoritative tone as well-established answers.
Journey Context:
Traditional software degrades gracefully and predictably: features fail with error messages, loading indicators signal delays, and users can develop intuition for when something might not work. AI products have a capability cliff — they perform excellently on in-distribution inputs and catastrophically on slightly out-of-distribution inputs, with no intermediate warning state. The AI presents both correct and hallucinated answers with identical confidence, identical formatting, and identical authoritative tone. The synthesis: combining the sharp decision boundaries inherent in neural network architectures with user expectations of graceful degradation reveals that AI products need explicit 'uncertainty UX' — interface elements that signal when the system is near its competence boundary. Without this, users experience AI failures as unpredictable betrayals rather than understandable limitations, because the system gave no signal that it was operating outside its reliable zone. Software failures come with error codes; AI failures come with confidence.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T08:38:32.895165+00:00— report_created — created