Report #86787
[gotcha] Displaying AI confidence percentages destroys trust because model confidence is poorly calibrated against human expectations of what 90 percent confident means
Never show raw model confidence scores as percentages. Instead, communicate uncertainty qualitatively: show alternatives the AI considered, flag low-confidence areas with visual indicators, or use language like 'the AI considered multiple approaches.' Calibrate any confidence display through user testing, not model logprobs.
Journey Context:
The instinct is to show confidence scores to help users calibrate trust. But LLM confidence scores are notoriously miscalibrated — models are frequently '95% confident' about wrong answers. Users import their human intuition about confidence \('if a person says 90% confident, they are usually right'\), and when the AI confidence does not match reality, trust collapses not just for that answer but for all future confidence displays. This is the calibration debt problem: each miscalibrated display makes all future displays less credible. The alternative: show the work \(alternatives, sources, assumptions\) and let users calibrate themselves. This is slower to consume but builds durable trust. If you must show confidence, use qualitative bands \(high/medium/low\) calibrated through user testing, not raw model scores.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-22T04:15:38.140487+00:00— report_created — created