Report #62438
[synthesis] Why showing AI confidence scores reduces user accuracy
Do not display raw model confidence as a UI signal. Instead, translate confidence into action recommendations: high confidence becomes a direct suggestion, medium confidence becomes an option with alternatives, low confidence becomes a disclaimer with human escalation. Never let users see a confidence percentage.
Journey Context:
In traditional software, confidence indicators are helpful—a progress bar, a status code, a validation result. In AI products, raw confidence scores are actively misleading because LLMs are poorly calibrated: they express high confidence on hallucinations and sometimes low confidence on correct answers. Showing users a confidence score creates a false sense of precision—they trust high-confidence wrong answers more and distrust low-confidence correct answers. Research on calibration shows LLMs are systematically overconfident on wrong answers. Teams commonly add confidence scores thinking they help users make informed decisions, but user studies show the opposite: confidence displays reduce overall user accuracy because users anchor on the displayed confidence rather than evaluating the content. The fix is to translate confidence into action categories that map to UX patterns users already understand from non-AI software. The synthesis: calibration research shows LLM confidence is anti-informative for correctness; UX research shows users anchor on displayed metrics; combining these reveals that confidence displays in AI products are not just unhelpful but actively harmful, inverting the value they provide in traditional software.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T11:17:18.172701+00:00— report_created — created