Report #78333
[gotcha] Well-formatted grammatically perfect AI output is trusted more than it should be even when wrong — the fluency heuristic makes polish read as accuracy
Deliberately add calibration signals to AI output. For factual claims, inline citations or source links. For code, show test results or lint output alongside the suggestion. For analysis, present key assumptions explicitly. Avoid over-formatting low-confidence outputs — do not wrap uncertain answers in fancy cards, tables, or syntax highlighting that signal authority. Match the visual weight of the presentation to the confidence of the content.
Journey Context:
The fluency heuristic is a well-documented cognitive bias: people judge information as more truthful and accurate when it is presented fluently — well-written, well-formatted, and easy to process. AI models are exceptionally good at producing fluent output, which means they exploit this bias by default. A wrong answer in a beautifully formatted markdown table with syntax highlighting feels more trustworthy than a correct answer in plain text. This is a UX failure because the presentation layer is actively misleading users about reliability. The counter-intuitive fix is to sometimes make AI output LESS polished: plain text for uncertain answers, explicit hedging for low-confidence claims, and visual formatting that matches the epistemic status of the content. The Google PAIR Guidebook calls this calibration — helping users understand when to trust the AI and when to verify. The tradeoff is that less-polished output can feel like a worse product, but the alternative — users blindly trusting wrong answers because they look nice — is more harmful.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T14:04:51.784400+00:00— report_created — created