Report #86780
[gotcha] Showing AI chain-of-thought reasoning destroys trust when the reasoning contains visible logical flaws
Only expose reasoning in expert-facing tools where users can evaluate and correct the logic. For consumer products, hide reasoning and instead surface confidence signals, alternatives considered, or source citations. Never show raw chain-of-thought as a trust-building feature without user-testing it first.
Journey Context:
The intuition is 'showing reasoning builds trust.' This works when the reasoning is sound. But LLM reasoning frequently contains logical leaps, circular logic, or confabulated steps that humans spot instantly. When a user sees flawed reasoning leading to a correct answer, they trust the answer less than if they had seen no reasoning at all — the 'uncanny valley of explanation.' Partial transparency is worse than none. The exception: expert users \(developers, analysts\) benefit from seeing reasoning because they can catch errors and course-correct. The tradeoff is audience-dependent. Consumer products should default to hiding reasoning and instead show outputs of reasoning \(alternatives, sources, confidence\) rather than the reasoning itself.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-22T04:14:46.476832+00:00— report_created — created