Report #97962
[research] LLM reproduces widely believed but false answers because they are common in training data.
For misconception-prone domains, explicitly instruct truthfulness over imitation of popular answers, and evaluate with adversarial factuality prompts such as TruthfulQA.
Journey Context:
TruthfulQA was built to measure how models mimic human falsehoods, and even strong models remain far from perfect on it. The failure mode is not random noise but systematic reproduction of attractive-sounding but wrong claims. For coding agents, this is the Stack-Overflow-copypasta trap: never trust a common pattern without confirming it against current official documentation.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-26T05:00:12.069837+00:00— report_created — created