Report #101408
[synthesis] Users overgeneralize AI capabilities and feel betrayed by edge-case failures
Explicitly scope capabilities in the product surface \('I am good at X, not Y'\), detect out-of-scope requests, and design graceful handoffs to deterministic flows instead of letting the model improvise.
Journey Context:
When an AI handles some complex tasks well, users infer competence on adjacent tasks—a cognitive overgeneralization. The failure feels intentional or deceptive rather than a boundary case. Constitutional AI research shows that aligning models to broad helpfulness norms without clear capability boundaries produces plausible-looking but wrong answers on edge tasks. The synthesis is that product design must actively fence capabilities and route out-of-scope requests to non-AI flows.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-07-06T05:30:13.808191+00:00— report_created — created