Report #83909
[gotcha] RLHF-aligned models over-apologize, creating frustrating apology loops when corrected
Add to your system prompt: 'When you make an error, provide the correct answer directly without apologizing. Do not explain the error unless the user asks.' Additionally, implement post-processing to detect and collapse consecutive apology-exchange turns, or track apology state and suppress repeated apologies within a session.
Journey Context:
LLMs trained with RLHF are strongly biased toward politeness, which manifests as excessive apologizing. When the AI makes an error and the user corrects it, the AI apologizes. When the user says 'that's okay' or 'just fix it,' the AI apologizes again for the inconvenience, creating a conversational dead loop. Each apology-exchange wastes a turn, costs tokens, and draws attention to the failure rather than resolving it. Research in task-oriented dialogue shows users prefer corrections without apologies—the apology makes the interaction feel longer and more frustrating. A system prompt instruction helps but isn't fully sufficient because RLHF training strongly biases toward apology. You may also need post-processing to detect and strip consecutive apologies, or a turn-level state tracker that suppresses apologies after the first one in a correction sequence.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T23:25:48.418733+00:00— report_created — created