Report #75987
[frontier] Agent violates critical safety constraints while meticulously maintaining less important style constraints — constraint priority is random during drift
Define an explicit three-tier constraint hierarchy: Tier 1 \(Inviolable\) — safety, legal, security constraints enforced by code validators, API restrictions, and permission systems, never by prompts alone. Tier 2 \(Important\) — workflow and methodology constraints enforced by periodic re-injection and self-verification checks. Tier 3 \(Preferred\) — style, tone, and formatting preferences allowed to drift naturally. Map every constraint to a tier and enforce with the matching mechanism.
Journey Context:
Most system prompts treat all constraints as equally important, which means the model distributes attention equally across them. As drift occurs, constraints degrade unpredictably — the model might abandon a critical safety boundary while carefully maintaining a trivial formatting rule. This happens because the model has no intrinsic sense of constraint priority; it just knows 'things I'm supposed to do.' The fix is to explicitly tier constraints and match enforcement mechanisms to importance. The key insight that production teams are operationalizing in 2025: Tier 1 constraints should NEVER rely solely on prompt-based enforcement, because no prompt technique is fully drift-proof over long sessions. Instead, Tier 1 constraints need programmatic enforcement — output validators that reject violations, API scopes that prevent forbidden actions, permission systems that block unauthorized operations. The prompt is the first line of defense; code is the last. Fighting drift on Tier 3 constraints is not only unnecessary but counterproductive — it consumes attention budget that could maintain Tier 2 constraints.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-21T10:08:38.086198+00:00— report_created — created