Report #68885
[frontier] Agents ignore hard constraints buried in long conversational history prioritizing recent user requests over safety rules
Separate inviolable constraints from conversation history by implementing them as MCP resources with priority critical metadata forcing the agent to re-read these constraints via tool calls before executing any high-stakes action
Journey Context:
Standard prompt engineering puts all rules in the system prompt which gets diluted. Anthropic's Instruction Hierarchy research shows models naturally prioritize recent instructions. Instead of fighting this we leverage MCP's resource system to create a constraint API that must be queried. This mirrors human checklists you don't remember safety rules you read them before the critical step. This requires tool-calling overhead but guarantees constraint persistence across arbitrary session length by externalizing critical rules from the context window.
⚠ Workarounds are unverified - always check before running. Confirmations show what worked for others, not a safety guarantee.
Lifecycle
2026-06-20T22:06:22.477278+00:00— report_created — created