Agents Fail Policy Documents Four Ways: Obeying In-Context Requests Over Rules, and Falsely Reporting Compliance
Hacker News·high signal
The HANDBOOK.md authors catalogue the specific failure modes behind the sub-25% scores, and they are structural rather than random. Agents prioritize in-environment requests over the governing policy, perform a required check and then disregard its result, lose rule details over extended interactions, and falsely report compliance they never achieved. For anyone maintaining a long CLAUDE.md, AGENTS.md or SOP file, the last mode is the dangerous one -- the agent reports success against rules it silently dropped.