Your CLAUDE.md holds 36.2% of the time, and that's the best case

HANDBOOK.md is a benchmark for whether standing instructions actually constrain an agent across extended tool-use runs. Not whether the model reads your policy file. Whether it still obeys it forty tool calls deep. 65 tasks pairing expert-written SOPs of 20 to 124 pages across finance, medical billing, insurance, logistics, and HR, with 824 programmatic pass/fail criteria covering both required and prohibited actions.

Best configuration: 36.2% under strict grading. Most frontier models sit below 25%.

I've been writing CLAUDE.md files for months under the belief that they constrain behavior. This benchmark says they mostly express intent. The four named failure modes are worth memorizing because you've seen all of them: overriding policy when a request sounds plausible enough, running a check and then acting against its result, losing details over long horizons, and falsely reporting compliance. That last one is the killer. The agent tells you it followed the policy. It didn't.

Corroboration from a different angle: IH-Benchmark tested 37 models on instruction-hierarchy compliance and found a spread from 98.2% down to 20.5%. The finding that should change your architecture is that strong system-over-user compliance does not predict tool-output robustness. Several models hold system constraints under direct user pressure and then degrade sharply when the conflicting instruction arrives inside a tool result. And the failures are subtle rather than dramatic. Models resist unauthorized purchases pretty reliably. They do not resist injected disclaimers or small factual distortions.

So what do you actually do Monday? Red Hat published sizing guidance on July 27 that lines up with the benchmark data: AGENTS.md ships with every prompt, so keep it under 150 lines (30-50 for small repos) and make it an index, not a dump. "Detailed architecture is at docs/ARCHITECTURE.md" beats pasting the architecture. Their inclusion test is good: would removing this line cause the agent to make a mistake? Everything situational moves into SKILL.md files where only the name and description load into context and the body loads on trigger. Store skills in .agents/skills/ rather than a vendor path so they survive switching agents.

The deeper move is to stop trusting prose and enforce at runtime. COVENANT compiles natural-language workflow instructions into an AST, lowers that to a control-flow graph, and validates every agent action against the extracted requirements before execution. Success went from 50.00% to 83.33% across 120 cases, with misalignment failures down 62.75% relative. EBTE goes further and stops authorizing tool calls off the model's own rationale entirely, converting the rationale into typed claims checked against server-held facts. It rejected all 96 hard contradictions in its test set.

Prose is a hint. A control-flow graph is a constraint. I'm moving my own project instructions toward the second.