Only 4 to 16 percent of CLAUDE.md rules match real controls

Researchers checked 481 public CLAUDE.md files against Claude Code's real controls, per a security audit posted to arXiv. Only 4% to 16% of those rules matched a permission setting, a sandbox boundary, or a hook that could stop the model. The strictest matching standard puts the number at 4.4%, with a 95% confidence interval of 2.6% to 6.7%. For anyone running an agent unattended, the gap sits between the rule you wrote and the rule Claude Code will enforce.

The format mixes three kinds of instruction without marking which is which. One kind a permission setting can deny. Another a sandbox boundary can contain. The last is just English, and the model may or may not follow it this turn. Two security practitioners labeled the rules independently and agreed on the classifications, so the count isn't one researcher's read of ambiguous prose.

My own project instructions file has lines about never touching credentials and never committing database files. I'd have called those controls until I read this paper. They're wishes with good intentions behind them. The permission system and the settings file are where enforcement lives, and I hadn't moved a single one of those lines over.

Two related findings sit in the same hole. NVIDIA's skill-lift check on skill-file structure predicts usefulness at a 0.14 rank correlation, and Repo2Skill-Evo found every release transition invalidated part of a skill set. None of the three checks whether the markdown does anything once it's written.

For every 'never do X' line in your instructions file, check whether a deny permission rule, a sandbox setting, or a hook can express it. If it can, move it there and delete the prose version. If it can't, it stays advice, not a control.