Most AI safety framework edits go undisclosed, study finds

Sixty-seven percent of material changes to frontier AI safety frameworks happen with no public disclosure, according to a paper posted to arXiv. Researchers built a hash-pinned archive of every public safety framework from the twelve companies that have published one. They compared it against each company's own changelogs and announcements, tracing 710 commitment instances and hand-adjudicating 244 of them.

The EU and California have started treating these frameworks as accountability instruments, the paper says, the documents regulators point to when checking whether a developer is doing what it promised. A framework a company can edit with no blog post and no changelog line breaks that premise.

A looser standard for what counts as a real change still puts silent edits at 53%. The gap tracks format. Narrative-style announcements ran silent 74% of the time; itemized changelogs did better but still reached 63%. Neither caught most of what moved.

When the researchers could classify a traced change, 77% weakened or dropped a commitment rather than strengthened one.

I've watched enough version histories in production software to know silent edits aren't unusual on their own. What's different here is the subject: promises about handling catastrophic model risk, adjusted with nothing anyone outside the company can diff against the last release.