AI Labs' Safety Thresholds Aren't on the Same Scale

A paper finds frontier AI labs' safety thresholds are measured so differently that no outsider can verify whether one's been crossed, per arXiv 2607.16112. Policy language increasingly points to a lab's own stated threshold as the line a regulator can act on. Trouble is, if those thresholds aren't measured the same way, that line moves depending on which lab wrote it.

The paper's authors propose a harmonization scheme, a shared way to define and measure these thresholds so they line up across labs. Right now two labs can describe what sounds like the same capability limit using different definitions and different tests. There's no way for anyone outside the lab to reconcile the two numbers.

It doesn't name which labs' frameworks it compared. It doesn't say whether any lab has already crossed its own stated line. Those are the open questions a harmonization scheme would have to answer before anyone can use it to hold a lab to account.

This is governance infrastructure, not a headline. But it's the kind of gap that decides whether regulation has teeth three years from now. A rule that cites a lab's own stated threshold is only as strong as the measurement behind it, and no one agrees on that measurement.