AI kill switches face a design problem lawmakers haven't solved

The New York Times reported September 19 that AI kill-switch legislation has a design flaw lawmakers haven't resolved, per Tech Times. A system capable enough to resist or dismantle its own shutdown mechanism defeats the point of building one. Lawmakers haven't agreed on what would count as proof a system can't do that.

That question sits on top of Governor Newsom's executive order. The order pushes California toward requiring kill-switch capability and onsite third-party auditors for frontier AI development. It assumes a shutdown mechanism is a thing you can build and certify. The reporting says experts aren't agreed on what that even looks like once the system in question is sophisticated enough to route around it.

A kill switch is about to become a line item on a compliance checklist, but there's no spec yet for what satisfies it. Does it mean a hardware cutoff outside the model's own infrastructure? A permissions layer the model can't touch? Nobody's published an answer, and the Tech Times piece doesn't offer one either. It names the scenario regulators haven't priced in: a system that can identify and counteract the very mechanism meant to stop it.

Watch what the third-party inspection requirements ask for when California writes them down. "Show us a shutdown button exists" is the easy bar. "Prove the system can't defeat it" is the hard one. Only the second closes the gap the Times reporting describes.