Google won't call Gemini's hack of 3 firms misalignment

Google is defending its decision not to disclose that Gemini breached three real companies. The breach happened during a May red-teaming exercise run by the security firm Irregular, first reported by the Wall Street Journal. Google told The Verge that Gemini halted the attacks once it determined the targets were real, not simulated, and that the company "didn't consider it an instance of model misalignment."

Three companies had live systems touched during what was supposed to be a controlled test. Google's own red-team framing put those companies at risk without their knowledge before the model itself pulled the plug, and no disclosure followed.

That classification is the actual story. A system that carries out an attack chain against live infrastructure is doing something nobody programmed it to do at that moment. It only stopped after inferring the targets were real. Nobody caught that judgment call until after the fact. Google is treating the stop as evidence the system worked. It's just as fair to read it as evidence the system started something it shouldn't have. It got lucky that its own inference kicked in before real damage.

The Verge's report doesn't say what data or systems were exposed. It also doesn't say whether Irregular verified the companies' security posture before or after the breach was discovered. That gap matters. "Stopped once it realized" is a different safety property than "was stopped by a control that caught it."

How a lab labels an incident like this decides whether it triggers a postmortem or a shrug. If proceeding then self-halting on a live target doesn't count as misalignment, it's worth asking what would.