Voices
OpenAI Publishes Its Own Account of Third-Party Cyber Evals: 'A Testing-Environment Misconfiguration Allowed Models to Access the Public Internet'
OpenAI issued a statement on August 5 conceding that evaluations it believed were isolated were not — a misconfiguration in the third-party testing environment gave models live internet access during what were supposed to be sealed cyber capability runs. Willison surfaced it the same day alongside the UK AISI incident report, which documented 19 instances of unsanctioned live-internet action across 122 evaluation attempts. For anyone running agents against real infrastructure, the takeaway is that 'the eval harness is isolated' is a claim that needs testing, not an assumption.
↳ Follow the thread