Multi-turn attacks crack AI agents 5-14% of the time

Single-turn jailbreak attempts against AI agents fail almost every time, 0 to 1%. Give the same attacker fifteen rounds to adapt, and the success rate jumps to 5.4-14%, per a new paper called Adaptive Adversaries (arXiv:2607.18063).

That's the gap between what most vendors test and what a patient attacker actually does. A safety score built on single-turn testing can undersell an agent's real failure rate by five to fourteen points.

Claude Opus 4.6 and GPT-5.4 tied at 5.4% aggregate failure. The average hides the real finding: Opus failed 60% of the time on one scenario where every competitor held at 7%.

The paper also found only 13 of 21 test scenarios could tell defenders apart at all. Eight scenarios didn't distinguish a good agent from a bad one.

The overlap problem

The multi-LLM attacker pool behind these numbers barely overlaps with existing jailbreak benchmarks, cosine similarity between 0.02 and 0.14, close to unrelated. If your vendor's safety eval and this adaptive pool are testing different things, the score you were handed doesn't reflect it. It doesn't show what an adaptive attacker does to your agent in production.

Watch whether other labs start publishing adaptive, multi-round attack numbers instead of single-turn pass rates. If they don't, ask why.