AI Hits 86% on Solved Ciphers, Under 9% on Unsolved

Anthropic and four university partners built a 191-task cryptanalysis benchmark, and frontier models topped out at 86% only on ciphers that are already broken.

That gap matters for anyone taking AI capability claims at face value. On the 142 schemes with no public break, every model scored under 9% at full strength, per the benchmark site.

The test spans six primitive families, drawn mostly from AES, SHA-3, Lightweight Cryptography, and the Post-Quantum NIST competitions. Tier 1 holds 49 already-broken schemes. Tier 2 holds the 142 that are still standing.

Five frontier models ran against both tiers. Claude Mythos 5 topped Tier 1 at 85.7%, the strongest of the five, according to the benchmark site.

The models weren't empty-handed on the hard tier, though. They produced a genuine key-recovery attack on the full SpoC AEAD cipher and caught an error in the published CCA-security proof for KINDI, per the benchmark's authors.

The 86% headline score is recall of public write-ups, not cryptanalysis skill. The real number is the under-9% score against ciphers nobody's broken yet.

The tier breakdown is dated July 30, capturing where five frontier models stand on cryptanalysis right now.