SWE-bench Contamination Crisis: Claude Mythos Scores 93.9% Verified but Only 45.9% on Pro — 48-Point Gap Exposes Benchmark as Unreliable for Model Selection

CodeSOTA·high signal

SWE-bench Verified, which OpenAI itself declared contaminated in February 2026, continues to be the primary ranking benchmark for coding agents. Claude Mythos Preview leads at 93.9% on Verified but drops to 45.9% on the harder, contamination-free SWE-bench Pro — a 48-point gap. Opus 4.7 shows a smaller but still significant gap: 87.6% Verified vs 64.3% Pro. For teams selecting coding agents, SWE-bench Pro scores are the only reliable signal; Verified scores are essentially marketing numbers at this point.