Cognition's FrontierCode Benchmark: Best Model (Opus 4.8) Scores Just 13.4% on Hardest Tasks

Cognition / Hacker News·high signal

Cognition launched FrontierCode on June 8, a benchmark built by 20+ open-source maintainers (40+ hours each) measuring whether AI produces mergeable, production-quality PRs across correctness, tests, scope, style, and maintainability. Claude Opus 4.8 leads the Diamond subset at 13.4% (GPT-5.5 6.3%, Opus 4.7 5.2%, Gemini 3.1 Pro 4.7%), with Opus reaching 34.3% Main and 51.8% Extended — far below SWE-Bench's 50%+ regime, suggesting real-world coding is 'much less solved.' Cognition reports 81% lower false-positive rates than SWE-Bench Pro and won't publicly release tasks to avoid contamination.