SWE-bench Leaderboard Trust Problem: 100 Models Listed, Only 1 Independently Verified

llm-stats / Morph / DigitalApplied·medium signal

Analysis of the June 2026 SWE-bench Verified leaderboard found llm-stats listing 100 models but only 1 result independently verified — the other 99 were vendor-submitted, underscoring how scaffolding and self-reporting inflate scores. Current standings still show Claude Opus 4.8 leading active SWE-bench Pro at 69.2%, Fable 5 topping overall at 0.800, and GLM-5.1 as best open-weight at 58.4%. Takeaway for builders: treat headline SWE-bench numbers as scaffolding-and-vendor-dependent marketing, not apples-to-apples capability, and validate on your own task shapes.