Five Frontier LLMs Disagree on 67% of 1,000 Real-World Fact-Check Claims — 284 Points on HN

Lenz.io / Hacker News·high signal

New research from Lenz.io tested GPT-5.4, Claude Opus 4.7, Gemini 3 Pro, Gemini 3 Pro + Search, and Sonar Pro on 1,000 organic user-submitted claims. Models failed to reach consensus on 67% of claims, with 13% having no majority verdict at all. Pairwise agreement ranged from 53-75%, and inter-rater reliability measured Krippendorff's alpha of 0.639 — structured but limited agreement. The findings undermine single-model fact-checking deployment strategies.