ARC-AGI-3 Humbles Every Frontier Model. A Simple CNN Beats Them All by 30x.

GPT-5.4 scores 0.26%. Opus 4.6 scores 0.25%. Grok-4.20 scores 0.00%. Humans score 100%. The Decoder covered the ARC-AGI-3 launch on March 25, and the results make every "AGI is here" claim look premature.

François Chollet launched ARC-AGI-3 at Y Combinator HQ alongside a fireside conversation with Sam Altman. The benchmark is fully interactive: hundreds of game-style environments with no instructions, no rules, and no stated goals. Agents must figure out what to do by exploring and learning from environmental feedback. Sustained sequential reasoning, state tracking across hundreds of steps, real-time adaptation. Everything current language models can't do.

Here's what stopped me cold: simple CNN and graph-search approaches scored 12.58%. Over 30x better than any frontier LLM. Not a fine-tuned model, not a billion-dollar training run. Basic pattern matching over a narrow domain demolishes trillion-parameter models on tasks requiring actual novel reasoning. The models interpolate beautifully from training data. They can't extrapolate at all. The gap between "looks smart" and "is smart" has never been measured this precisely.

The Chollet-Altman pairing is notable. They agreed on a timeline: AGI "probably by early 2030s, around ARC-AGI 6 or 7" (OfficeChai). The creator of the hardest AI benchmark and the CEO of the company most invested in scaling sat together and acknowledged that current approaches aren't sufficient. Meanwhile, GPT-5.4 saturated USAMO 2026 at 95%, up from near-zero last year (MathArena). So models are getting dramatically better at pattern-matchable math competitions while scoring essentially zero on tasks requiring genuine novel reasoning. That divergence is the whole story.

ARC Prize 2026 offers $2M+ in prizes. All solutions must be open-sourced. If you want to work on the hardest unsolved problem in AI, the benchmark and the funding are waiting. This connects directly to the Anthropic harness story: if individual models can't reason sequentially, you build systems where reasoning is distributed across specialized agents with external feedback loops. Multi-agent architectures aren't just a design pattern. They're a workaround for a fundamental limitation.