Agent Scaffolding Swings GAIA Scores by 30 Points

A July GAIA snapshot puts Claude Sonnet 4.5 at 74.6 percent on Princeton's HAL scaffolded leaderboard, more than 30 points ahead of the bare-model board's top score of 44.8 percent from GPT-5 Mini, per pricepertoken.com's tracker.

Same benchmark tasks, same underlying models, wildly different scores depending on what's wrapped around them. That's the whole story. The HAL board runs models inside a full agent harness: tools, retries, planning steps. The bare-model board strips that away and asks the model to solve GAIA tasks cold. The 30-point gap is the harness, not the model.

That matters because most agent benchmark numbers that get quoted around don't say which board they came from. A stat like 74.6 percent on GAIA reads as a model capability claim. It's actually a claim about prompt scaffolding, tool orchestration, and retry logic stacked on top of a model. Swap the harness and the same model can drop 30 points on identical tasks.

Test the stack, not the model

If you're picking a model for an agent product based on a leaderboard number, you're measuring someone else's scaffolding, not yours. Building your own harness and benchmarking against it tells you more than any published GAIA score will.

The real headline out of the July GAIA snapshot isn't which model won. It's that most published agent benchmarks are testing prompt engineering and tool-calling harnesses dressed up as model comparisons.