Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-18 · high
Five coding harnesses that pass identical tests burn up to ten times more tokens than each other, and extra spend can't recover a fact that's missing.
Why now: The paper posted to arXiv as 2608.16630 on August 18, 2026.
A new study tested seven models across five coding harnesses, supplying facts, withholding them, and injecting faults to see what broke. Five harness setups in the study passed every test, yet differed by more than tenfold in tokens spent. Each one rebuilds the same facts at a different rate, and that gap is real cost for anyone running these tools daily.
They also swapped in each model's own parametric memory instead of supplying the fact directly. What decided the outcome wasn't how far a fact sat from the edit. It was whether the fact was available at all. A supplied fact worked as well far from the change as right next to it, per the paper.
When a fact isn't supplied, the agent doesn't stop and say it doesn't know. It fabricates a file or guesses a value and keeps going. Anyone watching agent runs by checking whether a read happened will see the hole already filled with something wrong.
Spending more tokens doesn't fix a fact that's genuinely absent, the study found. And where a CLAUDE.md-style convention file disagrees with what the code actually does, agents follow the stale file over the code.
Each link below shares sources, entities, or timing with this story.
Same source domain / Semantically similar
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.78).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.77).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.77).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.76).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.76).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.76).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.75).
Reported by the same outlet (arxiv.org); covers closely related ground (similarity 0.76).