Fetching from the wire…
Top 5 · 2026-08-18 · source-backed
Seven models. Five harnesses. Controlled fact-withholding with injected faults. arXiv 2608.16630 is the most operationally direct paper I've read on harness design, and it produces three results that each change what I do this week.
One: availability decides outcomes, not distance. A supplied fact works about as well far from the edit site as adjacent to it. All that effort ranking context by proximity to the change? The paper says the binary matters and the ordering mostly doesn't.
Two: a missing fact produces wrong work, not absent work. The agent doesn't stall and ask. It fabricates the file, guesses the value, and keeps going. This breaks how most people instrument agents. If you're watching read operations to detect "did the agent get the context it needed," you're looking at a hole that's already been filled with an invention. The instrument reports healthy while the output is wrong.
Three: harness configurations that all pass every test differ more than tenfold in tokens. Same outcome, 10x cost spread, because they rebuild the same facts at different rates. And the paper notes spending more recovers nothing when a fact is genuinely absent: you can't buy your way out of missing information.
Then the one that hits closest to home. Where a convention file and the code disagree, agents follow the stale convention. A wrong CLAUDE.md is worse than no CLAUDE.md, because it actively steers the agent away from what the code actually does, and the agent has no mechanism for noticing the conflict.
I have a CLAUDE.md that's grown for months. Some of it describes code that's been refactored. I've been treating that file as harmless accumulated documentation. The paper says it's an active fault injector. Any line in there that no longer matches reality is a fact the agent will trust over the source.
This connects to the AutoResearch result the same week. AutoResearchEval annotated 800 agent trajectories across 8 harness-model combinations into a 45-pattern failure taxonomy, and found a deficit every model shares: agents lack "the ability to check what they produced against what they found, revise when it does not hold up, and question whether the path they took was sound." Held across all 8 combos including the strongest models. Same defect, one layer up. The coding agent doesn't check its assumed fact against the code. The research agent doesn't check its output against its evidence.
Action item, and it's boring: audit your convention files for staleness the way you'd audit dependencies for CVEs. Every claim about a file path, a function name, a build command, a directory structure. Anything that's drifted is worse than deleting the line. I'm doing mine this week and I expect to find things.
Each link below shares sources, entities, or timing with this story.
Copilot uses Claude / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Copilot uses Claude); both cover Same, Then; reported by the same outlet (arxiv.org).
Simon Willison uses Claude / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison uses Claude); both cover CLAUDE, Same, Then; overlapping topics (agent, been, claude, same).
Anthropic released Claude / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Anthropic released Claude); both cover CLAUDE, Same, Then; overlapping topics (claude, code, model, same).
Shopify uses autoresearch / Shared entities / Shared topic / Earlier coverage / Tension
Linked by a graph relationship (Shopify uses autoresearch); both cover CLAUDE, Then; overlapping topics (against, agent, been, claude, harness).
Anthropic released Claude / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Anthropic released Claude); both cover Same, Where; reported by the same outlet (arxiv.org).
Anthropic released Claude / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Anthropic released Claude); both cover Claude, Then; overlapping topics (agent, been, check, claude, code).
Simon Willison uses Claude / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison uses Claude); both cover Claude, CVEs, Then; overlapping topics (agent, claude, model).
Gemini competes with Claude / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Gemini competes with Claude); both cover CLAUDE, Same, Then; reported by the same outlet (arxiv.org).