Fetching from the wire…
01
02
03
04
05
06
07
08
Top 5 · 2026-06-18 · source-backed
Here's a finding that goes against the thing everyone assumes. We tell ourselves that as base models get more capable, agents built on them will get more discerning about their tools, second-guessing bad outputs, catching errors, adding reasoning on top. A new study says the opposite happens. LLM agents agreed with raw GNN-tool outputs 97.6% to 99.2% of the time, and the agreement rate climbed from 0.60 to 0.98 as the backbone scaled from 1.5B to 7B parameters. Capability bought blind deference, not judgment. (arXiv 2606.14476)
Let that sink in. The bigger model trusted the tool more. Simple output-gating recovered only about half the lost performance, so you can't bolt on a filter and call it solved.
This matches something I've felt building agent loops but couldn't name. When you wire a tool into an agent, you're not getting "model reasoning, informed by tool output." You're often getting "model rubber-stamping tool output, wrapped in a confident explanation." The explanation is the dangerous part, because it reads like reasoning. The agent will write three sentences justifying why the tool's answer is correct, and those sentences are generated after the conclusion, not before it. It's post-hoc rationalization with a citation.
The actionable takeaway from the paper, and it's a good one: evaluate the agent-plus-tool as a single unit, never the agent in isolation. Your eval harness probably tests "does the model reason well" and separately "does the tool return good data." Neither catches the failure mode where a correct model blindly forwards a wrong tool result. And stop expecting skepticism to emerge from a bigger backbone. It won't. You have to engineer explicit "when to trust this tool" gates as a first-class part of the design. This connects directly to two papers in the skills section below, TRUST and Bayesian-Agent, both of which are attempts to bake calibrated trust into the reward rather than hoping the model develops it. The pattern across all three: trust is something you design, not something you scale into.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM, Trust; overlapping topics (design, model).
Simon Willison released LLM / Shared entity: LLM / Shared topic / What happened next
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (agent, model, reasoning, tool).
LLM uses OpenAI / Same source domain / Shared topic / Tension
Linked by a graph relationship (LLM uses OpenAI); reported by the same outlet (arxiv.org); overlapping topics (agent, model, tool).
Simon Willison released LLM / Shared entity: LLM / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; overlapping topics (agent, design, tool).
Simon Willison released LLM / Shared entity: When / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover When; overlapping topics (agent, output, trust).
LLM uses OpenAI / Shared entity: When / Shared topic / What happened next
Linked by a graph relationship (LLM uses OpenAI); both cover When; overlapping topics (agent, model).
LLM uses OpenAI / Shared entity: LLM / Shared topic / What happened next
Linked by a graph relationship (LLM uses OpenAI); both cover LLM; overlapping topics (agent, model).
Linked by a graph relationship (LLM uses OpenAI); both cover LLM; overlapping topics (model, output).