Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-31 · high
A structured-output constraint turned off tool calls, and the model fabricated the source text it was supposed to extract from.
Why now: The account of the incident and the detector built to catch it are current as of August 31, 2026.
A structured-output constraint disabled tool use during model qualification for a datasheet-extraction service, per a paper detailing the incident. The model fabricated the source text it was supposed to extract from, and the qualification check still marked the run a pass.
The failure matters beyond this one service. Any automated check that grades only the final answer can't tell a model that skipped its tools from one that used them correctly.
To catch that gap, the authors built a detector. They logged every tool call across 37 curated claims, then wrote two rules that check which tools got called, never the values those tools returned.
Against 207 clean extractions that passed fidelity checks across three model families, the detector raised no false alarms. Against 50 planted tool-withholding faults, it caught all 50.
They're upfront about the limit. Perfect recall against the planted faults is true by construction, since the faults were built to trip these exact rules. What the detector catches when a model calls the right tools but still gets the answer wrong is left unmeasured.
This check watches which tools got called, never what came back, and catches one specific failure: tools disabled with no error to flag it. It says nothing about a model that calls the right tool, gets a real result, and still botches what it does with that result. Nobody has measured that gap yet.
Each link below shares sources, entities, or timing with this story.
Same source domain / Shared topic / Tension
Reported by the same outlet (arxiv.org); overlapping topics (against, answer, author, called, clean); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (against, call, model, tool); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (against, answer, model, tool); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (against, check, claim, model); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (against, call, check, tool); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (against, call, model, tool); pushes against this story (against).
Reported by the same outlet (arxiv.org); overlapping topics (against, call, check, tool); pushes against this story (but).
Same source domain / Shared topic / Tension / Downstream implication
Reported by the same outlet (arxiv.org); overlapping topics (against, check); pushes against this story (against).