Silent Tool Failures: 91 Cases in 15 ToolUniverse Scientific Tools Where Calls Return 'Success' With Missing Data
arXiv·low signal
arXiv 2609.26836 (21 Sep) audits agent-to-tool calls that look successful but return incomplete data or functionality without telling the agent or user. Across 15 scientific tools it found 91 manually validated silent failures, mostly missing fields and inconsistent search, filter or ranking behavior, with 51 in the API layer and 25 in the wrapper layer. Wrapper-level schema checks on tool output would catch a large share of these.