Database agent failures mostly start after the first tool call

Three quarters of database agent failures happened only after the model had already called a tool, a study of 8,199 runs found.

For anyone building agent tooling around smaller open-weight models, that reshapes where the debugging effort should go. Transport failures, meaning a tool got called but never returned a deliverable, were the largest failure class at 36.2%. The model failing to call a tool at all, the failure most people assume is happening, made up just 17.3%.

Researchers behind a benchmark of small-model database agents tested 39 locally served open-weight models plus one hosted control. Each ran the agent mode of an open-source SQL client across six task types, logging 110,711 ledger events and 14,008 refused tool calls. Of 2,100 losses attributed to the model, 1,590 of them, 75.7%, came from runs where the agent had already invoked at least one tool. That split held in 99.7% of resampled trials.

The production ledgers recorded refusal codes but never the arguments a model had sent with a tool call. That blind spot hid why the agents were failing for ten days. Once the team started logging arguments, they found five server defects, including one tool that required a field its sibling tool wouldn't allow. Fixing those five server-side bugs, without touching any model, moved the failure numbers.