Bash alone beat typed tool catalogs by 21.8 to 24.5 points, on 19-72% fewer tokens

I've spent the better part of a year building typed tool definitions. JSON schemas, argument validation, careful descriptions so the model picks the right one. Everyone I know building agents has done the same thing. MCP made it a standard. A new controlled study says we may have been optimizing the wrong interface the whole time.

Is Bash All You Need? compares five tool interfaces on TheAgentCompany and APEX-Agents with Opus-4.8 and GPT-5.5: typed tools, typed plus bash, bash alone, bash with agent-synthesized persistent tools, and programmatic tool calling where generated programs are restricted to a typed catalog. Bash alone won both benchmarks. On TheAgentCompany it beat the typed catalog by 21.8 to 24.5 percentage points depending on model. On APEX-Agents by 4.8 to 7.4. And it did it with 19-72% fewer total tokens.

Read that cost number again, because it's the part that changes budgets. The typed catalog isn't just losing on quality, it's losing while costing up to three and a half times more. Every tool definition sits in the context window on every turn. A shell is one tool description and the model already knows how shells work from pretraining.

The two additions people would reach for next also failed to help. Layering typed tools on top of bash produced no detectable pooled gain. Neither did letting the agent synthesize persistent tools for itself, which is the pattern a lot of "self-improving agent" writeups are built around. Programmatic tool calling cut tokens against direct typed calls but still generally lost to bash on both axes.

The authors' recommendation is blunt: use bash wherever you can isolate execution, and reserve programmatic tool calling for cases where a fixed, auditable catalog is a compliance requirement. That second clause is doing real work. If you're in a regulated environment where every callable action has to be enumerated in advance, a shell is not available to you and this paper doesn't change that. Everyone else has an easier decision than they thought.

What I'd do with this in a codebase I own: stop writing tool wrappers for things that are already CLIs. gh, psql, curl, jq, your own management commands. The model composes them better than you'll schema them. Where I'd hesitate is the isolation requirement, which the paper treats as an assumption rather than a contribution. Handing a model an unrestricted shell without a sandbox is how you get the incidents the rest of this newsletter is about. Two benchmarks and two models is also not the whole world, and I'd want to see this replicated on repo-shaped work before I tear out a working tool layer.

But the direction is uncomfortable and the numbers aren't close. I was building the expensive version.