Tool interface design changes coding-agent behavior more than tool capability: 41.6% fewer steps, 56.3% fewer tokens

arXiv 2608.11386·high signalXBlueskyLinkedInCopy link

A study across six tool architectures and 11,700 repository-issue-fixing trajectories found that structured interfaces improved run-to-run consistency by up to 4.7x, natural-language search tools raised relevant-file discovery by over 11%, and Python CodeAct-style interfaces cut steps 41.6% and tokens 56.3% — while cognitive-scaffolding tools like reasoning logs showed almost no effect. The capabilities were held roughly equal; only the interface shape changed. For builders, this says the highest-leverage tuning knob on a coding agent is the tool schema, not the prompt or the scaffold.