Giving a coding agent a boot probe kills nearly every launch failure at a third of the cost of a shell

arXiv 2608.28795·high signalXBlueskyLinkedInCopy link

A controlled study built a minimal coding agent and ran it across 1,116 web applications with six models and eight tool configurations, holding everything else constant. Without verification tools roughly 1 in 7 apps failed to launch; adding a boot probe eliminated nearly all launch failures at about 35% of the token cost of a full shell, while the full shell cost 2.35x the no-tools baseline. Screenshots added nothing for non-visual failures like scroll performance on a 100,000-row list, so the practical rule is to match each verification tool to a failure mode you actually have rather than granting the agent everything.