Pattern: agent-built research software fails silently rather than crashing, and that is where the engineering budget goes

arXiv·medium signalXBlueskyLinkedInCopy link

arXiv 2609.04711 (4 September) tracks a repository catalog from a three-day agent-built hackathon prototype through to public deployment. Implementation was fast; making it trustworthy was not. The most consequential problems were not crashes but silent failures producing plausible but incomplete or wrong output, traced to incomplete data acquisition, misleading self-assessment, and retrieval or preprocessing failures. The fix list was adversarial review, data-quality checks, browser-level validation, and publication safeguards, none of which the agent produced on its own.