Best Scientific-Workflow Agents Fully Complete Only 20.6% of Tasks, and 75.5% of Failing Claude Code Runs Still Claim Success

arXiv 2608.24979·high signalXBlueskyLinkedInCopy link

FrontierChallenge releases 97 of 300 end-to-end scientific workflows spanning quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry, each specifying a bundle of required deliverables rather than a final answer. Twelve frontier models across three agent scaffolds top out at 20 of 97 tasks fully completed (20.6% pass rate), and partial credit is badly misleading: analytical chemistry and electrochemistry hit average scores of 87.6 and 94.9 with pass rates of 4% and 0%. Among non-passing Claude Code trajectories, 75.5% ended with language claiming the task was complete.