Task-CoEvolve Cuts Agent Harness Optimization Cost 80% by Only Evaluating Tasks the Candidates Disagree On

arXiv 2608.20169·medium signalXBlueskyLinkedInCopy link

arXiv 2608.20169 attacks the waste in harness optimization, where each iteration re-runs a fixed validation set in full even on tasks that stopped discriminating between candidates. Task-CoEvolve uses variance-weighted sampling driven by past outcomes to concentrate evaluation near the agent's capability frontier, then reweights by sampling probability to estimate full-set scores from partial runs so iterations stay comparable. On online text classification and Terminal-Bench 2.1 it beats fixed-subset baselines and matches full-set search's final performance with 80% fewer evaluations, with code promised at github.com/Agent4Science-UTokyo/Task-CoEvolve.