Long-Horizon-Terminal-Bench: Best Frontier Agent Clears Just 15% of Real Terminal Tasks, Mean 4.3%

arXiv (Tencent Hunyuan) / HuggingFace Daily Papers·high signal

Tencent Hunyuan's new benchmark (arXiv:2607.08964, HuggingFace Daily Papers #1 today with 41 votes) puts 15 frontier models against 46 long-horizon terminal tasks across nine categories — experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing — using dense reward-based grading that scores partial progress via graded subtasks rather than pass/fail. The strongest model reached only 15.2% pass@1 at a 0.95 partial-reward threshold (10.9% at perfect 1.0), with the cross-model mean at 4.3% and 1.7%. Tasks are brutally expensive — roughly 9.9M tokens, 231 episodes, and 85 minutes of execution each — a sobering reality check for anyone shipping autonomous CLI agents.