Computer Anthology: A Continuously Evolving Benchmark Family for Computer-Use Agents
Vetto / Hacker News·low signal
Posted to HN (27 points), Computer Anthology proposes terminal-task benchmarks that keep evolving rather than freezing, explicitly targeting the saturation-and-contamination problem that makes static agent benchmarks stop discriminating within months of release. The design bet is that a benchmark which regenerates tasks is harder to overfit than one with a fixed test set. Single-source and early — no independent replication of its numbers yet, so treat the framing as more interesting than the leaderboard.