Epoch and METR's MirrorCode Benchmark: Opus 4.7 Reimplemented a 61K-Line Program in 14 Hours for $251 — Work Estimated at 2–17 Human Weeks

Import AI 466 (Jack Clark)·high signalXBlueskyLinkedInCopy link

Import AI 466 details MirrorCode, a new Epoch/METR benchmark that asks models to re-implement real software from scratch using only CLI access to the original binary's input/output behavior. Across 25 target programs (22 released publicly) spanning six languages and 16k–61k lines, 17/25 hit perfect scores and 21/25 reached 99%+ accuracy, including Apple's pkl configuration language; the hardest holdouts were ruff, giac_subset, and mailauth. The builder takeaway is the cost curve, not the ceiling — gotree was reproduced for $100–400, which reframes 'rewrite this dependency' as a budget line rather than a quarter of engineering.