Karpathy Declares the Pelican-on-a-Bicycle Test Obsolete After Spending $10 and 1M Reasoning Tokens on a Procedural 3D Lord of the Rings World

X/Twittervia @karpathy·low signalXBlueskyLinkedInCopy link

Karpathy fed Claude Opus the opening paragraph of The Lord of the Rings with a 1 million token reasoning budget (~$10) and asked for a Three.js rendering; the model worked roughly two hours and emitted ~5,500 lines that procedurally generate the scene, placing polygon assets in (x,y,z) and animating them. His framing is that LLM evaluation is 'starting to leave the territory where you'd test an LLM by e.g. create an svg of pelican on a bicycle' — a direct retirement of the benchmark he himself popularized. Dated Aug 2, so older than this cycle's window, but it has not been covered here and the eval-methodology point is the durable part.