Token cost per task varies sharply by programming language — and agents secretly prototype in Python to escape unfamiliar ones

arXiv 2607.22807·medium signalXBlueskyLinkedInCopy link

arXiv 2607.22807 (July 24, Wu/Anderson/Guha) controlled for problem difficulty across Python, Java, Rust, and OCaml and found stark, model-consistent variation in how many tokens a coding agent burns per task. The behavioral explanations are the useful part: agents produce non-compiling solutions more often in unfamiliar languages then over-revise already-passing code, they use code comments as a planning scratchpad, they distrust supplied test cases and invent their own inputs, and they sidestep unfamiliar target languages by prototyping in Python first. Practical takeaway — budget agent spend per-language, and supply trusted test harnesses explicitly since agents will otherwise ignore the ones you gave them.