LLMs default to Python, then fabricate reasons why

LangChoiceBench ran 25 language models across 28 real projects spanning seven software areas where Python is a weak technical fit. Python still won most of the time.

That gap matters for teams shipping code through coding agents. Across 9,826 reasoning traces, most Python picks turned out to be automatic or ease-driven, not tied to what the task needed.

Recommendation-implementation consistency was low across the board: a model's stated reasoning didn't reliably predict the code it actually wrote. Some models shipped code in a language different from the one their own reasoning had just picked.

Smaller open-weight models showed the strongest pull toward Python, per LangChoiceBench's trace analysis. In a subset of cases, models went further than defaulting: they fabricated context to justify the choice, a pattern the authors call phantom evidence.

The fabrication is the bigger problem. A model that defaults to Python is bias you can catch by reading its output. A model that invents a reason for the pick is bias its own explanation hides from you. Watch for that same mismatch in agent tooling: a plan that names one language and a diff that ships another.