Open-weight code models fabricate on 60% of impossible tasks and refuse only 27%.

An adversarial suite of 270 deliberately unsatisfiable prompts across six languages and 24 subcategories, paired with 91 matched solvable controls, judged by a two-tier protocol validated at 82% human agreement and kappa 0.73. Across twelve models and 4,332 judged responses, ungrounded code appeared on about 60% of unsatisfiable prompts, refusal on 27%, and wrongful refusal on 0% of the solvable controls. The failure is one-directional, which means refusal training has room to move without costing valid work. arXiv 2609.03267