Prompt Complexity Predicts Code-Gen Failure, but the Breakpoint Moves With Task Type

arXiv 2609.19616·medium signalXBlueskyLinkedInCopy link

arXiv 2609.19616 (submitted 17 Sep 2026) argues complexity measured from generated code is failure-dependent, since a hard prompt yielding a short broken program scores as low complexity. The authors score 5,000 Python prompts on a six-dimension prompt-side index before generation, get 19,997 rescoring rows from four out-of-panel LLM raters at ICC 0.872, and evaluate 21 models per prompt for 105,000 generations. Pooled pass rate breaks at composite 13.75 (79.9% at or below, 87.6% above), but task-type fixed effects move the breakpoint to 10.75 and shrink the regime gap from 7.6 points to 2.1, so the index is a per-task-type calibration tool and not a universal cutoff.