A code veto lifts AI edit accuracy to 98%

Deterministic code, not the language model, decided every accept and reject in a new AI-editing benchmark, per arXiv 2609.19710. That matters for anyone shipping a feature that has to land inside a hard numeric constraint, like a word count or a readability score. Asking the model to hit it directly worked only 21.1% to 31.6% of the time across 114 test runs.

The setup keeps the LLM on drafting and revising only. A five-stage loop calls the model to write and edit. Separate code compares a composite score against the target. It rejects any edit that drops a source entity, a number, or a keyword the draft was supposed to keep.

Across 240 closed-loop trials on the same four models, accuracy ranged from 92.5% to 98.8%, converging within two edit rounds on average.

Four models failed single-shot prompting at similar rates and recovered once code, not the model, owned the accept decision. Building a checker that can say no, and looping the model against it, is what worked here. The study doesn't say whether the same design holds for targets that aren't cleanly numeric, like tone or factual accuracy. A composite score is harder to define for those cases.