Iterative LLM Infrastructure-as-Code Repair Silently Breaks Security Checks in 3.3% of Scenarios; Stop at Iteration 3

arXiv 2608.13404·high signalXBlueskyLinkedInCopy link

Analyzing 5,968 IaC-Eval scenario timelines across 15 configurations (4,440 iteration transitions with Checkov data on both sides), the authors track 30 CIS Benchmark check IDs to find cases where a previously-passing check fails after a repair iteration. Under strict detection, 3.3% of scenarios regress (5.2% of transitions); resource restructuring is the dominant root cause at 79.0%, and regressing transitions show 2.6x more code churn (Cohen's d=0.90). 36.6% of standard-mode regressions self-correct within an average of 1.2 iterations, and the paper identifies iteration 3 as the optimal stopping point — concrete iteration-budget guidance for anyone running a validator-in-the-loop repair agent.