GNU Patch Applies 96% of LLM-Generated Kubernetes Diffs and Silently Misapplies One in Seven
Evaluating how an LLM agent should apply a config fix in a GitOps workflow on real Kubernetes manifests, the authors find no text-generation strategy is safe for unattended automation. Under strict patching almost no unified diff applies, but a tolerant tool applies 96% while silently misapplying 14 to 20% with no error signal; full-file rewrite corrupts files on a small model and is non-deterministic on a frontier model, silently dropping a field or editing a neighbor on some runs at O(file size) cost per edit. Their alternative has the model emit only a structured field-change intent, then locates the target scalar's character span via the YAML parser's node position marks and replaces only that span, released as KubeAstra under Apache-2.0.
↳ Follow the thread