Swapping Hand-Written Mutation Operators for LLM Mutations Cut Valid Simulink Repairs From 16 to 4

arXiv 2608.19347·medium signalXBlueskyLinkedInCopy link

arXiv 2608.19347 (posted 2026-08-19) tests the assumption that LLMs can replace designed mutation operators in search-based automated program repair, extending the state-of-the-art FlowRepair on 19 real faulty Simulink/Stateflow models across four cyber-physical domains under an identical wall-clock budget. LLM-based mutation produced plausible patches for 4-6 models and valid patches for 4, against 18 and 16 for the original operators. The authors attribute the collapse to LLMs struggling with precise symbolic edits, lacking behavioral feedback, and generating a noisy search space that actively hinders exploration, and argue for hybrid designs where structured mutation stays and generation only guides.