PLCBench Runs 240 Real-PLC Episodes and Finds 31.3% Sustain Adverse Physical Impact

arXiv 2608.26882·medium signalXBlueskyLinkedInCopy link

arXiv 2608.26882 presents what it describes as the first real-PLC hardware-in-the-loop framework for measuring whether a tool-using LLM agent can convert network-reachable PLC access into sustained physical impact, combining vendor-native interaction, commercial PLC execution, closed-loop reduced-order process simulation and independent outcome verification. Across four commercial PLCs, four closed-loop workloads and five LLM families, 75 of 240 episodes (31.3%) sustain their physical objective; 98 stop before a valid native read and 62 reach a process-linked write without sustaining. Richer process observation raises conditional objective attainment after a process-linked write from 44.2% to 64.0%.