Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code (CodeSpear)

arXiv·high signal

CodeSpear shows that simply applying a benign code grammar constraint during decoding can jailbreak LLMs into producing malicious code, raising attack success by more than 30 percentage points on average across 10 popular models. The authors propose CodeShield, which generates harmless honeypot code under grammar constraints while preserving natural-language refusals. A warning for anyone using constrained/structured decoding in code pipelines.