One Token Away from Collapse: Simple Lexical Constraints Break Instruction-Tuned LLM Helpfulness

arXiv·high signal

Baghaei Potraghloo et al. demonstrate that banning a single punctuation character or common word causes instruction-tuned LLMs to collapse into unhelpful, unstructured outputs. This reveals that instruction-tuned helpfulness is far more fragile than assumed — alignment doesn't generalize to trivially constrained generation. Practitioners building constrained-output systems (JSON schemas, structured outputs) should be aware their models may degrade unpredictably.