The best LLM catches 47% of expert-identified requirement defects while false-flagging 11%.

First benchmark of off-the-shelf LLMs against expert-derived ground truth built on INCOSE criteria, ten models across two families and five generations each, one hundred independent runs, two requirement sets, five temperatures. The error profile is asymmetric, and performance degrades exactly where systems-engineering judgment is needed: necessity and correctness issues are almost always missed. First-pass filter, not a review-cycle replacement. arXiv 2609.03230