Talk is (Not) Cheap: 932-Paper STRIDE Taxonomy Reveals 75% of LLM Attack Surface Untested by Any Benchmark

arXiv·high signal

Researchers constructed a 4×6 Target×Technique STRIDE matrix from 932 arXiv security studies (2023–2026), producing a 507-leaf taxonomy with 2,521 unique attack groups. Auditing HarmBench, InjecAgent, and AgentDojo against this matrix reveals the three benchmarks occupy non-overlapping cells covering at most 25% of the threat surface. Two entire STRIDE categories — Service Disruption and Model Internals — lack any standardized evaluation despite published attacks achieving 46× token amplification and 96% success rates.