MaliciousSkillBench: scanners hold 95.6% recall but flag 62.4% of benign skills from unseen sources.

13 public sources consolidated into 9,740 skills (7,505 malicious, 2,235 benign) across 11 harmonized attack categories. Learned text detectors score 0.882-0.932 Macro-F1 under random splits but collapse to 0.653-0.665 source-disjoint. arXiv Three off-the-shelf skill scanners cut false positives only by giving up most of their malicious recall. Anyone gating a skill marketplace today is choosing between alert fatigue and blind spots, and the random-split numbers everyone quotes are measuring source style, not maliciousness.