16% of 3,171 public agent-harness setups carry a confirmed security defect, and 3.8% ship a skill that pre-approves your shell

arXiv·high signalXBlueskyLinkedInCopy link

A study of 3,171 public GitHub repositories (2,660 multi-component agent setups, 511 published skill collections) measured only byte-decidable defects in Claude Code, Cursor, Copilot and Codex configuration artifacts, validating every finding through an independent re-derivation, an LLM adjudicator and a second model session. Three security classes survived: 9.8% of setups install an MCP server with no version pinned, 3.1% pre-approve arbitrary execution behind a scoped-looking grant like Bash(python:*), and 3.8% carry a skill that pre-approves the shell for whoever installs it. Raw scanner rate was 25.5% against a confirmed 16.0%, so a marketplace scan that skips validation overstates by more than half; no credential-exfiltration path was confirmed.