Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-07 · high
Scale X's 409,000-decision study found npm run analyze fooled reviewers 64.7% of the time.
Why now: Scale X published the results on August 5, and vendors including AWS and HAR answered within 48 hours.
Reviewers missed 33.7% of malicious commands across 409,000-plus approve/deny decisions in an agent-permission game, per Scale X's August 5 results.
Every team that gates an agent's shell commands behind a human click is trusting that click as the security boundary. Scale X's numbers say it isn't: reviewers missed one in three malicious commands, and 32.9% of sessions ended net harmful, the reviewer approving more damage than they blocked.
The misses aren't spread evenly. Reviewers caught the obviously destructive stuff, missing rm -rf / only 11.7% of the time. The blind spot is routine-looking tooling: npm run analyze got waved through 64.7% of the time, npm run setup 48.0%, npm run deploy 44.9%, a 52.5% miss rate pooled across npm-script variants.
The over-correction is just as bad. 59% of reviewers blocked a harmless npm config set registry command, and 45% blocked rm -rf dist/, a routine cleanup call.
I've clicked through enough of these prompts to know I stop reading carefully by the fortieth one in a session. The data says I'm not the only one.
HAR answered with deterministic validation gates that bind a validated hash to the exact code that passed review. A reviewer checks the hash and the artifacts, not an agent's self-report.
AWS open-sourced Dogwood, Apache 2.0, a Cedar-derived policy language that checks an agent's sequence of prior calls instead of judging one command alone. It runs at the gateway, outside the agent's own code, so prompt injection can't route around it, and it's free even outside Bedrock. It can require a value from one call to match what an earlier call returned, or cut permissions once human oversight ends.
Not everyone drew the same lesson. Microsoft moved Agent Framework Harness and Foundry Hosted Agents to general availability within days, with tool approval on by default, the exact human-in-the-loop check the numbers just measured at a 33.7% failure rate.
Soloop took #2 on Product Hunt that week, marketing itself as approval-first. Rough timing for that pitch.
Each link below shares sources, entities, or timing with this story.
Microsoft uses AWS / Shared entities / Same source / Shared topic
Linked by a graph relationship (Microsoft uses AWS); both cover Agent Framework Harness, AI Observability, Apache, August; cite the same source (Dogwood, Scale X published results, Coldtea.ai).
.NET built by Microsoft / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (.NET built by Microsoft); both cover Agent Framework Harness, Microsoft, NET, OpenTelemetry; overlapping topics (agent, call).
Microsoft released OpenClaw / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Microsoft released OpenClaw); both cover Microsoft, OpenTelemetry, Product Hunt; reported by the same outlet (producthunt.com).
Microsoft uses AWS / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Microsoft uses AWS); both cover Apache, August, AWS, Product Hunt; overlapping topics (agent, approval).
Microsoft supports Agent Skills / Shared entities / Shared topic
Linked by a graph relationship (Microsoft supports Agent Skills); both cover August, AWS, Microsoft, Then; overlapping topics (agent, code).
.NET built by Microsoft / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (.NET built by Microsoft); both cover Microsoft, NET, Python; overlapping topics (agent, approval, command).
Linked by a graph relationship (.NET built by Microsoft); both cover Microsoft, NET, Python; overlapping topics (agent, approval).
Linked by a graph relationship (.NET built by Microsoft); both cover NET, OpenTelemetry, Python; overlapping topics (agent, approval).