Bot Detectors Miss 39% of AI Browser Agents

Binary bot detectors misclassify up to 39% of AI browser agents as human, according to a new study posted to arXiv. An MLP classifier misrouted 39.1% of real agent sessions into the human bucket, and a SAINT transformer still missed 34.5%.

That gap matters for any system using a binary human-or-bot gate to filter automated traffic. Nearly two in five automated sessions in the study passed as a person, per the paper's numbers.

The researchers' fix isn't a smarter binary model. It's adding a third label. Splitting classification into human, bot, and agent categories pushed agent F1 to 1.000 across all 30 test runs.

They stress-tested that fix hard. Against a five-level evasion ladder that included GAN-generated mouse trajectories and replayed human cursor data, the three-class approach caught every agent. Zero misses across 22,990 per-seed predictions.

The tell behind it is structural, not behavioral. An exhaustive search over 9,401 gradient-boosted models found two features, mouse_event_rate and teleport_click_ratio, that deliver 100% agent recall at every evasion level.

Playwright, the automation framework many agents run on, doesn't emit the raw pointer-move and wheel-delta streams a physical mouse produces. Its clicks teleport between points instead of gliding to them.