Bot Detectors Misclassify 39% of AI Agents as Human

Binary human-or-bot classifiers misclassify roughly two in five real AI browser agent sessions as human, per arXiv 2607.26935 (July 29). An MLP model missed 39.1% of agent traffic. A SAINT transformer missed 34.5%. Anyone screening web traffic for bots is letting a third or more of AI agents through the front door, mislabeled as people.

The researchers tested their classifiers against a five-level evasion ladder, including GAN-generated cursor trajectories and replayed recordings of real human mouse movement. That added up to 2,299 evasion sessions. The binary models kept failing. Then the researchers added a third label, agent, instead of forcing every session into human or bot. Across 30 runs, agent detection F1 hit 1.000.

They also ran an exhaustive search over 9,401 gradient-boosted models to find the minimum signal needed. Two features did it: mouse_event_rate and teleport_click_ratio, both giving 100% agent recall at every evasion level. Zero misses across 22,990 predictions. The reason is mundane. Playwright, the browser automation layer underneath these agents, doesn't generate the raw pointer-move and wheel-delta stream a physical mouse produces. You can fake a trajectory. You can't fake telemetry that was never recorded.

The miss rate isn't the real finding. The binary label was the bug. Feed the same data into a classifier with a third box to check, and detection goes from unreliable to a perfect score. Any bot system still outputting a human/not-human verdict is structurally guaranteed to misroute a third of its agent traffic. It doesn't matter how well the model underneath is tuned. A related report on Cursor, Copilot, and Codex security incidents found the same shape of failure. It traces to how the harness around a model is built, not the model itself.