A CAPTCHA Solver That Learns From Every Encounter Recovers From Adversarial Perturbation Every Round for a Year

arXiv 2609.02393·high signalXBlueskyLinkedInCopy link

The system pairs a fine-tuned YOLOv8 detector with an open-weight VLM behind a confidence-based router, running entirely from screenshots and OS input events with no browser automation or DOM access, reaching 85.4% overall and 84.2% macro accuracy across 16 classes — better than either component alone. Every VLM answer doubles as a training label, so the detector picks up categories it was never trained on after one or two encounters with no human annotation. A CAPTCHA operator can perturb images against the publicly released detector and drive its accuracy to 0%, but the perturbations leave the VLM untouched and its labels repair the detector; under a year-long simulated arms race with monthly re-crafted perturbations the solver recovered every round, and a cheap ~70%-accurate open-weight teacher hardened it as well as a perfect oracle.