A CAPTCHA solver keeps healing itself after every attack

A CAPTCHA-solving system keeps recovering from adversarial attacks by turning each attack into a training label, according to a paper posted to arXiv. That undercuts a core assumption behind CAPTCHAs: a provider can outrun a solver by re-crafting the puzzle faster than the solver adapts. It pairs a fine-tuned YOLOv8 detector with an open-weight vision-language model behind a confidence-based router. The whole thing runs from screenshots and OS input events, with no browser automation and no DOM access. Across 16 CAPTCHA classes it reached 85.4% overall accuracy and 84.2% macro accuracy, beating either the detector or the VLM running alone.

When an operator fights back, the split pays off. An operator can perturb images to fool the released detector specifically and drop its accuracy to 0%. Those perturbations don't touch the VLM, though, and every VLM answer doubles as a training label for the detector, including for CAPTCHA categories it was never trained on, after one or two encounters, with no human labeling. In a simulated year-long arms race with monthly re-crafted perturbations, the solver recovered every round.

There's a cost angle too. A cheap, roughly 70%-accurate open-weight model hardened the detector almost as well as a perfect oracle did. The repair loop doesn't need an expensive frontier model watching over it, just something noisy enough to catch the detector's blind spots.