Next-Gen CAPTCHAs: Leveraging the Cognitive Gap for Scalable and Diverse GUI-Agent Defense

The rapid evolution of GUI-enabled agents has rendered traditional CAPTCHAs obsolete. While previous benchmarks like OpenCaptchaWorld established a baseline for evaluating multimodal agents, recent advancements in reasoning-heavy models, such as Gemini3-Pro-High and GPT-5.2-Xhigh have effectively collapsed this security barrier, achieving pass rates as high as 90\% on complex logic puzzles like ''Bingo''. In response, we introduce Next-Gen CAPTCHAs, a scalable defense framework designed to secure the next-generation web against the advanced agents. Unlike static datasets, our benchmark is built upon a robust data generation pipeline, allowing for large-scale and easily scalable evaluations, notably, for backend-supported types, our system is capable of generating effectively unbounded CAPTCHA instances. We exploit the persistent human--agent ``Cognitive Gap'' in interactive perception, memory, decision-making, and action. By engineering dynamic tasks that require adaptive intuition rather than granular planning, we re-establish a robust distinction between biological users and artificial agents, offering a scalable and diverse defense mechanism for the agentic era.

Websites use CAPTCHAs to check whether a visitor is a real person or an automated program. However, modern AI agents can now look at webpages, read instructions, click buttons, and solve many older CAPTCHA puzzles. This makes it harder for websites to stop spam, fake accounts, scraping, and other automated abuse. We propose Next-Gen CAPTCHAs, a new set of interactive challenges that are easy for people but difficult for today’s web-browsing AI agents. Instead of making puzzles harder in the usual way, these challenges use tasks where humans and AI agents still behave differently, such as following motion, remembering changing visual information, understanding spatial layouts, and dragging objects to the right place. We also build a web-based test platform to measure how well different AI agents do on these challenges. Our experiments show a large gap between people and current AI agents. People solve the tasks reliably, while strong AI agents still have very low success rates. This work offers a practical direction for keeping CAPTCHA-style defenses useful as AI agents become more capable.