It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking. Their reliance on dynamic web content, however, makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task. We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), a benchmark for studying how persuasion techniques misguide autonomous web agents on realistic tasks. Across six frontier models, agents are susceptible to prompt injection in 25\% of tasks on average (13\% for GPT-5 to 43\% for DeepSeek-R1), with small interface or contextual changes often doubling success rates and revealing systemic, psychologically driven vulnerabilities in web-based agents. We also provide a modular social-engineering injection framework with controlled experiments on high-fidelity website clones, allowing for further benchmark expansion.

Web-based agents (AI assistants) powered by large language models are increasingly used for tasks such as email management, hotel booking, professional networking etc. The same openness that lets them act on the web also makes them gullible as anyone who can edit a webpage (write a public review, post a comment, send a calendar invite, etc.) can plant hidden instructions that hijack the agent away from its task. We built TRAP, a benchmark that tests how easily six leading models - including GPT, Claude, and Gemini - fall for these traps. Rather than treating each attack as a one-off, we broke them into building blocks - which classic persuasion principle the attacker uses (based on Cialdini's work - authority, scarcity, social proof etc.), which jailbreak technique, what kind of clickable element (button vs hyperlink), and where on the page it sits. We then ran the agents through faithful clones of Gmail, Amazon, LinkedIn, and three other popular sites. Agents took the bait in 25% of cases on average, with the most vulnerable fooled 43% of the time. Small choices matter, e.g. a fake button is over three times more effective than a fake link, and tailoring an injection to mention the user's specific task can multiply success sixfold. Securing web agents is not just a technical problem about prompt handling - it means anticipating the same social-engineering tactics long used against humans.