Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents

LLM-based agents are becoming increasingly capable, yet their safety lags behind. This creates a gap between what agents can do and should do. This gap widens as agents engage in multi-turn interactions and employ diverse tools, introducing new risks overlooked by existing benchmarks. To systematically scale safety testing in multi-turn, tool-realistic settings, we propose a principled taxonomy that transforms single-turn harmful tasks into multi-turn attack sequences. Using this taxonomy, we construct MT-AgentRisk (Multi-Turn Agent Risk Benchmark), the first benchmark to evaluate tool-using agent safety under multi-turn, harmful-distribution settings. Our experiments reveal substantial safety degradation: the Attack Success Rate (ASR) increases by 16\% on average across open and closed models in multi-turn settings. To close this gap, we propose ToolShield, a training-free, tool-agnostic, self-exploration defense: when encountering a new tool, the agent autonomously generates test cases, executes them to observe downstream effects, and distills safety experiences for deployment. Experiments show that ToolShield effectively reduces ASR by 30\% on average in multi-turn interactions.

AI agents that can use tools: browsing the web, running terminal commands, editing files, querying databases, are quickly becoming part of everyday work. These tasks can naturally spawn multiple exchanges between the users and the AIs. However, the harmfulness for such agents can also be distributed across multiple turns. To scale agent safety evaluation into multi-turn, we built a systematic way to decompose a harmful task across multiple turns, and used it to create the first benchmark that tests tool-using agents under these realistic, multi-turn, harmful-distribution attacks. The attack success rate increased by 16% on average across the 6 models we tested compared with the original single-turn tasks. To improve safety for agents, we developed a defense that lets an agent safely self-explore a new tool before deployment: it inspects how the tool could be misused, tries those scenarios in a sandbox, and writes down lessons it can later apply. This effectively reduces the multi-turn attack success rate by 30%, with minimal overhead.