AIR: Improving Agent Safety through Incident Response

Large Language Model (LLM) agents are increasingly deployed in practice across a wide range of autonomous applications. Yet current safety mechanisms for LLM agents focus almost exclusively on preventing failures in advance, providing limited capabilities for responding to, containing, or recovering from incidents after they inevitably arise. In this work, we introduce AIR, the first incident response framework for LLM agent systems. AIR defines a domain-specific language for managing the incident response lifecycle autonomously in LLM agent systems, and integrates it into the agent's execution loop to (1) detect incidents via semantic checks grounded in the current environment state and recent context, (2) guide the agent to execute containment and recovery actions via its tools, and (3) synthesize guardrail rules during eradication to block similar incidents in future executions. We evaluate AIR on three representative agent types. Results show that AIR achieves detection, remediation, and eradication success rates all exceeding 90%. Extensive experiments further confirm the necessity of AIR's key design components, show the timeliness and moderate overhead of AIR, and demonstrate that LLM-generated rules can approach the effectiveness of developer-authored rules across domains. These results show that incident response is both feasible and essential as a first-class mechanism for improving agent safety.

This paper introduces AIR, a framework designed to help LLM agents respond to safety incidents during execution. Modern LLM agents can use tools, browse websites, and interact with digital or physical environments, but existing safety methods mainly focus on preventing risks in advance. In practice, unsafe behaviors can still occur, and current systems provide limited support for handling them after they happen. AIR enables LLM agents to detect unsafe situations, reduce their impact, recover the environment to a safe state, and prevent similar incidents from happening again. For example, AIR can help an agent identify leaked sensitive files, phishing emails, or dangerous actions in embodied environments, then guide the agent to take corrective actions through its available tools. AIR can also automatically generate new safety rules from previous incidents to improve future protection. We evaluate AIR on three representative agent types: coding agents, embodied agents, and computer-use agents. Experimental results show that AIR can effectively detect and respond to incidents across different environments while introducing only moderate runtime overhead. Overall, this work demonstrates that incident response is both feasible and important for building safer and more reliable LLM agent systems.