Learning Robust Multi-Agent Policies via Selective Adversarial Fault Induction

We study robustness to agent malfunctions in cooperative multi-agent reinforcement learning (MARL), a failure mode that is critical in practice yet underexplored in existing theory. We introduce MARTA, a plug-and-play robustness layer that augments standard MARL algorithms with a Switcher–Adversary mechanism which selectively induces malfunctions in performance-critical states. This formulation defines a fault-switching $(N+2)$-player Markov game in which the Switcher chooses when and which agent fails, and the Adversary controls the resulting faulty behaviour via random or worst-case policies. We develop a Q-learning-type scheme and show that the associated Bellman operator is a contraction, yielding existence and uniqueness of the minimax value, convergence to a Markov perfect equilibrium. MARTA integrates seamlessly with MARL algorithms without architectural modification and consistently improves robustness across Traffic Junction (TJ), Level-Based Foraging (LBF), MPE SimpleTag, and SMAC (v2). In these domains, MARTA achieves large gains in final performance of up to 116.7\% in SMAC, 21.4\% in MPE SimpleTag, and 44.6\% in LBF, while significantly reducing failure rates under train–test mismatched fault regimes. These results establish MARTA as a theoretically grounded and practically deployable mechanism for fault-tolerant MARL.

This paper studies how to make teams of AI agents more reliable when some of the agents fail or behave unpredictably. In many real-world systems, such as fleets of robots, autonomous vehicles, or distributed control systems, AI agents must coordinate closely to complete tasks. However, if even one agent malfunctions, the whole system can break down. Existing approaches often assume that failures happen randomly or constantly, which can make systems overly cautious and inefficient. We introduce MARTA, a new training framework that deliberately injects targeted failures into multi-agent systems during training so the agents learn how to recover from them. MARTA includes a “Switcher” component that learns when and where failures would be most disruptive, and an “Adversary” component that simulates the faulty behaviour. By exposing the agents to difficult but carefully selected failures, the system learns stronger and more reliable coordination strategies. We also provide mathematical guarantees showing that the training process converges to a stable solution under standard assumptions. Across a range of benchmark environments involving traffic coordination, robotic teamwork, pursuit tasks, and StarCraft battles, MARTA consistently improves robustness and reduces failure rates compared to existing methods. Importantly, MARTA can be added to existing multi-agent AI systems without redesigning their architectures, making it practical for real-world deployment in safety-critical settings.