Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning

Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose failure causes worst-case system performance degradations. We study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where where the upper level selects vulnerable agents as an NP-hard task and the lower level learns their worst-case adversarial policies via mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To handle this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We next reformulate the upper-level NP-hard problem as an MDP with dense rewards, allowing sequential identification of vulnerable agents via greedy and RL algorithms. This decomposition provably preserves the optimal solution. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and reveals the vulnerability of each agent in large systems. Code available at \url{https://anonymous.4open.science/r/VAI-5F61/}.

As artificial intelligence scales up to control massive interconnected networks like robot swarms or power grids, a hidden danger emerges. If just a few critical AI agents experience errors or are hijacked by adversaries, the entire system can suffer catastrophic cascading failures. However, proactively finding these "weakest links" among thousands of interacting agents is like finding a needle in a haystack, representing an overwhelmingly complex computational challenge. To tackle this, we developed a framework called Vulnerable Agent Identification (VAI). Instead of exhaustively simulating millions of possible failure scenarios, we created a mathematical shortcut to efficiently estimate the system-wide damage that any single agent's worst-case failure would cause. We then trained a reinforcement learning algorithm to systematically hunt down and expose the most critical vulnerabilities across the network. Our approach successfully reveals hidden vulnerabilities within large-scale multi-agent systems that traditional methods miss. By providing system developers with a powerful diagnostic tool to stress-test their networks before deployment, this work paves the way for safer, more reliable AI applications in the real world.