Secure Multi-agent Reinforcement Learning for Service Systems with Affinity and Byzantine Nodes: Stability Analysis and Protection Design

We study decentralized multi-agent reinforcement learning (MARL) for networked service systems with affinity in the presence of Byzantine nodes. The way that a server processes a job depends on an affinity state that captures the correlation between the job and the server. Each node learns a local control policy via an actor-critic algorithm with linear function approximation over inherently unbounded space of traffic states, while exchanging parameter information with neighbors through a communication graph. A set of Byzantine agents can exploit the unbounded state space to compromise the consensus mechanism, destabilizing both learning and queuing processes. To address this vulnerability, we propose a resilient consensus-based MARL algorithm, which mitigates adversarial parameter manipulation and guarantees traffic stability under mild assumptions. We prove that the cooperative agents’ policies converge almost surely to a bounded neighborhood of a stationary solution of the global objective. We demonstrate the effectiveness and generality of the proposed framework in several representative service systems, including semantic routing for large language model serving, distributed polling in cloud computing, and smart manufacturing logistics.

Modern service systems rely on many distributed servers that must coordinate under highly dynamic workloads. In practice, some servers become better at handling certain tasks over time because of accumulated experience or cached information. However, if a few servers are compromised and begin sending malicious information, the entire system can become unstable, leading to severe congestion and long delays. We develop a resilient multi-agent reinforcement learning framework that allows distributed servers to learn cooperative decision-making strategies while remaining robust to malicious participants. Each server independently learns how to route or process jobs by interacting with its local environment and exchanging information with neighboring servers. Our method introduces a secure consensus mechanism that filters out corrupted information and a stability-aware learning strategy that prevents workloads from growing uncontrollably, even when system states become very large. Unlike many existing approaches that focus only on learning performance, our framework jointly addresses both learning stability and traffic stability in decentralized service networks. We provide theoretical guarantees showing that the proposed algorithm converges reliably under mild conditions despite adversarial behavior. We further evaluate the framework on several representative applications, including routing requests for large language model services, distributed cloud computing systems, and smart manufacturing logistics. Our results demonstrate that resilient decentralized AI can improve the reliability, scalability, and safety of future intelligent service infrastructures.