Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning

Extensive research has highlighted the severe threats posed by backdoor attacks to deep reinforcement learning (DRL). However, prior studies primarily focus on vanilla scenarios, while plasticity interventions have emerged as indispensable built-in components of modern DRL agents. Despite their effectiveness in mitigating plasticity loss, the impact of these interventions on DRL backdoor vulnerabilities remains underexplored, and this lack of systematic investigation poses risks in practical DRL deployments. To bridge this gap, we empirically study 14,664 cases integrating representative interventions and attack scenarios. We find that only one intervention (i.e., SAM) exacerbates backdoor threats, while other interventions mitigate them. Pathological analysis identifies that the exacerbation is attributed to backdoor gradient amplification, while the mitigation stems from activation pathway disruption and representation space compression. From these findings, we derive two novel insights: (1) a conceptual framework SCC for robust backdoor injection that deconstructs the mechanistic interplay between interventions and backdoors in DRL, and (2) abnormal loss landscape sharpness as a key indicator for DRL backdoor detection.

AI systems are expected to continuously learn and adapt as they interact with the environment. To keep their "brains" flexible, scientists equip them with special adaptability tools. However, a major blind spot remained: do these well-intentioned upgrades accidentally create new security risks or make the AI easier to hack? By putting AI through over 14k attack scenarios, we uncovered an interesting double-edged sword. While most of these adaptability tools actually provide a slight shield against attacks, a few specific ones drastically amplify the danger. Even more shockingly, we found that unfortunate combinations of these tools can make the AI even more vulnerable. This is a wake-up call for the AI industry. It highlights that as developers invent smarter ways for AI to learn, they cannot treat security as an afterthought. It also gives safety experts a new roadmap to catch these evolving threats before continuously learning machines are re deployed in real-world applications.