Timezone: »

 
Poster
Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning
Amin Rakhsha · Goran Radanovic · Rati Devidze · Jerry Zhu · Adish Singla

Tue Jul 14 10:00 AM -- 10:45 AM & Tue Jul 14 10:00 PM -- 10:45 PM (PDT) @
in Poster Session 4 »

We study a security threat to reinforcement learning where an attacker poisons the learning environment to force the agent into executing a target policy chosen by the attacker. As a victim, we consider RL agents whose objective is to find a policy that maximizes average reward in undiscounted infinite-horizon problem settings. The attacker can manipulate the rewards or the transition dynamics in the learning environment at training-time and is interested in doing so in a stealthy manner. We propose an optimization framework for finding an \emph{optimal stealthy attack} for different measures of attack cost. We provide sufficient technical conditions under which the attack is feasible and provide lower/upper bounds on the attack cost. We instantiate our attacks in two settings: (i) an \emph{offline} setting where the agent is doing planning in the poisoned environment, and (ii) an \emph{online} setting where the agent is learning a policy using a regret-minimization framework with poisoned feedback. Our results show that the attacker can easily succeed in teaching any target policy to the victim under mild conditions and highlight a significant security threat to reinforcement learning agents in practice.

Keywords: Reinforcement Learning · Other · Reinforcement Learning Theory · Reinforcement Learning - General

Author Information

Amin Rakhsha (Max Planck Institute for Software Systems (MPI-SWS))
Goran Radanovic (Max Planck Institute for Software Systems)
Rati Devidze (Max Planck Institute for Software Systems)
Jerry Zhu (University of Wisconsin-Madison)
Adish Singla (Max Planck Institute (MPI-SWS))

More from the Same Authors