Timezone: »
While deep reinforcement learning (deep RL) agents are effective at maximizing rewards, it is often unclear what strategies they use to do so. In this paper, we take a step toward explaining deep RL agents through a case study using Atari 2600 environments. In particular, we focus on using saliency maps to understand how an agent learns and executes a policy. We introduce a method for generating useful saliency maps and use it to show 1) what strong agents attend to, 2) whether agents are making decisions for the right or wrong reasons, and 3) how agents evolve during learning. We also test our method on non-expert human subjects and find that it improves their ability to reason about these agents. Overall, our results show that saliency information can provide significant insight into an RL agent's decisions and learning behavior.
Author Information
Samuel Greydanus (Oregon State University)
Anurag Koul (Oregon State University)
Deep Reinforcement Learning + Explainable Artificial Intelligence
Jonathan Dodge (Oregon State University)
Alan Fern (Oregon State University)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Oral: Visualizing and Understanding Atari Agents »
Thu. Jul 12th 12:20 -- 12:30 PM Room A1
More from the Same Authors
-
2021 : RL Explainability & Interpretability Panel »
Ofra Amir · Finale Doshi-Velez · Alan Fern · Zachary Lipton · Omer Gottesman · Niranjani Prasad -
2021 Poster: Re-understanding Finite-State Representations of Recurrent Policy Networks »
Mohamad H Danesh · Anurag Koul · Alan Fern · Saeed Khorram -
2021 Spotlight: Re-understanding Finite-State Representations of Recurrent Policy Networks »
Mohamad H Danesh · Anurag Koul · Alan Fern · Saeed Khorram -
2018 Poster: Open Category Detection with PAC Guarantees »
Si Liu · Risheek Garrepalli · Thomas Dietterich · Alan Fern · Dan Hendrycks -
2018 Oral: Open Category Detection with PAC Guarantees »
Si Liu · Risheek Garrepalli · Thomas Dietterich · Alan Fern · Dan Hendrycks