Timezone: »

 
Poster
Re-understanding Finite-State Representations of Recurrent Policy Networks
Mohamad H Danesh · Anurag Koul · Alan Fern · Saeed Khorram

Tue Jul 20 09:00 PM -- 11:00 PM (PDT) @ Virtual
in Poster Session 2 »

We introduce an approach for understanding control policies represented as recurrent neural networks. Recent work has approached this problem by transforming such recurrent policy networks into finite-state machines (FSM) and then analyzing the equivalent minimized FSM. While this led to interesting insights, the minimization process can obscure a deeper understanding of a machine's operation by merging states that are semantically distinct. To address this issue, we introduce an analysis approach that starts with an unminimized FSM and applies more-interpretable reductions that preserve the key decision points of the policy. We also contribute an attention tool to attain a deeper understanding of the role of observations in the decisions. Our case studies on 7 Atari games and 3 control benchmarks demonstrate that the approach can reveal insights that have not been previously noticed.

Keywords: Fairness, Accountability, and Transparency · Social Aspects of Machine Learning
[ Slides] 

Author Information

Mohamad H Danesh (Oregon State University)
Anurag Koul (Oregon State University)

Deep Reinforcement Learning + Explainable Artificial Intelligence

Alan Fern (Oregon State University)
Saeed Khorram (Oregon State University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors