Timezone: »

 
Poster
The Value Function Polytope in Reinforcement Learning
Robert Dadashi · Marc Bellemare · Adrien Ali Taiga · Nicolas Le Roux · Dale Schuurmans

Tue Jun 11 06:30 PM -- 09:00 PM (PDT) @ Pacific Ballroom #119
in Posters Tue »

We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of the nature of its shape: a general polytope (Aigner et al., 2010). To demonstrate this result, we exhibit several properties of the structural relationship between policies and value functions including the line theorem, which shows that the value functions of policies constrained on all but one state describe a line segment. Finally, we use this novel perspective and introduce visualizations to enhance the understanding of the dynamics of reinforcement learning algorithms.

Keywords: Planning and Control · Theory and Algorithms

Author Information

Robert Dadashi (Google AI Residency Program)
Marc Bellemare (Google Brain)
Adrien Ali Taiga (Université de Montréal)
Nicolas Le Roux (Google)
Dale Schuurmans (Google / University of Alberta)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors