Timezone: »

 
Oral
Statistics and Samples in Distributional Reinforcement Learning
Mark Rowland · Robert Dadashi · Saurabh Kumar · Remi Munos · Marc Bellemare · Will Dabney

Wed Jun 12 04:25 PM -- 04:30 PM (PDT) @ Room 104
in Reinforcement Learning Theory »

We present a unifying framework for designing and analysing distributional reinforcement learning (DRL) algorithms in terms of recursively estimating statistics of the return distribution. Our key insight is that DRL algorithms can be decomposed as the combination of some statistical estimator and a method for imputing a return distribution consistent with that set of statistics. With this new understanding, we are able to provide improved analyses of existing DRL algorithms as well as construct a new algorithm (EDRL) based upon estimation of the expectiles of the return distribution. We compare EDRL with existing methods on a variety of MDPs to illustrate concrete aspects of our analysis.

[ Slides]  [ Video] 

Author Information

Mark Rowland (DeepMind)
Robert Dadashi (Google AI Residency Program)
Saurabh Kumar (Google)
Remi Munos (DeepMind)
Marc Bellemare (Google Brain)
Will Dabney (DeepMind)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors