Timezone: »

 
Marginalized Operators for Off-Policy Reinforcement Learning
Yunhao Tang · Mark Rowland · Remi Munos · Michal Valko

In this work, we propose marginalized operators, a new class of off-policy evaluation operators for reinforcement learning. Marginalized operators strictly generalize generic multi-step operators, such as Retrace, as special cases. Marginalized operators also suggest a form of sample-based estimates with potential variance reduction, compared to sample-based estimates of the original multi-step operators. We show that the estimates for marginalized operators can be computed in a scalable way, which also generalizes prior results on marginalized importance sampling as special cases.

Author Information

Yunhao Tang (Columbia University)
Mark Rowland (DeepMind)
Remi Munos (DeepMind)
Michal Valko (DeepMind / Inria / ENS Paris-Saclay)

More from the Same Authors