Skip to yearly menu bar Skip to main content

Workshop: Workshop on Reinforcement Learning Theory

Marginalized Operators for Off-Policy Reinforcement Learning

Yunhao Tang · Mark Rowland · Remi Munos · Michal Valko


In this work, we propose marginalized operators, a new class of off-policy evaluation operators for reinforcement learning. Marginalized operators strictly generalize generic multi-step operators, such as Retrace, as special cases. Marginalized operators also suggest a form of sample-based estimates with potential variance reduction, compared to sample-based estimates of the original multi-step operators. We show that the estimates for marginalized operators can be computed in a scalable way, which also generalizes prior results on marginalized importance sampling as special cases.

Chat is not available.