Skip to yearly menu bar Skip to main content


Poster

Distributional Bellman Operators over Mean Embeddings

Li Kevin Wenliang · Gregoire Deletang · Matthew Aitchison · Marcus Hutter · Anian Ruoss · Arthur Gretton · Mark Rowland


Abstract:

We propose a novel algorithmic framework for distributional reinforcement learning, based on learning finite-dimensional mean embeddings of return distributions. The framework reveals a wide variety of new algorithms for dynamic programming and temporal-difference algorithms that rely on the sketch Bellman operator, which updates mean embeddings with simple linear-algebraic computations. We provide asymptotic convergence theory, and examine the empirical performance of the algorithms on a suite of tabular tasks. Further, we show that this approach can be straightforwardly combined with deep reinforcement learning.

Live content is unavailable. Log in and register to view live content