Timezone: »
Poster
Why is Posterior Sampling Better than Optimism for Reinforcement Learning?
Ian Osband · Benjamin Van Roy
Computational results demonstrate that posterior sampling for reinforcement learning (PSRL) dramatically outperforms existing algorithms driven by optimism, such as UCRL2. We provide insight into the extent of this performance boost and the phenomenon that drives it. We leverage this insight to establish an $\tilde{O}(H\sqrt{SAT})$ Bayesian regret bound for PSRL in finite-horizon episodic Markov decision processes. This improves upon the best previous Bayesian regret bound of $\tilde{O}(H S \sqrt{AT})$ for any reinforcement learning algorithm. Our theoretical results are supported by extensive empirical evaluation.
[
PDF]
[
Summary/Notes]
Summary/Notes]
Author Information
Ian Osband (Deepmind)
Benjamin Van Roy (Stanford University)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Talk: Why is Posterior Sampling Better than Optimism for Reinforcement Learning? »
Mon. Aug 7th 01:42 -- 02:00 AM Room C4.5
More from the Same Authors
-
2022 : Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learning »
Dilip Arumugam · Benjamin Van Roy -
2021 Poster: Deciding What to Learn: A Rate-Distortion Approach »
Dilip Arumugam · Benjamin Van Roy -
2021 Spotlight: Deciding What to Learn: A Rate-Distortion Approach »
Dilip Arumugam · Benjamin Van Roy -
2018 Poster: The Uncertainty Bellman Equation and Exploration »
Brendan O'Donoghue · Ian Osband · Remi Munos · Vlad Mnih -
2018 Oral: The Uncertainty Bellman Equation and Exploration »
Brendan O'Donoghue · Ian Osband · Remi Munos · Vlad Mnih -
2018 Poster: Coordinated Exploration in Concurrent Reinforcement Learning »
Maria Dimakopoulou · Benjamin Van Roy -
2018 Oral: Coordinated Exploration in Concurrent Reinforcement Learning »
Maria Dimakopoulou · Benjamin Van Roy -
2017 Poster: Minimax Regret Bounds for Reinforcement Learning »
Mohammad Gheshlaghi Azar · Ian Osband · Remi Munos -
2017 Talk: Minimax Regret Bounds for Reinforcement Learning »
Mohammad Gheshlaghi Azar · Ian Osband · Remi Munos