Timezone: »
Poster
Distributional Reinforcement Learning for Efficient Exploration
Borislav Mavrin · Hengshuai Yao · Linglong Kong · Kaiwen Wu · Yaoliang Yu
In distributional reinforcement learning (RL), the estimated distribution of value functions model both the parametric and intrinsic uncertainties. We propose a novel and efficient exploration method for deep RL that has two components. The first is a decaying schedule to suppress the intrinsic uncertainty. The second is an exploration bonus calculated from the upper quantiles of the learned distribution. In Atari 2600 games, our method achieves 483 % average gain across 49 games in cumulative rewards over QR-DQN. We also compared our algorithm with QR-DQN in a challenging 3D driving simulator (CARLA). Results show that our algorithm achieves nearoptimal safety rewards twice faster than QRDQN.
Author Information
Borislav Mavrin (University of Alberta)
Hengshuai Yao (Huawei Technologies)
Linglong Kong (University of Alberta)
Kaiwen Wu (University of Waterloo)
Yaoliang Yu (University of Waterloo)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: Distributional Reinforcement Learning for Efficient Exploration »
Tue Jun 11th 09:20 -- 09:25 PM Room Room 104
More from the Same Authors
-
2020 Poster: Tails of Lipschitz Triangular Flows »
Priyank Jaini · Ivan Kobyzev · Yaoliang Yu · Marcus Brubaker -
2020 Poster: Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space »
Yingyi Ma · Vignesh Ganapathiraman · Yaoliang Yu · Xinhua Zhang -
2020 Poster: Stronger and Faster Wasserstein Adversarial Attacks »
Kaiwen Wu · Allen Wang · Yaoliang Yu -
2019 Poster: Sum-of-Squares Polynomial Flow »
Priyank Jaini · Kira A. Selby · Yaoliang Yu -
2019 Oral: Sum-of-Squares Polynomial Flow »
Priyank Jaini · Kira A. Selby · Yaoliang Yu -
2018 Poster: Inductive Two-Layer Modeling with Parametric Bregman Transfer »
Vignesh Ganapathiraman · Zhan Shi · Xinhua Zhang · Yaoliang Yu -
2018 Oral: Inductive Two-Layer Modeling with Parametric Bregman Transfer »
Vignesh Ganapathiraman · Zhan Shi · Xinhua Zhang · Yaoliang Yu