Timezone: »
Poster
Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
Tongzhou Wang · Antonio Torralba · Phillip Isola · Amy Zhang
In goal-reaching reinforcement learning (RL), the optimal value function has a particular geometry, called quasimetrics structure. This paper introduces Quasimetric Reinforcement Learning (QRL), a new RL method that utilizes quasimetric models to learn optimal value functions. Distinct from prior approaches, the QRL objective is specifically designed for quasimetrics, and provides strong theoretical recovery guarantees. Empirically, we conduct thorough analyses on a discretized MountainCar environment, identifying properties of QRL and its advantages over alternatives. On offline and online goal-reaching benchmarks, QRL also demonstrates improved sample efficiency and performance, across both state-based and image-based observations.
Author Information
Tongzhou Wang (MIT)
Antonio Torralba (MIT)
Phillip Isola (MIT)
Amy Zhang (UT Austin / FAIR)
More from the Same Authors
-
2020 : Learning Invariant Representations for Reinforcement Learning without Reconstruction »
Amy Zhang -
2020 : Multi-Task Reinforcement Learning as a Hidden-Parameter Block MDP »
Amy Zhang -
2023 : Conditional Bisimulation for Generalization in Reinforcement Learning »
Anuj Mahajan · Amy Zhang -
2023 Workshop: Challenges in Deployable Generative AI »
Swami Sankaranarayanan · Thomas Hartvigsen · Camille Bilodeau · Ryutaro Tanno · Cheng Zhang · Florian Tramer · Phillip Isola -
2023 Poster: Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks »
Minyoung Huh · Brian Cheung · Pulkit Agrawal · Phillip Isola -
2023 Poster: LIV: Language-Image Representations and Rewards for Robotic Control »
Yecheng Jason Ma · Vikash Kumar · Amy Zhang · Osbert Bastani · Dinesh Jayaraman -
2022 : Invited talks 3, Q/A, Amy, Rich and Liting »
Liting Sun · Amy Zhang · Richard Zemel -
2022 : Invited talks 3, Amy Zhang, Rich Zemel and Liting Sun »
Amy Zhang · Richard Zemel · Liting Sun -
2022 Poster: Online Decision Transformer »
Qinqing Zheng · Amy Zhang · Aditya Grover -
2022 Poster: Robust Policy Learning over Multiple Uncertainty Sets »
Annie Xie · Shagun Sodhani · Chelsea Finn · Joelle Pineau · Amy Zhang -
2022 Poster: Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning »
Philippe Hansen-Estruch · Amy Zhang · Ashvin Nair · Patrick Yin · Sergey Levine -
2022 Spotlight: Robust Policy Learning over Multiple Uncertainty Sets »
Annie Xie · Shagun Sodhani · Chelsea Finn · Joelle Pineau · Amy Zhang -
2022 Spotlight: Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning »
Philippe Hansen-Estruch · Amy Zhang · Ashvin Nair · Patrick Yin · Sergey Levine -
2022 Oral: Online Decision Transformer »
Qinqing Zheng · Amy Zhang · Aditya Grover -
2022 Poster: Denoised MDPs: Learning World Models Better Than the World Itself »
Tongzhou Wang · Simon Du · Antonio Torralba · Phillip Isola · Amy Zhang · Yuandong Tian -
2022 Spotlight: Denoised MDPs: Learning World Models Better Than the World Itself »
Tongzhou Wang · Simon Du · Antonio Torralba · Phillip Isola · Amy Zhang · Yuandong Tian -
2020 : Paper spotlight: Learning Invariant Representations for Reinforcement Learning without Reconstruction »
Amy Zhang -
2020 Poster: Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere »
Tongzhou Wang · Phillip Isola -
2020 Poster: Estimating Generalization under Distribution Shifts via Domain-Invariant Representations »
Ching-Yao Chuang · Antonio Torralba · Stefanie Jegelka -
2018 Poster: Composable Planning with Attributes »
Amy Zhang · Sainbayar Sukhbaatar · Adam Lerer · Arthur Szlam · Facebook Rob Fergus -
2018 Oral: Composable Planning with Attributes »
Amy Zhang · Sainbayar Sukhbaatar · Adam Lerer · Arthur Szlam · Facebook Rob Fergus