Timezone: »
While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is often challenging and requires substantial interaction with the environment. A wide variety of domains have dynamics that share common foundations like the laws of classical mechanics, which are rarely exploited by existing algorithms. In fact, humans continuously acquire and use such dynamics priors to easily adapt to operating in new environments. In this work, we propose an approach to learn task-agnostic dynamics priors from videos and incorporate them into an RL agent. Our method involves pre-training a frame predictor on task-agnostic physics videos to initialize dynamics models (and fine-tune them) for unseen target environments. Our frame prediction architecture, SpatialNet, is designed specifically to capture localized physical phenomena and interactions. Our approach allows for both faster policy learning and convergence to better policies, outperforming competitive approaches on several different environments. We also demonstrate that incorporating this prior allows for more effective transfer between environments.
Author Information
Yilun Du (MIT)
Karthik Narasimhan (Princeton)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: Task-Agnostic Dynamics Priors for Deep Reinforcement Learning »
Tue. Jun 11th 09:25 -- 09:30 PM Room Hall B
More from the Same Authors
-
2022 Poster: Streaming Inference for Infinite Feature Models »
Rylan Schaeffer · Yilun Du · Gabrielle K Liu · Ila R. Fiete -
2022 Poster: Planning with Diffusion for Flexible Behavior Synthesis »
Michael Janner · Yilun Du · Josh Tenenbaum · Sergey Levine -
2022 Spotlight: Streaming Inference for Infinite Feature Models »
Rylan Schaeffer · Yilun Du · Gabrielle K Liu · Ila R. Fiete -
2022 Oral: Planning with Diffusion for Flexible Behavior Synthesis »
Michael Janner · Yilun Du · Josh Tenenbaum · Sergey Levine -
2022 Poster: Learning Iterative Reasoning through Energy Minimization »
Yilun Du · Shuang Li · Josh Tenenbaum · Igor Mordatch -
2022 Spotlight: Learning Iterative Reasoning through Energy Minimization »
Yilun Du · Shuang Li · Josh Tenenbaum · Igor Mordatch -
2021 Poster: Accelerating Safe Reinforcement Learning with Constraint-mismatched Baseline Policies »
Jimmy Yang · Justinian Rosca · Karthik Narasimhan · Peter Ramadge -
2021 Poster: Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning »
Austin W. Hanjie · Victor Zhong · Karthik Narasimhan -
2021 Spotlight: Accelerating Safe Reinforcement Learning with Constraint-mismatched Baseline Policies »
Jimmy Yang · Justinian Rosca · Karthik Narasimhan · Peter Ramadge -
2021 Spotlight: Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning »
Austin W. Hanjie · Victor Zhong · Karthik Narasimhan -
2021 Poster: Improved Contrastive Divergence Training of Energy-Based Models »
Yilun Du · Shuang Li · Josh Tenenbaum · Igor Mordatch -
2021 Spotlight: Improved Contrastive Divergence Training of Energy-Based Models »
Yilun Du · Shuang Li · Josh Tenenbaum · Igor Mordatch -
2020 : Invited Talk: Karthik Narasimhan »
Karthik Narasimhan -
2020 Poster: Calibration, Entropy Rates, and Memory in Language Models »
Mark Braverman · Xinyi Chen · Sham Kakade · Karthik Narasimhan · Cyril Zhang · Yi Zhang