Timezone: »

 
Poster
Temporal Difference Learning for Model Predictive Control
Nicklas Hansen · Hao Su · Xiaolong Wang

Thu Jul 21 03:00 PM -- 05:00 PM (PDT) @ Hall E #910
in Poster Session 3 »

Data-driven model predictive control has two key advantages over model-free methods: a potential for improved sample efficiency through model learning, and better performance as computational budget for planning increases. However, it is both costly to plan over long horizons and challenging to obtain an accurate model of the environment. In this work, we combine the strengths of model-free and model-based methods. We use a learned task-oriented latent dynamics model for local trajectory optimization over a short horizon, and use a learned terminal value function to estimate long-term return, both of which are learned jointly by temporal difference learning. Our method, TD-MPC, achieves superior sample efficiency and asymptotic performance over prior work on both state and image-based continuous control tasks from DMControl and Meta-World. Code and videos are available at https://nicklashansen.github.io/td-mpc.

Keywords: RL: Continuous Action · RL: Deep RL · RL: Online · RL: Planning
[ Slides]  [ Poster]  [ Paper PDF] 

Author Information

Nicklas Hansen (University of California, San Diego)
Hao Su (UCSD)
Xiaolong Wang (UCSD)
Xiaolong Wang

Our group has a broad interest around the directions of Computer Vision, Machine Learning and Robotics. Our focus is on learning 3D and dynamics representations through videos and physical robotic interaction data. We explore various means of supervision signals from the data itself, language, and common sense knowledge. We leverage these comprehensive representations to facilitate the learning of robot skills, with the goal of generalizing the robot to interact effectively with a wide range of objects and environments in the real physical world. Please check out our individual research topic of Self-Supervised Learning, Video Understanding, Common Sense Reasoning, RL and Robotics, 3D Interaction, Dexterous Hand.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors