Timezone: »

 
Oral
AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
Zhixuan Liang · Yao Mu · Mingyu Ding · Fei Ni · Masayoshi Tomizuka · Ping Luo

Tue Jul 25 08:30 PM -- 08:38 PM (PDT) @ Ballroom A
in Oral A4 Diffusion »

Diffusion models have demonstrated their powerful generative capability in many tasks, with great potential to serve as a paradigm for offline reinforcement learning. However, the quality of the diffusion model is limited by the insufficient diversity of training data, which hinders the performance of planning and the generalizability to new tasks. This paper introduces AdaptDiffuser, an evolutionary planning method with diffusion that can self-evolve to improve the diffusion model hence a better planner, not only for seen tasks but can also adapt to unseen tasks. AdaptDiffuser enables the generation of rich synthetic expert data for goal-conditioned tasks using guidance from reward gradients. It then selects high-quality data via a discriminator to finetune the diffusion model, which improves the generalization ability to unseen tasks. Empirical experiments on two benchmark environments and two carefully designed unseen tasks in KUKA industrial robot arm and Maze2D environments demonstrate the effectiveness of AdaptDiffuser. For example, AdaptDiffuser not only outperforms the previous art Diffuser by 20.8% on Maze2D and 7.5% on MuJoCo locomotion, but also adapts better to new tasks, e.g., KUKA pick-and-place, by 27.9% without requiring additional expert data. More visualization results and demo videos could be found on our project page.

[ PDF] 

Author Information

Zhixuan Liang (The University of Hong Kong)
Yao Mu (The University of Hong Kong)

I am currently a Ph.D. Candidate of Computer Science at the University of Hong Kong, supervised by Prof. Ping Luo. Previously I obtained the M.Phil Degree under the supervision of Prof. Bo Cheng and Prof. Shengbo Li at the Intelligent Driving Laboratory from Tsinghua University in June 2021. Research Interests: Reinforcement Learning, Representation Learning, Autonomous Driving, and Computer Vision.

Mingyu Ding (UC Berkeley)
Fei Ni (Tianjin university)
Masayoshi Tomizuka (University of California, Berkeley)
Ping Luo (The University of Hong Kong)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors