Timezone: »
Imitation learning trains control policies by mimicking pre-recorded expert demonstrations. In partially observable settings, imitation policies must rely on observation histories, but many seemingly paradoxical results show better performance for policies that only access the most recent observation. Recent solutions ranging from causal graph learning to deep information bottlenecks have shown promising results, but failed to scale to realistic settings such as visual imitation. We propose a solution that outperforms these prior approaches by upweighting demonstration keyframes corresponding to expert action changepoints. This simple approach easily scales to complex visual imitation settings. Our experimental results demonstrate consistent performance improvements over all baselines on image-based Gym MuJoCo continuous control tasks. Finally, on the CARLA photorealistic vision-based urban driving simulator, we resolve a long-standing issue in behavioral cloning for driving by demonstrating effective imitation from observation histories. Supplementary materials and code at: \url{https://tinyurl.com/imitation-keyframes}.
Author Information
Chuan Wen (Tsinghua University)
Jierui Lin (UT Austin)
Jianing Qian (University of Pennsylvania)
Yang Gao (Tsinghua University)
Dinesh Jayaraman (University of Pennsylvania)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Poster: Keyframe-Focused Visual Imitation Learning »
Wed. Jul 21st 04:00 -- 06:00 AM Room
More from the Same Authors
-
2022 : Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning »
Zhecheng Yuan · Zhecheng Yuan · Zhengrong Xue · Zhengrong Xue · Bo Yuan · Bo Yuan · Xueqian Wang · Xueqian Wang · Yi Wu · Yi Wu · Yang Gao · Yang Gao · Huazhe Xu · Huazhe Xu -
2023 Poster: For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal »
Yingdong Hu · Renhao Wang · Li Li · Yang Gao -
2023 Poster: Policy Contrastive Imitation Learning »
Jialei Huang · Zhao-Heng Yin · Yingdong Hu · Yang Gao -
2023 Poster: LIV: Language-Image Representations and Rewards for Robotic Control »
Yecheng Jason Ma · Vikash Kumar · Amy Zhang · Osbert Bastani · Dinesh Jayaraman -
2022 Poster: Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching »
Yecheng Jason Ma · Andrew Shen · Dinesh Jayaraman · Osbert Bastani -
2022 Poster: Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming »
Chuan Wen · Jianing Qian · Jierui Lin · Jiaye Teng · Dinesh Jayaraman · Yang Gao -
2022 Spotlight: Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching »
Yecheng Jason Ma · Andrew Shen · Dinesh Jayaraman · Osbert Bastani -
2022 Spotlight: Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming »
Chuan Wen · Jianing Qian · Jierui Lin · Jiaye Teng · Dinesh Jayaraman · Yang Gao -
2020 Poster: Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings »
Jesse Zhang · Brian Cheung · Chelsea Finn · Sergey Levine · Dinesh Jayaraman