Timezone: »
It has been a challenge to learning skills for an agent from long-horizon unannotated demonstrations. Existing approaches like Hierarchical Imitation Learning(HIL) are prone to compounding errors or suboptimal solutions. In this paper, we propose Option-GAIL, a novel method to learn skills at long horizon. The key idea of Option-GAIL is modeling the task hierarchy by options and train the policy via generative adversarial optimization. In particular, we propose an Expectation-Maximization(EM)-style algorithm: an E-step that samples the options of expert conditioned on the current learned policy, and an M-step that updates the low- and high-level policies of agent simultaneously to minimize the newly proposed option-occupancy measurement between the expert and the agent. We theoretically prove the convergence of the proposed algorithm. Experiments show that Option-GAIL outperforms other counterparts consistently across a variety of tasks.
Author Information
Mingxuan Jing (Tsinghua University)
Wenbing Huang (Tsinghua University)
Fuchun Sun (Tsinghua)
Xiaojian Ma (University of California, Los Angeles)
Tao Kong (Bytedance)
Chuang Gan (MIT-IBM Watson AI Lab)
Lei Li (ByteDance AI Lab)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Adversarial Option-Aware Hierarchical Imitation Learning »
Wed. Jul 21st 02:40 -- 02:45 PM Room
More from the Same Authors
-
2023 Poster: Reparameterized Policy Learning for Multimodal Trajectory Optimization »
Zhiao Huang · Litian Liang · Zhan Ling · Xuanlin Li · Chuang Gan · Hao Su -
2023 Poster: On the Forward Invariance of Neural ODEs »
Wei Xiao · Johnson Tsun-Hsuan Wang · Ramin Hasani · Mathias Lechner · Yutong Ban · Chuang Gan · Daniela Rus -
2023 Poster: End-to-End Full-Atom Antibody Design »
Xiangzhe Kong · Wenbing Huang · Yang Liu -
2023 Poster: Subequivariant Graph Reinforcement Learning in 3D Environments »
Runfa Chen · Jiaqi Han · Fuchun Sun · Wenbing Huang -
2023 Oral: Subequivariant Graph Reinforcement Learning in 3D Environments »
Runfa Chen · Jiaqi Han · Fuchun Sun · Wenbing Huang -
2023 Oral: Reparameterized Policy Learning for Multimodal Trajectory Optimization »
Zhiao Huang · Litian Liang · Zhan Ling · Xuanlin Li · Chuang Gan · Hao Su -
2023 Poster: Learning Neural Constitutive Laws from Motion Observations for Generalizable PDE Dynamics »
Pingchuan Ma · Peter Yichen Chen · Bolei Deng · Josh Tenenbaum · Tao Du · Chuang Gan · Wojciech Matusik -
2022 Poster: Prompting Decision Transformer for Few-Shot Policy Generalization »
Mengdi Xu · Yikang Shen · Shun Zhang · Yuchen Lu · Ding Zhao · Josh Tenenbaum · Chuang Gan -
2022 Spotlight: Prompting Decision Transformer for Few-Shot Policy Generalization »
Mengdi Xu · Yikang Shen · Shun Zhang · Yuchen Lu · Ding Zhao · Josh Tenenbaum · Chuang Gan -
2021 Poster: Global Prosody Style Transfer Without Text Transcriptions »
Kaizhi Qian · Yang Zhang · Shiyu Chang · Jinjun Xiong · Chuang Gan · David Cox · Mark Hasegawa-Johnson -
2021 Oral: Global Prosody Style Transfer Without Text Transcriptions »
Kaizhi Qian · Yang Zhang · Shiyu Chang · Jinjun Xiong · Chuang Gan · David Cox · Mark Hasegawa-Johnson -
2021 Poster: AGENT: A Benchmark for Core Psychological Reasoning »
Tianmin Shu · Abhishek Bhandwaldar · Chuang Gan · Kevin Smith · Shari Liu · Dan Gutfreund · Elizabeth Spelke · Josh Tenenbaum · Tomer Ullman -
2021 Spotlight: AGENT: A Benchmark for Core Psychological Reasoning »
Tianmin Shu · Abhishek Bhandwaldar · Chuang Gan · Kevin Smith · Shari Liu · Dan Gutfreund · Elizabeth Spelke · Josh Tenenbaum · Tomer Ullman -
2020 Poster: Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation »
Wenxian Shi · Hao Zhou · Ning Miao · Lei Li -
2019 Poster: Neural Collaborative Subspace Clustering »
Tong Zhang · Pan Ji · Mehrtash Harandi · Wenbing Huang · HONGDONG LI -
2019 Oral: Neural Collaborative Subspace Clustering »
Tong Zhang · Pan Ji · Mehrtash Harandi · Wenbing Huang · HONGDONG LI