Timezone: »
We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data. CompILE uses a novel unsupervised, fully-differentiable sequence segmentation module to learn latent encodings of sequential data that can be re-composed and executed to perform new tasks. Once trained, our model generalizes to sequences of longer length and from environment instances not seen during training. We evaluate CompILE in a challenging 2D multi-task environment and a continuous control task, and show that it can find correct task boundaries and event encodings in an unsupervised manner. Latent codes and associated behavior policies discovered by CompILE can be used by a hierarchical agent, where the high-level policy selects actions in the latent code space, and the low-level, task-specific policies are simply the learned decoders. We found that our CompILE-based agent could learn given only sparse rewards, where agents without task-specific policies struggle.
Author Information
Thomas Kipf (University of Amsterdam)
Yujia Li (DeepMind)
Hanjun Dai (Georgia Tech)
Vinicius Zambaldi (Deepmind)
Alvaro Sanchez-Gonzalez (DeepMind)
Edward Grefenstette (Facebook AI Research / UCL)
Pushmeet Kohli (DeepMind)
Peter Battaglia (DeepMind)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Poster: CompILE: Compositional Imitation Learning and Execution »
Thu. Jun 13th 01:30 -- 04:00 AM Room Pacific Ballroom #56
More from the Same Authors
-
2022 : MultiScale MeshGraphNets »
Meire Fortunato · Tobias Pfaff · Peter Wirnsberger · Alexander Pritzel · Peter Battaglia -
2023 : Diffusion Generative Inverse Design »
Marin Vlastelica · Tatiana Lopez-Guevara · Kelsey Allen · Peter Battaglia · Arnaud Doucet · Kimberly Stachenfeld -
2023 : Do LLMs selectively encode the goal of an agent's reach? »
Laura Ruis · Arduin Findeis · Herbie Bradley · Hossein A. Rahmani · Kyoung Whan Choe · Edward Grefenstette · Tim Rocktäschel -
2023 Poster: Transformers Meet Directed Graphs »
Simon Markus Geisler · Yujia Li · Daniel Mankowitz · Taylan Cemgil · Stephan Günnemann · Cosmin Paduraru -
2022 Poster: Evolving Curricula with Regret-Based Environment Design »
Jack Parker-Holder · Minqi Jiang · Michael Dennis · Mikayel Samvelyan · Jakob Foerster · Edward Grefenstette · Tim Rocktäschel -
2022 Poster: Constraint-based graph network simulator »
Yulia Rubanova · Alvaro Sanchez-Gonzalez · Tobias Pfaff · Peter Battaglia -
2022 Spotlight: Constraint-based graph network simulator »
Yulia Rubanova · Alvaro Sanchez-Gonzalez · Tobias Pfaff · Peter Battaglia -
2022 Spotlight: Evolving Curricula with Regret-Based Environment Design »
Jack Parker-Holder · Minqi Jiang · Michael Dennis · Mikayel Samvelyan · Jakob Foerster · Edward Grefenstette · Tim Rocktäschel -
2021 Poster: Generating images with sparse representations »
Charlie Nash · Jacob Menick · Sander Dieleman · Peter Battaglia -
2021 Poster: Prioritized Level Replay »
Minqi Jiang · Edward Grefenstette · Tim Rocktäschel -
2021 Spotlight: Prioritized Level Replay »
Minqi Jiang · Edward Grefenstette · Tim Rocktäschel -
2021 Oral: Generating images with sparse representations »
Charlie Nash · Jacob Menick · Sander Dieleman · Peter Battaglia -
2020 : Invited Talk: Peter Battaglia (Q&A) »
Peter Battaglia -
2020 : Invited Talk: Peter Battaglia »
Peter Battaglia -
2020 Workshop: Bridge Between Perception and Reasoning: Graph Neural Networks & Beyond »
Jian Tang · Le Song · Jure Leskovec · Renjie Liao · Yujia Li · Sanja Fidler · Richard Zemel · Ruslan Salakhutdinov -
2020 Poster: PolyGen: An Autoregressive Generative Model of 3D Meshes »
Charlie Nash · Yaroslav Ganin · S. M. Ali Eslami · Peter Battaglia -
2020 Poster: Learning Reasoning Strategies in End-to-End Differentiable Proving »
Pasquale Minervini · Sebastian Riedel · Pontus Stenetorp · Edward Grefenstette · Tim Rocktäschel -
2020 Poster: Learning to Simulate Complex Physics with Graph Networks »
Alvaro Sanchez-Gonzalez · Jonathan Godwin · Tobias Pfaff · Rex (Zhitao) Ying · Jure Leskovec · Peter Battaglia -
2020 Poster: Scalable Deep Generative Modeling for Sparse Graphs »
Hanjun Dai · Azade Nova · Yujia Li · Bo Dai · Dale Schuurmans -
2019 Workshop: Learning and Reasoning with Graph-Structured Representations »
Ethan Fetaya · Zhiting Hu · Thomas Kipf · Yujia Li · Xiaodan Liang · Renjie Liao · Raquel Urtasun · Hao Wang · Max Welling · Eric Xing · Richard Zemel -
2019 Poster: Structured agents for physical construction »
Victor Bapst · Alvaro Sanchez-Gonzalez · Carl Doersch · Kimberly Stachenfeld · Pushmeet Kohli · Peter Battaglia · Jessica Hamrick -
2019 Oral: Structured agents for physical construction »
Victor Bapst · Alvaro Sanchez-Gonzalez · Carl Doersch · Kimberly Stachenfeld · Pushmeet Kohli · Peter Battaglia · Jessica Hamrick -
2019 Poster: Graph Matching Networks for Learning the Similarity of Graph Structured Objects »
Yujia Li · Chenjie Gu · Thomas Dullien · Oriol Vinyals · Pushmeet Kohli -
2019 Poster: Particle Flow Bayes' Rule »
Xinshi Chen · Hanjun Dai · Le Song -
2019 Oral: Graph Matching Networks for Learning the Similarity of Graph Structured Objects »
Yujia Li · Chenjie Gu · Thomas Dullien · Oriol Vinyals · Pushmeet Kohli -
2019 Oral: Particle Flow Bayes' Rule »
Xinshi Chen · Hanjun Dai · Le Song -
2018 Poster: Adversarial Attack on Graph Structured Data »
Hanjun Dai · Hui Li · Tian Tian · Xin Huang · Lin Wang · Jun Zhu · Le Song -
2018 Oral: Adversarial Attack on Graph Structured Data »
Hanjun Dai · Hui Li · Tian Tian · Xin Huang · Lin Wang · Jun Zhu · Le Song -
2018 Poster: Adversarial Risk and the Dangers of Evaluating Against Weak Attacks »
Jonathan Uesato · Brendan O'Donoghue · Pushmeet Kohli · Aäron van den Oord -
2018 Poster: Programmatically Interpretable Reinforcement Learning »
Abhinav Verma · Vijayaraghavan Murali · Rishabh Singh · Pushmeet Kohli · Swarat Chaudhuri -
2018 Poster: Graph Networks as Learnable Physics Engines for Inference and Control »
Alvaro Sanchez-Gonzalez · Nicolas Heess · Jost Springenberg · Josh Merel · Martin Riedmiller · Raia Hadsell · Peter Battaglia -
2018 Poster: Learning Steady-States of Iterative Algorithms over Graphs »
Hanjun Dai · Zornitsa Kozareva · Bo Dai · Alex Smola · Le Song -
2018 Oral: Adversarial Risk and the Dangers of Evaluating Against Weak Attacks »
Jonathan Uesato · Brendan O'Donoghue · Pushmeet Kohli · Aäron van den Oord -
2018 Oral: Programmatically Interpretable Reinforcement Learning »
Abhinav Verma · Vijayaraghavan Murali · Rishabh Singh · Pushmeet Kohli · Swarat Chaudhuri -
2018 Oral: Learning Steady-States of Iterative Algorithms over Graphs »
Hanjun Dai · Zornitsa Kozareva · Bo Dai · Alex Smola · Le Song -
2018 Oral: Graph Networks as Learnable Physics Engines for Inference and Control »
Alvaro Sanchez-Gonzalez · Nicolas Heess · Jost Springenberg · Josh Merel · Martin Riedmiller · Raia Hadsell · Peter Battaglia -
2017 Poster: Discovering Discrete Latent Topics with Neural Variational Inference »
Yishu Miao · Edward Grefenstette · Phil Blunsom -
2017 Talk: Discovering Discrete Latent Topics with Neural Variational Inference »
Yishu Miao · Edward Grefenstette · Phil Blunsom -
2017 Poster: Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs »
Rakshit Trivedi · Hanjun Dai · Yichen Wang · Le Song -
2017 Talk: Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs »
Rakshit Trivedi · Hanjun Dai · Yichen Wang · Le Song