Timezone: »

Workshop on Multi-Task and Lifelong Reinforcement Learning
Sarath Chandar · Shagun Sodhani · Khimya Khetarpal · Tom Zahavy · Daniel J. Mankowitz · Shie Mannor · Balaraman Ravindran · Doina Precup · Chelsea Finn · Abhishek Gupta · Amy Zhang · Kyunghyun Cho · Andrei A Rusu · Facebook Rob Fergus

Sat Jun 15 08:30 AM -- 06:00 PM (PDT) @ 102
Event URL: https://sites.google.com/view/mtlrl/ »

Website link: https://sites.google.com/view/mtlrl/

Significant progress has been made in reinforcement learning, enabling agents to accomplish complex tasks such as Atari games, robotic manipulation, simulated locomotion, and Go. These successes have stemmed from the core reinforcement learning formulation of learning a single policy or value function from scratch. However, reinforcement learning has proven challenging to scale to many practical real world problems due to problems in learning efficiency and objective specification, among many others. Recently, there has been emerging interest and research in leveraging structure and information across multiple reinforcement learning tasks to more efficiently and effectively learn complex behaviors. This includes:

1. curriculum and lifelong learning, where the problem requires learning a sequence of tasks, leveraging their shared structure to enable knowledge transfer
2. goal-conditioned reinforcement learning techniques that leverage the structure of the provided goal space to learn many tasks significantly faster
3. meta-learning methods that aim to learn efficient learning algorithms that can learn new tasks quickly
4. hierarchical reinforcement learning, where the reinforcement learning problem might entail a compositions of subgoals or subtasks with shared structure

Multi-task and lifelong reinforcement learning has the potential to alter the paradigm of traditional reinforcement learning, to provide more practical and diverse sources of supervision, while helping overcome many challenges associated with reinforcement learning, such as exploration, sample efficiency and credit assignment. However, the field of multi-task and lifelong reinforcement learning is still young, with many more developments needed in terms of problem formulation, algorithmic and theoretical advances as well as better benchmarking and evaluation.

The focus of this workshop will be on both the algorithmic and theoretical foundations of multi-task and lifelong reinforcement learning as well as the practical challenges associated with building multi-tasking agents and lifelong learning benchmarks. Our goal is to bring together researchers that study different problem domains (such as games, robotics, language, and so forth), different optimization approaches (deep learning, evolutionary algorithms, model-based control, etc.), and different formalisms (as mentioned above) to discuss the frontiers, open problems and meaningful next steps in multi-task and lifelong reinforcement learning.

Sat 8:45 a.m. - 9:00 a.m.
Opening Remarks [ Video
Sat 9:00 a.m. - 9:25 a.m.
Sergey Levine: Unsupervised Reinforcement Learning and Meta-Learning (Invited talk) [ Video
Sergey Levine
Sat 9:25 a.m. - 9:50 a.m.
Spotlight Presentations (Spotlights) [ Video
Sat 9:50 a.m. - 10:15 a.m.
Peter Stone: Learning Curricula for Transfer Learning in RL (Invited talk) [ Video
Peter Stone
Sat 10:15 a.m. - 10:30 a.m.
[ Video

10:15 Meta-Learning via Learned Loss Yevgen Chebotar, Artem Molchanov, Sarah Bechtle*, Ludovic Righetti, Franziska Meier, Gaurav Sukhatme 10:23 MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies Xue Bin Peng, Michael Chang, Grace Zhang Pieter Abbeel, Sergey Levine

Sat 10:30 a.m. - 11:00 a.m.
Posters and Break (Poster Session and Break)
Sat 11:00 a.m. - 11:25 a.m.
Jacob Andreas: Linguistic Scaffolds for Policy Learning (Invited talk) [ Video
Jacob Andreas
Sat 11:25 a.m. - 11:50 a.m.
Karol Hausman: Skill Representation and Supervision in Multi-Task Reinforcement Learning (Invited talk) [ Video
Karol Hausman
Sat 11:50 a.m. - 12:20 p.m.
[ Video

11:50 Which Tasks Should Be Learned Together in Multi-task Learning? Trevor Standley, Amir R. Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, Silvio Savarese 11:58 Finetuning Subpolicies for Hierarchical Reinforcement Learning Carlos Florensa, Alexander Li and Pieter Abbeel 12:06 Online Learning for Auxiliary Task Weighting for Reinforcement Learning Xingyu Lin, Harjatin Singh Baweja, George Kantor, David Held 12:14 Guided Meta-Policy Search Russell Mendonca, Abhishek Gupta, Rosen Kralev, Pieter Abbeel, Sergey Levine, Chelsea Finn

Sat 12:20 p.m. - 2:00 p.m.
Posters and Lunch Break (Poster Session and Lunch Break)
Sat 2:00 p.m. - 2:25 p.m.
Martha White: Learning Representations for Continual Learning (Invited talk) [ Video
Martha White
Sat 2:25 p.m. - 2:50 p.m.
Natalia Diaz-Rodriguez: Continual Learning and Robotics: an overview (Invited talk) [ Video
Natalia Diaz Rodriguez
Sat 2:50 p.m. - 3:30 p.m.
Posters and Break (Poster Session and Break)
Sat 3:30 p.m. - 3:55 p.m.
Jeff Clune: Towards Solving Catastrophic Forgetting with Neuromodulation & Learning Curricula by Generating Environments (Invited talk) [ Video
Jeff Clune
Sat 3:55 p.m. - 4:15 p.m.
[ Video

03:55 Online Continual Learning with Maximally Inferred Retrieval Rahaf Alundji, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, Tinne Tuytelaars 04:05 Skew-Fit: State-Covering Self-Supervised Reinforcement Learning Vitchyr H. Pong , Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, Sergey Levine

Sat 4:15 p.m. - 4:40 p.m.
Nicolas Heess: TBD (Invited talk) [ Video
Nicolas Heess
Sat 4:40 p.m. - 5:05 p.m.
Benjamin Rosman: Exploiting Structure For Accelerating Reinforcement Learning (Invited talk) [ Video
Benjamin Rosman
Sat 5:05 p.m. - 6:00 p.m.
Panel Discussion [ Video

Author Information

Sarath Chandar (Mila / University of Montreal)
Shagun Sodhani (University of Montreal)
Khimya Khetarpal (McGill University, Reasoning and Learning Lab)

Ph.D. Student

Tom Zahavy (Technion)
Daniel J. Mankowitz (Deepmind)
Shie Mannor (Technion)
Balaraman Ravindran (Indian Institute of Technology)
Doina Precup (McGill University / DeepMind)
Chelsea Finn (Stanford, Google, UC Berkeley)
Chelsea Finn

Chelsea Finn is an Assistant Professor in Computer Science and Electrical Engineering at Stanford University. Finn's research interests lie in the capability of robots and other agents to develop broadly intelligent behavior through learning and interaction. To this end, her work has included deep learning algorithms for concurrently learning visual perception and control in robotic manipulation skills, inverse reinforcement methods for learning reward functions underlying behavior, and meta-learning algorithms that can enable fast, few-shot adaptation in both visual perception and deep reinforcement learning. Finn received her Bachelor's degree in Electrical Engineering and Computer Science at MIT and her PhD in Computer Science at UC Berkeley. Her research has been recognized through the ACM doctoral dissertation award, the Microsoft Research Faculty Fellowship, the C.V. Ramamoorthy Distinguished Research Award, and the MIT Technology Review 35 under 35 Award, and her work has been covered by various media outlets, including the New York Times, Wired, and Bloomberg. Throughout her career, she has sought to increase the representation of underrepresented minorities within CS and AI by developing an AI outreach camp at Berkeley for underprivileged high school students, a mentoring program for underrepresented undergraduates across four universities, and leading efforts within the WiML and Berkeley WiCSE communities of women researchers.

Abhishek Gupta (UC Berkeley)
Amy Zhang (McGill University)
Kyunghyun Cho (New York University)
Andrei A Rusu (DeepMind)
Facebook Rob Fergus (Facebook AI Research, NYU)

More from the Same Authors