Timezone: »
Exploration is a key component of reinforcement learning (RL). While RL has begun to solve relatively simple tasks, current algorithms cannot complete complex tasks. Our existing algorithms often endlessly dither, failing to meaningfully explore their environments in search of high-reward states. If we hope to have agents autonomously learn increasingly complex tasks, these machines must be equipped with machinery for efficient exploration.
The goal of this workshop is to present and discuss exploration in RL, including deep RL, evolutionary algorithms, real-world applications, and developmental robotics. Invited speakers will share their perspectives on efficient exploration, and researchers will share recent work in spotlight presentations and poster sessions.
Sat 9:00 a.m. - 9:30 a.m.
|
Doina Precup
(Keynote)
Video »
|
|
Sat 9:30 a.m. - 10:00 a.m.
|
Spotlight Talks
Video »
|
|
Sat 10:00 a.m. - 11:00 a.m.
|
Poster Session #1
(Poster Session)
»
This is the first poster session and coffee break. All the papers will be presented at both poster sessions. |
Adrien Ali Taiga, Aniket Deshmukh, Tabish Rashid, Jonathan Binas, Niko Yasui, Vitchyr Pong, Takahisa Imagawa, Jesse Clifton, Sid Mysore, Shi-Chun Tsai, Caleb Chuck, Giulia Vezzani, Hannes Bengt Eriksson
|
Sat 11:00 a.m. - 11:30 a.m.
|
Emo Todorov
(Invited Talk)
Video »
|
|
Sat 11:30 a.m. - 12:00 p.m.
|
Best Paper Talks
Video »
|
|
Sat 12:00 p.m. - 12:30 p.m.
|
Pieter Abbeel
(Invited Talk)
Video »
|
|
Sat 12:30 p.m. - 2:00 p.m.
|
Lunch
|
|
Sat 2:00 p.m. - 2:30 p.m.
|
Raia Hadsell
(Invited Talk)
|
|
Sat 2:30 p.m. - 3:00 p.m.
|
Lightning Talks
|
|
Sat 3:00 p.m. - 4:00 p.m.
|
Poster Session #2
(Poster Session)
»
This is the second poster session and coffee break. All the papers will be presented at both poster sessions. |
|
Sat 4:00 p.m. - 4:30 p.m.
|
Martha White - Adapting Behaviour via Intrinsic Rewards to Learn Predictions
(Invited Talk)
|
|
Sat 4:30 p.m. - 5:30 p.m.
|
Panel Discussion
»
We will have a panel on exploration with panelists Martha White, Jeff Clune, Pulkit Agrawal, and Pieter Abbeel, and moderated by Doina Precup. |
Author Information
Benjamin Eysenbach (CMU, Google Brain)
Benjamin Eysenbach (Google)
Surya Bhupatiraju (Google Brain)
Shixiang Gu (Google)
Harrison Edwards (OpenAI / University of Edinburgh)
Martha White (University of Alberta)
Pierre-Yves Oudeyer (Inria)
Dr. Pierre-Yves Oudeyer is Research Director (DR1) at Inria and head of the Inria and Ensta-ParisTech FLOWERS team (France). Before, he has been a permanent researcher in Sony Computer Science Laboratory for 8 years (1999-2007). After working on computational models of language evolution, he is now working on developmental and social robotics, focusing on sensorimotor development, language acquisition and life-long learning in robots. Strongly inspired by infant development, the mechanisms he studies include artificial curiosity, intrinsic motivation, the role of morphology in learning motor control, human-robot interfaces, joint attention and joint intentional understanding, and imitation learning. He has published a book, more than 80 papers in international journals and conferences, holds 8 patents, gave several invited keynote lectures in international conferences, and received several prizes for his work in developmental robotics and on the origins of language. In particular, he is laureate of the ERC Starting Grant EXPLORERS. He is editor of the IEEE CIS Newsletter on Autonomous Mental Development, and associate editor of IEEE Transactions on Autonomous Mental Development, Frontiers in Neurorobotics, and of the International Journal of Social Robotics. He is also working actively for the diffusion of science towards the general public, through the writing of popular science articles and participation to radio and TV programs as well as science exhibitions. Web:http://www.pyoudeyer.com and http://flowers.inria.fr
Ken Stanley (Uber AI and University of Central Florida)

Kenneth O. Stanley leads a research team at OpenAI on the challenge of open-endedness. He was previously Charles Millican Professor of Computer Science at the University of Central Florida and was also a co-founder of Geometric Intelligence Inc., which was acquired by Uber to create Uber AI Labs, where he was head of Core AI research. He received a B.S.E. from the University of Pennsylvania in 1997 and received a Ph.D. in 2004 from the University of Texas at Austin. He is an inventor of the Neuroevolution of Augmenting Topologies (NEAT), HyperNEAT, , novelty search, and POET algorithms, as well as the CPPN representation, among many others. His main research contributions are in neuroevolution (i.e. evolving neural networks), generative and developmental systems, coevolution, machine learning for video games, interactive evolution, quality diversity, and open-endedness. He has won best paper awards for his work on NEAT, NERO, NEAT Drummer, FSMC, HyperNEAT, novelty search, Galactic Arms Race, and POET. His original 2002 paper on NEAT also received the 2017 ISAL Award for Outstanding Paper of the Decade 2002 - 2012 from the International Society for Artificial Life. He is a coauthor of the popular science book, "Why Greatness Cannot Be Planned: The Myth of the Objective" (published by Springer), and has spoken widely on its subject.
Emma Brunskill (Stanford University)
More from the Same Authors
-
2020 Workshop: Theoretical Foundations of Reinforcement Learning »
Emma Brunskill · Thodoris Lykouris · Max Simchowitz · Wen Sun · Mengdi Wang -
2020 Poster: Gradient Temporal-Difference Learning with Regularized Corrections »
Sina Ghiassian · Andrew Patterson · Shivam Garg · Dhawal Gupta · Adam White · Martha White -
2020 Poster: Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions »
Omer Gottesman · Joseph Futoma · Yao Liu · Sonali Parbhoo · Leo Celi · Emma Brunskill · Finale Doshi-Velez -
2020 Poster: Learning Near Optimal Policies with Low Inherent Bellman Error »
Andrea Zanette · Alessandro Lazaric · Mykel Kochenderfer · Emma Brunskill -
2020 Poster: Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling »
Yao Liu · Pierre-Luc Bacon · Emma Brunskill -
2020 Poster: Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data »
Felipe Petroski Such · Aditya Rawal · Joel Lehman · Kenneth Stanley · Jeffrey Clune -
2020 Poster: Selective Dyna-style Planning Under Limited Model Capacity »
Zaheer Abbas · Samuel Sokota · Erin Talvitie · Martha White -
2020 Poster: Enhanced POET: Open-ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions »
Rui Wang · Joel Lehman · Aditya Rawal · Jiale Zhi · Yulun Li · Jeffrey Clune · Kenneth Stanley -
2020 Poster: Optimizing for the Future in Non-Stationary MDPs »
Yash Chandak · Georgios Theocharous · Shiv Shankar · Martha White · Sridhar Mahadevan · Philip Thomas -
2019 Poster: Combining parametric and nonparametric models for off-policy evaluation »
Omer Gottesman · Yao Liu · Scott Sussex · Emma Brunskill · Finale Doshi-Velez -
2019 Oral: Combining parametric and nonparametric models for off-policy evaluation »
Omer Gottesman · Yao Liu · Scott Sussex · Emma Brunskill · Finale Doshi-Velez -
2019 Poster: CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning »
Cédric Colas · Pierre-Yves Oudeyer · Olivier Sigaud · Pierre Fournier · Mohamed Chetouani -
2019 Poster: Policy Certificates: Towards Accountable Reinforcement Learning »
Christoph Dann · Lihong Li · Wei Wei · Emma Brunskill -
2019 Poster: Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds »
Andrea Zanette · Emma Brunskill -
2019 Poster: Separable value functions across time-scales »
Joshua Romoff · Peter Henderson · Ahmed Touati · Yann Ollivier · Joelle Pineau · Emma Brunskill -
2019 Oral: Policy Certificates: Towards Accountable Reinforcement Learning »
Christoph Dann · Lihong Li · Wei Wei · Emma Brunskill -
2019 Oral: Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds »
Andrea Zanette · Emma Brunskill -
2019 Oral: Separable value functions across time-scales »
Joshua Romoff · Peter Henderson · Ahmed Touati · Yann Ollivier · Joelle Pineau · Emma Brunskill -
2019 Oral: CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning »
Cédric Colas · Pierre-Yves Oudeyer · Olivier Sigaud · Pierre Fournier · Mohamed Chetouani -
2019 Tutorial: Recent Advances in Population-Based Search for Deep Neural Networks: Quality Diversity, Indirect Encodings, and Open-Ended Algorithms »
Jeff Clune · Joel Lehman · Kenneth Stanley -
2018 Poster: Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control »
Yangchen Pan · Amir-massoud Farahmand · Martha White · Saleh Nabi · Piyush Grover · Daniel Nikovski -
2018 Oral: Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control »
Yangchen Pan · Amir-massoud Farahmand · Martha White · Saleh Nabi · Piyush Grover · Daniel Nikovski -
2018 Poster: Improving Regression Performance with Distributional Losses »
Ehsan Imani · Martha White -
2018 Poster: GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms »
Cédric Colas · Olivier Sigaud · Pierre-Yves Oudeyer -
2018 Poster: Decoupling Gradient-Like Learning Rules from Representations »
Philip Thomas · Christoph Dann · Emma Brunskill -
2018 Poster: The Mirage of Action-Dependent Baselines in Reinforcement Learning »
George Tucker · Surya Bhupatiraju · Shixiang Gu · Richard E Turner · Zoubin Ghahramani · Sergey Levine -
2018 Poster: Learning Policy Representations in Multiagent Systems »
Aditya Grover · Maruan Al-Shedivat · Jayesh K. Gupta · Yura Burda · Harrison Edwards -
2018 Oral: Decoupling Gradient-Like Learning Rules from Representations »
Philip Thomas · Christoph Dann · Emma Brunskill -
2018 Oral: Improving Regression Performance with Distributional Losses »
Ehsan Imani · Martha White -
2018 Oral: The Mirage of Action-Dependent Baselines in Reinforcement Learning »
George Tucker · Surya Bhupatiraju · Shixiang Gu · Richard E Turner · Zoubin Ghahramani · Sergey Levine -
2018 Oral: GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms »
Cédric Colas · Olivier Sigaud · Pierre-Yves Oudeyer -
2018 Oral: Learning Policy Representations in Multiagent Systems »
Aditya Grover · Maruan Al-Shedivat · Jayesh K. Gupta · Yura Burda · Harrison Edwards -
2018 Poster: Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings »
John Co-Reyes · Yu Xuan Liu · Abhishek Gupta · Benjamin Eysenbach · Pieter Abbeel · Sergey Levine -
2018 Poster: Problem Dependent Reinforcement Learning Bounds Which Can Identify Bandit Structure in MDPs »
Andrea Zanette · Emma Brunskill -
2018 Oral: Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings »
John Co-Reyes · Yu Xuan Liu · Abhishek Gupta · Benjamin Eysenbach · Pieter Abbeel · Sergey Levine -
2018 Oral: Problem Dependent Reinforcement Learning Bounds Which Can Identify Bandit Structure in MDPs »
Andrea Zanette · Emma Brunskill