In open-ended and changing environments, agents face a wide range of potential tasks that might not come with associated reward functions. Such autonomous learning agents must set their own tasks and build their own curriculum through an intrinsically motivated exploration. Because some tasks might prove easy and some impossible, agents must actively select which task to practice at any given moment to maximize their overall mastery on the set of learnable tasks. This paper proposes CURIOUS, an algorithm that leverages: 1) an extension of Universal Value Function Approximators to achieve within a unique policy, multiple tasks, each parameterized by multiple goals and 2) an automated curriculum learning mechanism that biases the attention of the agent towards tasks maximizing the absolute learning progress. Agents focus on achievable tasks first, and focus back on tasks that are being forgotten. Experiments conducted in a new multi-task multi-goal robotic environment show that our algorithm benefits from these two ideas and demonstrate properties of robustness to distracting tasks, forgetting and changes in body properties.
Author Information
Cédric Colas (Inria)
Pierre-Yves Oudeyer (Inria)
Dr. Pierre-Yves Oudeyer is Research Director (DR1) at Inria and head of the Inria and Ensta-ParisTech FLOWERS team (France). Before, he has been a permanent researcher in Sony Computer Science Laboratory for 8 years (1999-2007). After working on computational models of language evolution, he is now working on developmental and social robotics, focusing on sensorimotor development, language acquisition and life-long learning in robots. Strongly inspired by infant development, the mechanisms he studies include artificial curiosity, intrinsic motivation, the role of morphology in learning motor control, human-robot interfaces, joint attention and joint intentional understanding, and imitation learning. He has published a book, more than 80 papers in international journals and conferences, holds 8 patents, gave several invited keynote lectures in international conferences, and received several prizes for his work in developmental robotics and on the origins of language. In particular, he is laureate of the ERC Starting Grant EXPLORERS. He is editor of the IEEE CIS Newsletter on Autonomous Mental Development, and associate editor of IEEE Transactions on Autonomous Mental Development, Frontiers in Neurorobotics, and of the International Journal of Social Robotics. He is also working actively for the diffusion of science towards the general public, through the writing of popular science articles and participation to radio and TV programs as well as science exhibitions. Web:http://www.pyoudeyer.com and http://flowers.inria.fr
Olivier Sigaud (Sorbonne University)
Pierre Fournier (UPMC)
Mohamed Chetouani (UPMC)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Poster: CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning »
Tue Jun 11th 06:30 -- 09:00 PM Room Pacific Ballroom
More from the Same Authors
-
2019 Workshop: Exploration in Reinforcement Learning Workshop »
Benjamin Eysenbach · Benjamin Eysenbach · Surya Bhupatiraju · Shixiang Gu · Harrison Edwards · Martha White · Pierre-Yves Oudeyer · Kenneth Stanley · Emma Brunskill -
2018 Poster: GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms »
Cédric Colas · Olivier Sigaud · Pierre-Yves Oudeyer -
2018 Oral: GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms »
Cédric Colas · Olivier Sigaud · Pierre-Yves Oudeyer
