Timezone: »

 
Poster
Self-Paced Context Evaluation for Contextual Reinforcement Learning
Theresa Eimer · André Biedenkapp · Frank Hutter · Marius Lindauer

Wed Jul 21 09:00 AM -- 11:00 AM (PDT) @ Virtual
in Poster Session 3 »

Reinforcement learning (RL) has made a lot of advances for solving a single problem in a given environment; but learning policies that generalize to unseen variations of a problem remains challenging. To improve sample efficiency for learning on such instances of a problem domain, we present Self-Paced Context Evaluation (SPaCE). Based on self-paced learning, SPaCE automatically generates instance curricula online with little computational overhead. To this end, SPaCE leverages information contained in state values during training to accelerate and improve training performance as well as generalization capabilities to new \tasks from the same problem domain. Nevertheless, SPaCE is independent of the problem domain at hand and can be applied on top of any RL agent with state-value function approximation. We demonstrate SPaCE's ability to speed up learning of different value-based RL agents on two environments, showing better generalization capabilities and up to 10x faster learning compared to naive approaches such as round robin or SPDRL, as the closest state-of-the-art approach.

Keywords: Reinforcement Learning and Planning
[ Slides] 

Author Information

Theresa Eimer (Leibniz Universität Hannover)
André Biedenkapp (University of Freiburg)

Since October 2017 I am a PhD student at the Machine Learning Group under the supervision of Frank Hutter and Marius Lindauer. Before that I completed my master and bachelor degrees in computer science at the University of Freiburg. Research Interests I am interested in all facets of artificial intelligence. My research focuses on new ways to control the behavior of algorithms online. More precisely my research areas include: Dynamic Algorithm Configuration/Algorithm Control Learning to Learn (Deep) Reinforcement Learning Bayesian Optimization Automated Hyperparameter Optimization

Frank Hutter (University of Freiburg and Bosch Center for Artificial Intelligence)
Frank Hutter

Frank Hutter is a Full Professor for Machine Learning at the Computer Science Department of the University of Freiburg (Germany), where he has been a faculty member since 2013. Before that, he was at the University of British Columbia (UBC) for eight years, for his PhD and postdoc. Frank's main research interests lie in machine learning, artificial intelligence and automated algorithm design. For his 2009 PhD thesis on algorithm configuration, he received the CAIAC doctoral dissertation award for the best thesis in AI in Canada that year, and with his coauthors, he received several best paper awards and prizes in international competitions on automated machine learning, SAT solving, and AI planning. Since 2016 he holds an ERC Starting Grant for a project on automating deep learning based on Bayesian optimization, Bayesian neural networks, and deep reinforcement learning.

Marius Lindauer (Leibniz University Hannover)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors