Timezone: »

 
Oral
Hyperparameter Selection for Imitation Learning
Léonard Hussenot · Marcin Andrychowicz · Damien Vincent · Robert Dadashi · Anton Raichuk · Sabela Ramos · Nikola Momchev · Sertan Girgin · Raphael Marinier · Lukasz Stafiniak · Emmanuel Orsini · Olivier Bachem · Matthieu Geist · Olivier Pietquin

Tue Jul 20 07:00 PM -- 07:20 PM (PDT) @
in Reinforcement Learning 9 »

We address the issue of tuning hyperparameters (HPs) for imitation learning algorithms in the context of continuous-control, when the underlying reward function of the demonstrating expert cannot be observed at any time. The vast literature in imitation learning mostly considers this reward function to be available for HP selection, but this is not a realistic setting. Indeed, would this reward function be available, it could then directly be used for policy training and imitation would not be necessary. To tackle this mostly ignored problem, we propose a number of possible proxies to the external reward. We evaluate them in an extensive empirical study (more than 10'000 agents across 9 environments) and make practical recommendations for selecting HPs. Our results show that while imitation learning algorithms are sensitive to HP choices, it is often possible to select good enough HPs through a proxy to the reward function.

Keywords: Reinforcement Learning and Planning

Author Information

Léonard Hussenot (Google Research, Brain Team)
Marcin Andrychowicz (Google)
Damien Vincent (Google Brain)
Robert Dadashi (Google Research)
Anton Raichuk (Google)
Sabela Ramos (Google Research)
Nikola Momchev (Google)
Sertan Girgin (Google Brain)
Raphael Marinier (Google)
Lukasz Stafiniak (Google)
Emmanuel Orsini (Google Brain)
Olivier Bachem (Google Brain)
Matthieu Geist (Google)
Olivier Pietquin (GOOGLE BRAIN)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors