Timezone: »

 
Spotlight
The State of Sparse Training in Deep Reinforcement Learning
Laura Graesser · Utku Evci · Erich Elsen · Pablo Samuel Castro

Wed Jul 20 01:40 PM -- 01:45 PM (PDT) @ Room 309
in Deep Learning/Optimization »

The use of sparse neural networks has seen rapid growth in recent years, particularly in computer vision. Their appeal stems largely from the reduced number of parameters required to train and store, as well as in an increase in learning efficiency. Somewhat surprisingly, there have been very few efforts exploring their use in Deep Reinforcement Learning (DRL). In this work we perform a systematic investigation into applying a number of existing sparse training techniques on a variety of DRL agents and environments. Our results corroborate the findings from sparse training in the computer vision domain –sparse networks perform better than dense networks for the same parameter count– in the DRL domain. We provide detailed analyses on how the various components in DRL are affected by the use of sparse networks and conclude by suggesting promising avenues for improving the effectiveness of sparse training methods, as well as for advancing their use in DRL.

Keywords: Deep Learning · DL: Algorithms · DL: Everything Else · RL: Deep RL · MISC: Scalable Algorithms

Author Information

Laura Graesser (Google)
Utku Evci (Google)
Erich Elsen (Google)
Pablo Samuel Castro (Google Brain)

Pablo was born and raised in Quito, Ecuador, and moved to Montreal after high school to study at McGill. He stayed in Montreal for the next 10 years, finished his bachelors, worked at a flight simulator company, and then eventually obtained his masters and PhD at McGill, focusing on Reinforcement Learning. After his PhD Pablo did a 10-month postdoc in Paris before moving to Pittsburgh to join Google. He has worked at Google for almost 6 years, and is currently a research Software Engineer in Google Brain in Montreal, focusing on fundamental Reinforcement Learning research, as well as Machine Learning and Music. Aside from his interest in coding/AI/math, Pablo is an active musician (https://www.psctrio.com), loves running (5 marathons so far, including Boston!), and discussing politics and activism.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors