Timezone: »

Theoretical Physics for Deep Learning
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna

Fri Jun 08:30 AM -- 06:00 PM PDT @ 104 C

Though the purview of physics is broad and includes many loosely connected subdisciplines, a unifying theme is the endeavor to provide concise, quantitative, and predictive descriptions of the often large and complex systems governing phenomena that occur in the natural world. While one could debate how closely deep learning is connected to the natural world, it is undeniably the case that deep learning systems are large and complex; as such, it is reasonable to consider whether the rich body of ideas and powerful tools from theoretical physicists could be harnessed to improve our understanding of deep learning. The goal of this workshop is to investigate this question by bringing together experts in theoretical physics and deep learning in order to stimulate interaction and to begin exploring how theoretical physics can shed light on the theory of deep learning.

We believe ICML is an appropriate venue for this gathering as members from both communities are frequently in attendance and because deep learning theory has emerged as a focus at the conference, both as an independent track in the main conference and in numerous workshops over the last few years. Moreover, the conference has enjoyed an increasing number of papers using physics tools and ideas to draw insights into deep learning.

08:30 AM Opening Remarks Video »  Jaehoon Lee, Jeffrey Pennington, Yasaman Bahri, Max Welling, Surya Ganguli, Joan Bruna
08:40 AM Linearized two-layers neural networks in high dimension (Invited talk)|| Video »  Andrea Montanari
09:10 AM Loss landscape and behaviour of algorithms in the spiked matrix-tensor model (Invited talk)|| Video »  Lenka Zdeborova
09:40 AM Poster spotlights (Spotlight)|| Roman Novak, Frederic Dreyer, Siavash Golkar, Irina Higgins, Joe Antognini, Rio Karakida, Rohan Ghosh
10:20 AM Break and poster discussion (Break and Poster)||
11:00 AM On the Interplay between Physics and Deep Learning (Invited talk)|| Video »  Kyle Cranmer
11:30 AM Why Deep Learning Works: Traditional and Heavy-Tailed Implicit Self-Regularization in Deep Neural Networks (Invited talk)|| Video »  Michael Mahoney
12:00 PM Analyzing the dynamics of online learning in over-parameterized two-layer neural networks (Oral)|| Video »  Sebastian Goldt
12:15 PM Convergence Properties of Neural Networks on Separable Data (Oral)|| Video »  Remi Tachet des Combes
12:30 PM Lunch (Break)||
02:00 PM Is Optimization a sufficient language to understand Deep Learning? (Invited talk)|| Video »  Sanjeev Arora
02:30 PM Towards Understanding Regularization in Batch Normalization (Oral)|| Video » 
02:45 PM How Noise during Training Affects the Hessian Spectrum (Oral)|| Video » 
03:00 PM Break and poster discussion (Break and Poster)||
03:30 PM Understanding overparameterized neural networks (Invited talk)|| Video »  Jascha Sohl-Dickstein
04:00 PM Asymptotics of Wide Networks from Feynman Diagrams (Oral)|| Video »  Guy Gur-Ari
04:15 PM A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off (Oral)|| Video »  Dar Gilboa
04:30 PM Deep Learning on the 2-Dimensional Ising Model to Extract the Crossover Region (Oral)|| Video »  Nick Walker
04:45 PM Learning the Arrow of Time (Oral)|| Video »  Nasim Rahaman
05:00 PM Poster discussion (Poster Session)||
Roman Novak, Maxime Gabella, Frederic Dreyer, Siavash Golkar, Anh Tong, Irina Higgins, Mirco Milletari, Joe Antognini, Sebastian Goldt, Adín Ramírez Rivera, Roberto Bondesan, Rio Karakida, Remi Tachet des Combes, Michael Mahoney, Nick Walker, Stanislav Fort, Samuel Smith, Rohan Ghosh, Aristide Baratin, Diego Granziol, Stephen Roberts, Dmitry Vetrov, Andrew Wilson, César Laurent, Valentin Thomas, Simon Lacoste-Julien, Dar Gilboa, Daniel Soudry, Anupam Gupta, Anirudh Goyal, Yoshua Bengio, Erich Elsen, Soham De, Stanislaw Jastrzebski, Charles H Martin, Samira Shabanian, Aaron Courville, Shorato Akaho, Lenka Zdeborova, Ethan Dyer, Maurice Weiler, Pim de Haan, Taco Cohen, Max Welling, Ping Luo, zhanglin peng, Nasim Rahaman, Loic Matthey, Danilo J. Rezende, Jaesik Choi, Kyle Cranmer, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Jeffrey Pennington, Greg Yang, Jiri Hron, Jascha Sohl-Dickstein, Guy Gur-Ari

Author Information

Jaehoon Lee (Google Brain)
Jeffrey Pennington (Google Brain)
Yasaman Bahri (Google Brain)
Max Welling (University of Amsterdam & Qualcomm)
Surya Ganguli (Stanford)
Joan Bruna (New York University)

More from the Same Authors