Workshop
Theoretical Physics for Deep Learning
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna
Fri 14 Jun, 8:30 a.m. PDT
Though the purview of physics is broad and includes many loosely connected subdisciplines, a unifying theme is the endeavor to provide concise, quantitative, and predictive descriptions of the often large and complex systems governing phenomena that occur in the natural world. While one could debate how closely deep learning is connected to the natural world, it is undeniably the case that deep learning systems are large and complex; as such, it is reasonable to consider whether the rich body of ideas and powerful tools from theoretical physicists could be harnessed to improve our understanding of deep learning. The goal of this workshop is to investigate this question by bringing together experts in theoretical physics and deep learning in order to stimulate interaction and to begin exploring how theoretical physics can shed light on the theory of deep learning.
We believe ICML is an appropriate venue for this gathering as members from both communities are frequently in attendance and because deep learning theory has emerged as a focus at the conference, both as an independent track in the main conference and in numerous workshops over the last few years. Moreover, the conference has enjoyed an increasing number of papers using physics tools and ideas to draw insights into deep learning.
Schedule
Fri 8:30 a.m. - 8:40 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
|
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna 🔗 |
Fri 8:40 a.m. - 9:10 a.m.
|
Linearized two-layers neural networks in high dimension
(
Invited talk
)
>
|
Andrea Montanari 🔗 |
Fri 9:10 a.m. - 9:40 a.m.
|
Loss landscape and behaviour of algorithms in the spiked matrix-tensor model
(
Invited talk
)
>
|
Lenka Zdeborova 🔗 |
Fri 9:40 a.m. - 10:20 a.m.
|
Poster spotlights
(
Spotlight
)
>
|
Roman Novak · Frederic Dreyer · Siavash Golkar · Irina Higgins · Joe Antognini · Ryo Karakida · Rohan Ghosh 🔗 |
Fri 10:20 a.m. - 11:00 a.m.
|
Break and poster discussion
(
Break and Poster
)
>
|
🔗 |
Fri 11:00 a.m. - 11:30 a.m.
|
On the Interplay between Physics and Deep Learning
(
Invited talk
)
>
|
Kyle Cranmer 🔗 |
Fri 11:30 a.m. - 12:00 p.m.
|
Why Deep Learning Works: Traditional and Heavy-Tailed Implicit Self-Regularization in Deep Neural Networks
(
Invited talk
)
>
|
Michael Mahoney 🔗 |
Fri 12:00 p.m. - 12:15 p.m.
|
Analyzing the dynamics of online learning in over-parameterized two-layer neural networks
(
Oral
)
>
|
Sebastian Goldt 🔗 |
Fri 12:15 p.m. - 12:30 p.m.
|
Convergence Properties of Neural Networks on Separable Data
(
Oral
)
>
|
Remi Tachet des Combes 🔗 |
Fri 12:30 p.m. - 2:00 p.m.
|
Lunch
|
🔗 |
Fri 2:00 p.m. - 2:30 p.m.
|
Is Optimization a sufficient language to understand Deep Learning?
(
Invited talk
)
>
|
Sanjeev Arora 🔗 |
Fri 2:30 p.m. - 2:45 p.m.
|
Towards Understanding Regularization in Batch Normalization
(
Oral
)
>
|
🔗 |
Fri 2:45 p.m. - 3:00 p.m.
|
How Noise during Training Affects the Hessian Spectrum
(
Oral
)
>
|
🔗 |
Fri 3:00 p.m. - 3:30 p.m.
|
Break and poster discussion
(
Break and Poster
)
>
|
🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Understanding overparameterized neural networks
(
Invited talk
)
>
|
Jascha Sohl-Dickstein 🔗 |
Fri 4:00 p.m. - 4:15 p.m.
|
Asymptotics of Wide Networks from Feynman Diagrams
(
Oral
)
>
|
Guy Gur-Ari 🔗 |
Fri 4:15 p.m. - 4:30 p.m.
|
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
(
Oral
)
>
|
Dar Gilboa 🔗 |
Fri 4:30 p.m. - 4:45 p.m.
|
Deep Learning on the 2-Dimensional Ising Model to Extract the Crossover Region
(
Oral
)
>
|
Nicholas Walker 🔗 |
Fri 4:45 p.m. - 5:00 p.m.
|
Learning the Arrow of Time
(
Oral
)
>
|
Nasim Rahaman 🔗 |
Fri 5:00 p.m. - 6:00 p.m.
|
Poster discussion
(
Poster Session
)
>
|
59 presentersRoman Novak · Maxime Gabella · Frederic Dreyer · Siavash Golkar · Anh Tong · Irina Higgins · Mirco Milletari · Joe Antognini · Sebastian Goldt · Adín Ramírez Rivera · Roberto Bondesan · Ryo Karakida · Remi Tachet des Combes · Michael Mahoney · Nicholas Walker · Stanislav Fort · Samuel Smith · Rohan Ghosh · Aristide Baratin · Diego Granziol · Stephen Roberts · Dmitry Vetrov · Andrew Wilson · César Laurent · Valentin Thomas · Simon Lacoste-Julien · Dar Gilboa · Daniel Soudry · Anupam Gupta · Anirudh Goyal · Yoshua Bengio · Erich Elsen · Soham De · Stanislaw Jastrzebski · Charles H Martin · Samira Shabanian · Aaron Courville · Shorato Akaho · Lenka Zdeborova · Ethan Dyer · Maurice Weiler · Pim de Haan · Taco Cohen · Max Welling · Ping Luo · zhanglin peng · Nasim Rahaman · Loic Matthey · Danilo J. Rezende · Jaesik Choi · Kyle Cranmer · Lechao Xiao · Jaehoon Lee · Yasaman Bahri · Jeffrey Pennington · Greg Yang · Jiri Hron · Jascha Sohl-Dickstein · Guy Gur-Ari |