Skip to yearly menu bar Skip to main content


( events)   Timezone:  
Workshop
Fri Jun 14 08:30 AM -- 06:00 PM (PDT) @ 104 C
Theoretical Physics for Deep Learning
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna

Though the purview of physics is broad and includes many loosely connected subdisciplines, a unifying theme is the endeavor to provide concise, quantitative, and predictive descriptions of the often large and complex systems governing phenomena that occur in the natural world. While one could debate how closely deep learning is connected to the natural world, it is undeniably the case that deep learning systems are large and complex; as such, it is reasonable to consider whether the rich body of ideas and powerful tools from theoretical physicists could be harnessed to improve our understanding of deep learning. The goal of this workshop is to investigate this question by bringing together experts in theoretical physics and deep learning in order to stimulate interaction and to begin exploring how theoretical physics can shed light on the theory of deep learning.

We believe ICML is an appropriate venue for this gathering as members from both communities are frequently in attendance and because deep learning theory has emerged as a focus at the conference, both as an independent track in the main conference and in numerous workshops over the last few years. Moreover, the conference has enjoyed an increasing number of papers using physics tools and ideas to draw insights into deep learning.

Opening Remarks
Linearized two-layers neural networks in high dimension (Invited talk)
Loss landscape and behaviour of algorithms in the spiked matrix-tensor model (Invited talk)
Poster spotlights (Spotlight)
Break and poster discussion (Break and Poster)
On the Interplay between Physics and Deep Learning (Invited talk)
Why Deep Learning Works: Traditional and Heavy-Tailed Implicit Self-Regularization in Deep Neural Networks (Invited talk)
Analyzing the dynamics of online learning in over-parameterized two-layer neural networks (Oral)
Convergence Properties of Neural Networks on Separable Data (Oral)
Lunch (Break)
Is Optimization a sufficient language to understand Deep Learning? (Invited talk)
Towards Understanding Regularization in Batch Normalization (Oral)
How Noise during Training Affects the Hessian Spectrum (Oral)
Break and poster discussion (Break and Poster)
Understanding overparameterized neural networks (Invited talk)
Asymptotics of Wide Networks from Feynman Diagrams (Oral)
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off (Oral)
Deep Learning on the 2-Dimensional Ising Model to Extract the Crossover Region (Oral)
Learning the Arrow of Time (Oral)
Poster discussion (Poster Session)