Reza Gheissari (Northwestern University), Local geometry and effective spectral theory of high-dimensional classification

We probe the local geometry of loss landscapes coming from high-dimensional Gaussian mixture classification tasks as seen from a typical training trajectory. We study this by proving limits for the empirical spectral distribution and locations of outliers, for the empirical Hessian at all points in parameter space. Notably, this limiting spectral theory only depends on the point in parameter space through some $O(1)$-many summary statistics of the parameter; these summary statistics also follow an autonomous evolution under training by stochastic gradient descent. This allows us to investigate the limiting spectral theory along the limiting training trajectory, and find interesting phenomena like splitting and emergence of outliers over the course of training. Based on joint works with G. Ben Arous, J. Huang, and A. Jagannath.