Taiji Suzuki: Convergence of mean field Langevin dynamics: Duality viewpoint and neural network optimization

The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the gradient Langevin dynamics (GLD) that minimizes an entropy regularized convex function defined on the space of probability distributions, and it naturally arises from the optimization of two-layer neural networks via (noisy) gradient descent. In this talk, I will present the convergence result of MFLD and explain how the convergence of MFLD is connected to its dual objective. Indeed, its convergence is characterized by the log-Sobolev inequality of the so-called proximal Gibbs measure corresponding to the current solution. Based on this duality principle, we can construct several optimization methods with convergence guarantees including the particle dual averaging method and particle stochastic dual coordinate ascent method. Finally, I will provide a general framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and stochastic gradient approximation.