Timezone: »

 
Poster
Stochastic Gradient and Langevin Processes
Xiang Cheng · Dong Yin · Peter Bartlett · Michael Jordan

Wed Jul 15 08:00 AM -- 08:45 AM & Wed Jul 15 09:00 PM -- 09:45 PM (PDT) @ None #None
in AcceptedPapersParent »

We prove quantitative convergence rates at which discrete Langevin-like processes converge to the invariant distribution of a related stochastic differential equation. We study the setup where the additive noise can be non-Gaussian and state-dependent and the potential function can be non-convex. We show that the key properties of these processes depend on the potential function and the second moment of the additive noise. We apply our theoretical findings to studying the convergence of Stochastic Gradient Descent (SGD) for non-convex problems and corroborate them with experiments using SGD to train deep neural networks on the CIFAR-10 dataset.

Keywords: Gaussian Processes · Monte Carlo Methods · Optimization · Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods

Author Information

Xiang Cheng (UC Berkeley)
Dong Yin (UC Berkeley)
Peter Bartlett (Berkeley)
Michael Jordan (UC Berkeley)

More from the Same Authors