Skip to yearly menu bar Skip to main content


Provable Smoothness Guarantees for Black-Box Variational Inference

Justin Domke

Keywords: [ Approximate Inference ] [ Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods ]


Black-box variational inference tries to approximate a complex target distribution through a gradient-based optimization of the parameters of a simpler distribution. Provable convergence guarantees require structural properties of the objective. This paper shows that for location-scale family approximations, if the target is M-Lipschitz smooth, then so is the “energy” part of the variational objective. The key proof idea is to describe gradients in a certain inner-product space, thus permitting the use of Bessel’s inequality. This result gives bounds on the location of the optimal parameters, and is a key ingredient for convergence guarantees.

Chat is not available.