Understanding and Accelerating Particle-Based Variational Inference

Particle-based variational inference methods (ParVIs) have gained notable attention in the Bayesian inference area, for their flexible approximation ability and effective iteration. In this work, we explore in-depth the perspective of ParVIs as Wasserstein gradient flows and make both theoretical and pragmatic contributions. On the theory side, we unify various finite-particle approximations that existing ParVIs use, and recognize that the approximation is essentially a compulsory smoothing treatment, in either of two equivalent forms. This novel understanding reveals the assumption and relations of existing ParVIs, and also inspires new ParVIs as we will demonstrate. On the technical side, we propose an acceleration framework and a principled bandwidth selection method for general ParVIs. They are based on the developed theory and our excavation on the geometry of the Wasserstein space. Experimental results show the improved convergence by the acceleration framework and enhanced sample accuracy by the bandwidth selection method.