Timezone: »

 
Poster
Accelerating the diffusion-based ensemble sampling by non-reversible dynamics
Futoshi Futami · Issei Sato · Masashi Sugiyama

Tue Jul 14 06:00 PM -- 06:45 PM & Wed Jul 15 04:00 AM -- 04:45 AM (PDT) @
in Poster Session 10 »

Posterior distribution approximation is a central task in Bayesian inference. Stochastic gradient Langevin dynamics (SGLD) and its extensions have been practically used and theoretically studied. While SGLD updates a single particle at a time, ensemble methods that update multiple particles simultaneously have been recently gathering attention. Compared with the naive parallel-chain SGLD that updates multiple particles independently, ensemble methods update particles with their interactions. Thus, these methods are expected to be more particle-efficient than the naive parallel-chain SGLD because particles can be aware of other particles' behavior through their interactions. Although ensemble methods numerically demonstrated their superior performance, no theoretical guarantee exists to assure such particle-efficiency and it is unclear whether those ensemble methods are really superior to the naive parallel-chain SGLD in the non-asymptotic settings. To cope with this problem, we propose a novel ensemble method that uses a non-reversible Markov chain for the interaction, and we present a non-asymptotic theoretical analysis for our method. Our analysis shows that, for the first time, the interaction causes a faster convergence rate than the naive parallel-chain SGLD in the non-asymptotic setting if the discretization error is appropriately controlled. Numerical experiments show that we can control the discretization error by tuning the interaction appropriately.

Keywords: Approximate Inference · Monte Carlo Methods · Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods

Author Information

Futoshi Futami (NTT)
Issei Sato (University of Tokyo / RIKEN)
Masashi Sugiyama (RIKEN / The University of Tokyo)

More from the Same Authors