Timezone: »
Oral
Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets
Rob Cornish · Paul Vanetti · Alexandre Bouchard-Côté · George Deligiannidis · Arnaud Doucet
Bayesian inference via standard Markov Chain Monte Carlo (MCMC) methods such as Metropolis--Hastings is too computationally intensive to handle large datasets, since the cost per step usually scales like $O(n)$ in the number of data points $n$. We propose the \emph{Scalable Metropolis--Hastings} (SMH) kernel that only requires processing on average $O(1)$ or even $O(1/\sqrt{n})$ data points per step. This scheme is based on a combination of factorized acceptance probabilities, procedures for fast simulation of Bernoulli processes, and control variate ideas. Contrary to many MCMC subsampling schemes such as fixed step-size Stochastic Gradient Langevin Dynamics, our approach is exact insofar as the invariant distribution is the true posterior and not an approximation to it. We characterise the performance of our algorithm theoretically, and give realistic and verifiable conditions under which it is geometrically ergodic. This theory is borne out by empirical results that demonstrate overall performance benefits over standard Metropolis-Hastings and various subsampling algorithms.
Author Information
Rob Cornish (Oxford)
Paul Vanetti (Oxford)
Alexandre Bouchard-Côté (UBC)
George Deligiannidis (Oxford)
Arnaud Doucet (Oxford University)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Poster: Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets »
Fri. Jun 14th 01:30 -- 04:00 AM Room Pacific Ballroom #202
More from the Same Authors
-
2022 : Riemannian Diffusion Schr\"odinger Bridge »
James Thornton · Valentin De Bortoli · Michael Hutchinson · Emile Mathieu · Yee Whye Teh · Arnaud Doucet -
2023 : Diffusion Generative Inverse Design »
Marin Vlastelica · Tatiana Lopez-Guevara · Kelsey Allen · Peter Battaglia · Arnaud Doucet · Kimberly Stachenfeld -
2023 : Categorical SDEs with Simplex Diffusion »
Pierre Richemond · Sander Dieleman · Arnaud Doucet -
2023 Poster: Generalization Bounds using Data-Dependent Fractal Dimensions »
Benjamin Dupuis · George Deligiannidis · Umut Simsekli -
2023 Poster: Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC »
Yilun Du · Conor Durkan · Robin Strudel · Josh Tenenbaum · Sander Dieleman · Rob Fergus · Jascha Sohl-Dickstein · Arnaud Doucet · Will Grathwohl -
2023 Poster: SE(3) diffusion model with application to protein backbone generation »
Jason Yim · Brian Trippe · Valentin De Bortoli · Emile Mathieu · Arnaud Doucet · Regina Barzilay · Tommi Jaakkola -
2021 Poster: Monte Carlo Variational Auto-Encoders »
Achille Thin · Nikita Kotelevskii · Arnaud Doucet · Alain Durmus · Eric Moulines · Maxim Panov -
2021 Spotlight: Monte Carlo Variational Auto-Encoders »
Achille Thin · Nikita Kotelevskii · Arnaud Doucet · Alain Durmus · Eric Moulines · Maxim Panov -
2021 Poster: Differentiable Particle Filtering via Entropy-Regularized Optimal Transport »
Adrien Corenflos · James Thornton · George Deligiannidis · Arnaud Doucet -
2021 Poster: Parallel tempering on optimized paths »
Saifuddin Syed · Vittorio Romaniello · Trevor Campbell · Alexandre Bouchard-Côté -
2021 Spotlight: Parallel tempering on optimized paths »
Saifuddin Syed · Vittorio Romaniello · Trevor Campbell · Alexandre Bouchard-Côté -
2021 Oral: Differentiable Particle Filtering via Entropy-Regularized Optimal Transport »
Adrien Corenflos · James Thornton · George Deligiannidis · Arnaud Doucet -
2021 Poster: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding »
Yangjun Ruan · Karen Ullrich · Daniel Severo · James Townsend · Ashish Khisti · Arnaud Doucet · Alireza Makhzani · Chris Maddison -
2021 Oral: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding »
Yangjun Ruan · Karen Ullrich · Daniel Severo · James Townsend · Ashish Khisti · Arnaud Doucet · Alireza Makhzani · Chris Maddison -
2020 Poster: Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows »
Rob Cornish · Anthony Caterini · George Deligiannidis · Arnaud Doucet -
2019 : Spotlight »
Tyler Scott · Kiran Thekumparampil · Jonathan Aigrain · Rene Bidart · Priyadarshini Panda · Dian Ang Yap · Yaniv Yacoby · Raphael Gontijo Lopes · Alberto Marchisio · Erik Englesson · Wanqian Yang · Moritz Graule · Yi Sun · Daniel Kang · Mike Dusenberry · Min Du · Hartmut Maennel · Kunal Menda · Vineet Edupuganti · Luke Metz · David Stutz · Vignesh Srinivasan · Timo Sämann · Vineeth N Balasubramanian · Sina Mohseni · Rob Cornish · Judith Butepage · Zhangyang Wang · Bai Li · Bo Han · Honglin Li · Maksym Andriushchenko · Lukas Ruff · Meet P. Vadera · Yaniv Ovadia · Sunil Thulasidasan · Disi Ji · Gang Niu · Saeed Mahloujifar · Aviral Kumar · SANGHYUK CHUN · Dong Yin · Joyce Xu Xu · Hugo Gomes · Raanan Rohekar -
2019 Poster: Replica Conditional Sequential Monte Carlo »
Alex Shestopaloff · Arnaud Doucet -
2019 Oral: Replica Conditional Sequential Monte Carlo »
Alex Shestopaloff · Arnaud Doucet -
2019 Poster: On the Impact of the Activation function on Deep Neural Networks Training »
Soufiane Hayou · Arnaud Doucet · Judith Rousseau -
2019 Oral: On the Impact of the Activation function on Deep Neural Networks Training »
Soufiane Hayou · Arnaud Doucet · Judith Rousseau -
2018 Poster: On Nesting Monte Carlo Estimators »
Tom Rainforth · Rob Cornish · Hongseok Yang · andrew warrington · Frank Wood -
2018 Oral: On Nesting Monte Carlo Estimators »
Tom Rainforth · Rob Cornish · Hongseok Yang · andrew warrington · Frank Wood