Timezone: »

 
Spotlight
Scalable Spike-and-Slab
Niloy Biswas · Lester Mackey · Xiao-Li Meng

Tue Jul 19 01:55 PM -- 02:00 PM (PDT) @ Room 307
in PM: Monte Carlo and Sampling Methods »

Spike-and-slab priors are commonly used for Bayesian variable selection, due to their interpretability and favorable statistical properties. However, existing samplers for spike-and-slab posteriors incur prohibitive computational costs when the number of variables is large. In this article, we propose Scalable Spike-and-Slab (S^3), a scalable Gibbs sampling implementation for high-dimensional Bayesian regression with the continuous spike-and-slab prior of George & McCulloch (1993). For a dataset with n observations and p covariates, S^3 has order max{n^2 pt, np} computational cost at iteration t where pt never exceeds the number of covariates switching spike-and-slab states between iterations t and t-1 of the Markov chain. This improves upon the order n^2 p per-iteration cost of state-of-the-art implementations as, typically, p_t is substantially smaller than p. We apply S^3 on synthetic and real-world datasets, demonstrating orders of magnitude speed-ups over existing exact samplers and significant gains in inferential quality over approximate samplers with comparable cost.

Keywords: Probabilistic Methods · PM: Bayesian Models and Methods · PM: Monte Carlo and Sampling Methods

Author Information

Niloy Biswas (Harvard University)
Lester Mackey (Microsoft Research)
Xiao-Li Meng (Harvard University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors