Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms---which build a small, weighted subset of the data that approximates the full dataset---are no exception. We demonstrate that these algorithms scale the coreset log-likelihood suboptimally, resulting in underestimated posterior uncertainty. To address this shortcoming, we develop greedy iterative geodesic ascent (GIGA), a novel algorithm for Bayesian coreset construction that scales the coreset log-likelihood optimally. GIGA provides geometric decay in posterior approximation error as a function of coreset size, and maintains the fast running time of its predecessors. The paper concludes with validation of GIGA on both synthetic and real datasets, demonstrating that it reduces posterior approximation error by orders of magnitude compared with previous coreset constructions.
Author Information
Trevor Campbell (MIT)
Tamara Broderick (MIT)
Tamara Broderick is the ITT Career Development Assistant Professor in the Department of Electrical Engineering and Computer Science at MIT. She is a member of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), the MIT Statistics and Data Science Center, and the Institute for Data, Systems, and Society (IDSS). She completed her Ph.D. in Statistics at the University of California, Berkeley in 2014. Previously, she received an AB in Mathematics from Princeton University (2007), a Master of Advanced Study for completion of Part III of the Mathematical Tripos from the University of Cambridge (2008), an MPhil by research in Physics from the University of Cambridge (2009), and an MS in Computer Science from the University of California, Berkeley (2013). Her recent research has focused on developing and analyzing models for scalable Bayesian machine learning---especially Bayesian nonparametrics. She has been awarded an NSF CAREER Award (2018), a Sloan Research Fellowship (2018), an Army Research Office Young Investigator Program award (2017), a Google Faculty Research Award, the ISBA Lifetime Members Junior Researcher Award, the Savage Award (for an outstanding doctoral dissertation in Bayesian theory and methods), the Evelyn Fix Memorial Medal and Citation (for the Ph.D. student on the Berkeley campus showing the greatest promise in statistical research), the Berkeley Fellowship, an NSF Graduate Research Fellowship, a Marshall Scholarship, and the Phi Beta Kappa Prize (for the graduating Princeton senior with the highest academic average).
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Oral: Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent »
Fri Jul 13th 05:00 -- 05:20 PM Room A4
More from the Same Authors
-
2019 Poster: The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions »
Raj Agrawal · Brian Trippe · Jonathan Huggins · Tamara Broderick -
2019 Oral: The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions »
Raj Agrawal · Brian Trippe · Jonathan Huggins · Tamara Broderick -
2019 Poster: LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations »
Brian Trippe · Jonathan Huggins · Raj Agrawal · Tamara Broderick -
2019 Oral: LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations »
Brian Trippe · Jonathan Huggins · Raj Agrawal · Tamara Broderick -
2018 Poster: Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models »
Raj Agrawal · Caroline Uhler · Tamara Broderick -
2018 Oral: Minimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models »
Raj Agrawal · Caroline Uhler · Tamara Broderick -
2018 Tutorial: Variational Bayes and Beyond: Bayesian Inference for Big Data »
Tamara Broderick
