Timezone: »
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target distribution. This approach was recently combined with the theory of reproducing kernels to define a closed-form kernel Stein discrepancy (KSD) computable by summing kernel evaluations across pairs of sample points. We develop a theory of weak convergence for KSDs based on Stein’s method, demonstrate that commonly used KSDs fail to detect non-convergence even for Gaussian targets, and show that kernels with slowly decaying tails provably determine convergence for a large class of target distributions. The resulting convergence-determining KSDs are suitable for comparing biased, exact, and deterministic sample sequences and simpler to compute and parallelize than alternative Stein discrepancies. We use our tools to compare biased samplers, select sampler hyperparameters, and improve upon existing KSD approaches to one-sample hypothesis testing and sample quality improvement.
Summary/NotesAuthor Information
Jackson Gorham (STANFORD)
Lester Mackey (Microsoft Research)
Lester Mackey is a machine learning researcher at Microsoft Research, where he develops new tools, models, and theory for large-scale learning tasks driven by applications from healthcare, climate, recommender systems, and the social good. Lester moved to Microsoft from Stanford University, where he was an assistant professor of Statistics and (by courtesy) of Computer Science. He earned his PhD in Computer Science and MA in Statistics from UC Berkeley and his BSE in Computer Science from Princeton University. He co-organized the second place team in the \$1M. Netflix Prize competition for collaborative filtering, won the \$50K Prise4Life ALS disease progression prediction challenge, won prizes for temperature and precipitation forecasting in the yearlong real-time \$800K Subseasonal Climate Forecast Rodeo, and received a best student paper award at the International Conference on Machine Learning.
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Poster: Measuring Sample Quality with Kernels »
Tue. Aug 8th 08:30 AM -- 12:00 PM Room Gallery #101
More from the Same Authors
-
2021 : Kernel Thinning »
Raaz Dwivedi · Lester Mackey -
2021 : Kernel Thinning »
Lester Mackey · Raaz Dwivedi -
2021 Poster: Online Learning with Optimism and Delay »
Genevieve Flaspohler · Francesco Orabona · Judah Cohen · Soukayna Mouatadid · Miruna Oprescu · Paulo Orenstein · Lester Mackey -
2021 Spotlight: Online Learning with Optimism and Delay »
Genevieve Flaspohler · Francesco Orabona · Judah Cohen · Soukayna Mouatadid · Miruna Oprescu · Paulo Orenstein · Lester Mackey -
2020 Poster: Single Point Transductive Prediction »
Nilesh Tripuraneni · Lester Mackey -
2020 Invited Talk: Doing Some Good with Machine Learning »
Lester Mackey -
2019 Workshop: AI For Social Good (AISG) »
Margaux Luck · Kris Sankaran · Tristan Sylvain · Sean McGregor · Jonnie Penn · Girmaw Abebe Tadesse · Virgile Sylvain · Myriam Côté · Lester Mackey · Rayid Ghani · Yoshua Bengio -
2019 : Networking Lunch (provided) + Poster Session »
Abraham Stanway · Alex Robson · Aneesh Rangnekar · Ashesh Chattopadhyay · Ashley Pilipiszyn · Benjamin LeRoy · Bolong Cheng · Ce Zhang · Chaopeng Shen · Christian Schroeder · Christian Clough · Clement DUHART · Clement Fung · Cozmin Ududec · Dali Wang · David Dao · di wu · Dimitrios Giannakis · Dino Sejdinovic · Doina Precup · Duncan Watson-Parris · Gege Wen · George Chen · Gopal Erinjippurath · Haifeng Li · Han Zou · Herke van Hoof · Hillary A Scannell · Hiroshi Mamitsuka · Hongbao Zhang · Jaegul Choo · James Wang · James Requeima · Jessica Hwang · Jinfan Xu · Johan Mathe · Jonathan Binas · Joonseok Lee · Kalai Ramea · Kate Duffy · Kevin McCloskey · Kris Sankaran · Lester Mackey · Letif Mones · Loubna Benabbou · Lynn Kaack · Matthew Hoffman · Mayur Mudigonda · Mehrdad Mahdavi · Michael McCourt · Mingchao Jiang · Mohammad Mahdi Kamani · Neel Guha · Niccolo Dalmasso · Nick Pawlowski · Nikola Milojevic-Dupont · Paulo Orenstein · Pedram Hassanzadeh · Pekka Marttinen · Ramesh Nair · Sadegh Farhang · Samuel Kaski · Sandeep Manjanna · Sasha Luccioni · Shuby Deshpande · Soo Kim · Soukayna Mouatadid · Sunghyun Park · Tao Lin · Telmo Felgueira · Thomas Hornigold · Tianle Yuan · Tom Beucler · Tracy Cui · Volodymyr Kuleshov · Wei Yu · yang song · Ydo Wexler · Yoshua Bengio · Zhecheng Wang · Zhuangfang Yi · Zouheir Malki -
2019 Poster: Stein Point Markov Chain Monte Carlo »
Wilson Ye Chen · Alessandro Barp · Francois-Xavier Briol · Jackson Gorham · Mark Girolami · Lester Mackey · Chris Oates -
2019 Oral: Stein Point Markov Chain Monte Carlo »
Wilson Ye Chen · Alessandro Barp · Francois-Xavier Briol · Jackson Gorham · Mark Girolami · Lester Mackey · Chris Oates -
2018 Poster: Stein Points »
Wilson Ye Chen · Lester Mackey · Jackson Gorham · Francois-Xavier Briol · Chris J Oates -
2018 Oral: Stein Points »
Wilson Ye Chen · Lester Mackey · Jackson Gorham · Francois-Xavier Briol · Chris J Oates -
2017 Poster: Improving Gibbs Sampler Scan Quality with DoGS »
Ioannis Mitliagkas · Lester Mackey -
2017 Talk: Improving Gibbs Sampler Scan Quality with DoGS »
Ioannis Mitliagkas · Lester Mackey