Timezone: »
Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. BO proposes solutions according to an explore-exploit trade-off criterion encoded in an acquisition function, many of which are computed from the posterior predictive of a probabilistic surrogate model. Prevalent among these is the expected improvement (EI). The need to ensure analytical tractability of the predictive often poses limitations that can hinder the efficiency and applicability of BO. In this paper, we cast the computation of EI as a binary classification problem, building on the link between class-probability estimation and density-ratio estimation, and the lesser-known link between density-ratios and EI. By circumventing the tractability constraints, this reformulation provides numerous advantages, not least in terms of expressiveness, versatility, and scalability.
Author Information
Louis Chi-Chun Tiao (University of Sydney)
Aaron Klein (AWS Berlin)
Matthias W Seeger (Amazon)
Edwin V. Bonilla (CSIRO's Data61)
I am a Science Leader for Foundational Machine Learning of the Analytics and Decision Sciences Research program at CSIRO’s Data61, Australia. My expertise is in probabilistic modelling and inference algorithms for the analysis of complex data, in areas such as scalable Bayesian inference, Gaussian processes and multi-task learning. I have worked in applications such as geophysical inversions, spatio-temporal modelling, computer vision and document analysis. My current interests include Gaussian processes, Bayesian optimization, optimal design of experiments, neural differential equations and graph neural networks.
Cedric Archambeau (Amazon Web Services)
Fabio Ramos (NVIDIA, University of Sydney)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Oral: BORE: Bayesian Optimization by Density-Ratio Estimation »
Tue. Jul 20th 12:00 -- 12:20 PM Room
More from the Same Authors
-
2021 : Dynamic Pruning of a Neural Network via Gradient Signal-to-Noise Ratio »
Julien N Siems · Aaron Klein · Cedric Archambeau · Maren Mahsereci -
2023 Oral: Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes »
Louis Chi-Chun Tiao · Vincent Dutordoir · Victor Picheny -
2023 Poster: Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes »
Louis Chi-Chun Tiao · Vincent Dutordoir · Victor Picheny -
2023 Poster: Transformed Distribution Matching for Missing Value Imputation »
He Zhao · Ke Sun · Amir Dezfouli · Edwin V. Bonilla -
2023 Poster: Optimizing Hyperparameters with Conformal Quantile Regression »
David Salinas · Jacek Golebiowski · Aaron Klein · Matthias Seeger · Cedric Archambeau -
2023 Poster: Free-Form Variational Inference for Gaussian Process State-Space Models »
Xuhui Fan · Edwin V. Bonilla · Terence O'kane · Scott SIsson -
2022 Poster: Learning Efficient and Robust Ordinary Differential Equations via Invertible Neural Networks »
Weiming Zhi · Tin Lai · Lionel Ott · Edwin V. Bonilla · Fabio Ramos -
2022 Spotlight: Learning Efficient and Robust Ordinary Differential Equations via Invertible Neural Networks »
Weiming Zhi · Tin Lai · Lionel Ott · Edwin V. Bonilla · Fabio Ramos -
2022 Poster: Optimizing Sequential Experimental Design with Deep Reinforcement Learning »
Tom Blau · Edwin V. Bonilla · Iadine Chades · Amir Dezfouli -
2022 Spotlight: Optimizing Sequential Experimental Design with Deep Reinforcement Learning »
Tom Blau · Edwin V. Bonilla · Iadine Chades · Amir Dezfouli -
2021 Poster: SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data »
Maud Lemercier · Cristopher Salvi · Thomas Cass · Edwin V. Bonilla · Theodoros Damoulas · Terry Lyons -
2021 Spotlight: SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data »
Maud Lemercier · Cristopher Salvi · Thomas Cass · Edwin V. Bonilla · Theodoros Damoulas · Terry Lyons -
2020 Poster: LEEP: A New Measure to Evaluate Transferability of Learned Representations »
Cuong Nguyen · Tal Hassner · Matthias W Seeger · Cedric Archambeau -
2020 Affinity Workshop: New In ML »
Zhen Xu · Sparkle Russell-Puleri · Zhengying Liu · Sinead A Williamson · Matthias W Seeger · Wei-Wei Tu · Samy Bengio · Isabelle Guyon -
2019 : Poster Session 1 (all papers) »
Matilde Gargiani · Yochai Zur · Chaim Baskin · Evgenii Zheltonozhskii · Liam Li · Ameet Talwalkar · Xuedong Shang · Harkirat Singh Behl · Atilim Gunes Baydin · Ivo Couckuyt · Tom Dhaene · Chieh Lin · Wei Wei · Min Sun · Orchid Majumder · Michele Donini · Yoshihiko Ozaki · Ryan P. Adams · Christian Geißler · Ping Luo · zhanglin peng · · Ruimao Zhang · John Langford · Rich Caruana · Debadeepta Dey · Charles Weill · Xavi Gonzalvo · Scott Yang · Scott Yak · Eugen Hotaj · Vladimir Macko · Mehryar Mohri · Corinna Cortes · Stefan Webb · Jonathan Chen · Martin Jankowiak · Noah Goodman · Aaron Klein · Frank Hutter · Mojan Javaheripi · Mohammad Samragh · Sungbin Lim · Taesup Kim · SUNGWOONG KIM · Michael Volpp · Iddo Drori · Yamuna Krishnamurthy · Kyunghyun Cho · Stanislaw Jastrzebski · Quentin de Laroussilhe · Mingxing Tan · Xiao Ma · Neil Houlsby · Andrea Gesmundo · Zalán Borsos · Krzysztof Maziarz · Felipe Petroski Such · Joel Lehman · Kenneth Stanley · Jeff Clune · Pieter Gijsbers · Joaquin Vanschoren · Felix Mohr · Eyke Hüllermeier · Zheng Xiong · Wenpeng Zhang · Wenwu Zhu · Weijia Shao · Aleksandra Faust · Michal Valko · Michael Y Li · Hugo Jair Escalante · Marcel Wever · Andrey Khorlin · Tara Javidi · Anthony Francis · Saurajit Mukherjee · Jungtaek Kim · Michael McCourt · Saehoon Kim · Tackgeun You · Seungjin Choi · Nicolas Knudde · Alexander Tornede · Ghassen Jerfel -
2019 Poster: NAS-Bench-101: Towards Reproducible Neural Architecture Search »
Chris Ying · Aaron Klein · Eric Christiansen · Esteban Real · Kevin Murphy · Frank Hutter -
2019 Oral: NAS-Bench-101: Towards Reproducible Neural Architecture Search »
Chris Ying · Aaron Klein · Eric Christiansen · Esteban Real · Kevin Murphy · Frank Hutter -
2019 Poster: Bayesian Deconditional Kernel Mean Embeddings »
Kelvin Hsu · Fabio Ramos -
2019 Oral: Bayesian Deconditional Kernel Mean Embeddings »
Kelvin Hsu · Fabio Ramos -
2018 Poster: Variational Network Inference: Strong and Stable with Concrete Support »
Amir Dezfouli · Edwin Bonilla · Richard Nock -
2018 Poster: BOHB: Robust and Efficient Hyperparameter Optimization at Scale »
Stefan Falkner · Aaron Klein · Frank Hutter -
2018 Oral: BOHB: Robust and Efficient Hyperparameter Optimization at Scale »
Stefan Falkner · Aaron Klein · Frank Hutter -
2018 Oral: Variational Network Inference: Strong and Stable with Concrete Support »
Amir Dezfouli · Edwin Bonilla · Richard Nock -
2017 Poster: Random Feature Expansions for Deep Gaussian Processes »
Kurt Cutajar · Edwin Bonilla · Pietro Michiardi · Maurizio Filippone -
2017 Poster: Bayesian Optimization with Tree-structured Dependencies »
Rodolphe Jenatton · Cedric Archambeau · Javier González · Matthias Seeger -
2017 Talk: Random Feature Expansions for Deep Gaussian Processes »
Kurt Cutajar · Edwin Bonilla · Pietro Michiardi · Maurizio Filippone -
2017 Talk: Bayesian Optimization with Tree-structured Dependencies »
Rodolphe Jenatton · Cedric Archambeau · Javier González · Matthias Seeger