Timezone: »

 
Poster
BORE: Bayesian Optimization by Density-Ratio Estimation
Louis Chi-Chun Tiao · Aaron Klein · Matthias W Seeger · Edwin V. Bonilla · Cedric Archambeau · Fabio Ramos

Tue Jul 20 09:00 AM -- 11:00 AM (PDT) @
in Poster Session 1 »

Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. BO proposes solutions according to an explore-exploit trade-off criterion encoded in an acquisition function, many of which are computed from the posterior predictive of a probabilistic surrogate model. Prevalent among these is the expected improvement (EI). The need to ensure analytical tractability of the predictive often poses limitations that can hinder the efficiency and applicability of BO. In this paper, we cast the computation of EI as a binary classification problem, building on the link between class-probability estimation and density-ratio estimation, and the lesser-known link between density-ratios and EI. By circumventing the tractability constraints, this reformulation provides numerous advantages, not least in terms of expressiveness, versatility, and scalability.

Keywords: Optimization · Algorithms · AutoML · Algorithms; Optimization -> Convex Optimization; Optimization -> Stochastic Optimization; Theory · Computational Complexity

Author Information

Louis Chi-Chun Tiao (University of Sydney)
Aaron Klein (AWS Berlin)
Matthias W Seeger (Amazon)
Edwin V. Bonilla (CSIRO's Data61)

I am a Science Leader for Foundational Machine Learning of the Analytics and Decision Sciences Research program at CSIRO’s Data61, Australia. My expertise is in probabilistic modelling and inference algorithms for the analysis of complex data, in areas such as scalable Bayesian inference, Gaussian processes and multi-task learning. I have worked in applications such as geophysical inversions, spatio-temporal modelling, computer vision and document analysis. My current interests include Gaussian processes, Bayesian optimization, optimal design of experiments, neural differential equations and graph neural networks.

Cedric Archambeau (Amazon Web Services)
Fabio Ramos (NVIDIA, University of Sydney)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors