Timezone: »
Talk
Multi-fidelity Bayesian Optimisation with Continuous Approximations
kirthevasan kandasamy · Gautam Dasarathy · Barnabás Póczos · Jeff Schneider
Bandit methods for black-box optimisation, such as Bayesian optimisation,
are used in a variety of applications including hyper-parameter tuning and
experiment design.
Recently, \emph{multi-fidelity} methods have garnered
considerable attention since function evaluations have become increasingly expensive in
such applications.
Multi-fidelity methods use cheap approximations to the function of
interest to speed up the overall optimisation process.
However, most multi-fidelity methods assume only a finite number of approximations.
On the other hand, in many practical applications, a continuous spectrum of approximations might be
available.
For instance, when tuning an expensive neural network, one might choose to approximate the
cross validation performance using less data $N$ and/or few training iterations $T$.
Here, the approximations are best viewed as arising out of a continuous two dimensional
space $(N,T)$.
In this work, we develop a Bayesian optimisation method, \boca, for this setting.
We characterise its theoretical properties and show that it achieves better regret than
than strategies which ignore the approximations.
\bocas outperforms several other baselines in synthetic and real experiments.
Author Information
kirthevasan kandasamy (CMU)
Gautam Dasarathy (Rice University)
Barnabás Póczos (CMU)
Jeff Schneider (CMU/Uber)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Poster: Multi-fidelity Bayesian Optimisation with Continuous Approximations »
Wed. Aug 9th 08:30 AM -- 12:00 PM Room Gallery #49
More from the Same Authors
-
2023 : Distributional Distance Classifiers for Goal-Conditioned Reinforcement Learning »
Ravi Tej Akella · Benjamin Eysenbach · Jeff Schneider · Ruslan Salakhutdinov -
2023 : Kernelized Offline Contextual Dueling Bandits »
Viraj Mehta · Ojash Neopane · Vikramjeet Das · Sen Lin · Jeff Schneider · Willie Neiswanger -
2023 Poster: Learning Temporally AbstractWorld Models without Online Experimentation »
Benjamin Freed · Siddarth Venkatraman · Guillaume Sartoretti · Jeff Schneider · Howie Choset -
2020 Poster: VideoOneNet: Bidirectional Convolutional Recurrent OneNet with Trainable Data Steps for Video Processing »
Zoltán Á. Milacski · Barnabás Póczos · Andras Lorincz -
2019 Poster: Myopic Posterior Sampling for Adaptive Goal Oriented Design of Experiments »
Kirthevasan Kandasamy · Willie Neiswanger · Reed Zhang · Akshay Krishnamurthy · Jeff Schneider · Barnabás Póczos -
2019 Oral: Myopic Posterior Sampling for Adaptive Goal Oriented Design of Experiments »
Kirthevasan Kandasamy · Willie Neiswanger · Reed Zhang · Akshay Krishnamurthy · Jeff Schneider · Barnabás Póczos -
2018 Poster: Multi-Fidelity Black-Box Optimization with Hierarchical Partitions »
Rajat Sen · kirthevasan kandasamy · Sanjay Shakkottai -
2018 Poster: Transformation Autoregressive Networks »
Junier Oliva · Kumar Avinava Dubey · Manzil Zaheer · Barnabás Póczos · Ruslan Salakhutdinov · Eric Xing · Jeff Schneider -
2018 Oral: Transformation Autoregressive Networks »
Junier Oliva · Kumar Avinava Dubey · Manzil Zaheer · Barnabás Póczos · Ruslan Salakhutdinov · Eric Xing · Jeff Schneider -
2018 Oral: Multi-Fidelity Black-Box Optimization with Hierarchical Partitions »
Rajat Sen · kirthevasan kandasamy · Sanjay Shakkottai -
2018 Poster: Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima »
Simon Du · Jason Lee · Yuandong Tian · Aarti Singh · Barnabás Póczos -
2018 Poster: Ultra Large-Scale Feature Selection using Count-Sketches »
Amirali Aghazadeh · Ryan Spring · Daniel LeJeune · Gautam Dasarathy · Anshumali Shrivastava · Richard Baraniuk -
2018 Oral: Ultra Large-Scale Feature Selection using Count-Sketches »
Amirali Aghazadeh · Ryan Spring · Daniel LeJeune · Gautam Dasarathy · Anshumali Shrivastava · Richard Baraniuk -
2018 Oral: Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima »
Simon Du · Jason Lee · Yuandong Tian · Aarti Singh · Barnabás Póczos -
2017 Poster: The Statistical Recurrent Unit »
Junier Oliva · Barnabás Póczos · Jeff Schneider -
2017 Poster: Nonparanormal Information Estimation »
Shashank Singh · Barnabás Póczos -
2017 Talk: Nonparanormal Information Estimation »
Shashank Singh · Barnabás Póczos -
2017 Talk: The Statistical Recurrent Unit »
Junier Oliva · Barnabás Póczos · Jeff Schneider -
2017 Poster: Equivariance Through Parameter-Sharing »
Siamak Ravanbakhsh · Jeff Schneider · Barnabás Póczos -
2017 Talk: Equivariance Through Parameter-Sharing »
Siamak Ravanbakhsh · Jeff Schneider · Barnabás Póczos