Timezone: »
We tackle the problem of optimizing a black-box objective function defined over a highly-structured input space. This problem is ubiquitous in science and engineering. In machine learning, inferring the structure of a neural network or the Automatic Statistician (AS), where the optimal kernel combination for a Gaussian process is selected, are two important examples. We use the \as as a case study to describe our approach, that can be easily generalized to other domains. We propose an Structure Generating Variational Auto-encoder (SG-VAE) to embed the original space of kernel combinations into some low-dimensional continuous manifold where Bayesian optimization (BO) ideas are used. This is possible when structural knowledge of the problem is available, which can be given via a simulator or any other form of generating potentially good solutions. The right exploration-exploitation balance is imposed by propagating into the search the uncertainty of the latent space of the SG-VAE, that is computed using variational inference. The key aspect of our approach is that the SG-VAE can be used to bias the search towards relevant regions, making it suitable for transfer learning tasks. Several experiments in various application domains are used to illustrate the utility and generality of the approach described in this work.
Author Information
Xiaoyu Lu (University of Oxford)
Javier González (Amazon)
Zhenwen Dai (Amazon.com)
Neil Lawrence (Amazon)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Poster: Structured Variationally Auto-encoded Optimization »
Thu. Jul 12th 04:15 -- 07:00 PM Room Hall B
More from the Same Authors
-
2017 Poster: Preferential Bayesian Optmization »
Javier González · Zhenwen Dai · Andreas Damianou · Neil Lawrence -
2017 Poster: Bayesian Optimization with Tree-structured Dependencies »
Rodolphe Jenatton · Cedric Archambeau · Javier González · Matthias Seeger -
2017 Talk: Bayesian Optimization with Tree-structured Dependencies »
Rodolphe Jenatton · Cedric Archambeau · Javier González · Matthias Seeger -
2017 Talk: Preferential Bayesian Optmization »
Javier González · Zhenwen Dai · Andreas Damianou · Neil Lawrence