Timezone: »
We present a theoretically founded approach for high-dimensional Bayesian optimization based on low-dimensional subspace embeddings. We prove that the error in the Gaussian process model is bounded tightly when going from the original high-dimensional search domain to the low-dimensional embedding. This implies that the optimization process in the low-dimensional embedding proceeds essentially as if it were run directly on an unknown active subspace of low dimensionality. The argument applies to a large class of algorithms and GP models, including non-stationary kernels. Moreover, we provide an efficient implementation based on hashing and demonstrate empirically that this subspace embedding achieves considerably better results than the previously proposed methods for high-dimensional BO based on Gaussian matrix projections and structure-learning.
Author Information
Amin Nayebi (University of Arizona)
Alexander Munteanu (TU Dortmund)
Matthias Poloczek (Uber AI Labs & The University of Arizona)
**Matthias Poloczek** is a Principal Scientist at Amazon. He works in **machine learning and optimization** with applications in Robotics, Advertisement, Recommender Systems and more. Moreover, Matthias leads large-scale initiative on ML Operational Excellence. Previously, Matthias was a Senior Manager at Uber AI where he led the AutoML initiative and drove impactful solutions based on Bayesian optimization.
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: A Framework for Bayesian Optimization in Embedded Subspaces »
Wed. Jun 12th 12:00 -- 12:05 AM Room Room 101
More from the Same Authors
-
2022 Poster: Bounding the Width of Neural Networks via Coupled Initialization - A Worst Case Analysis »
Alexander Munteanu · Simon Omlor · Zhao Song · David Woodruff -
2022 Spotlight: Bounding the Width of Neural Networks via Coupled Initialization - A Worst Case Analysis »
Alexander Munteanu · Simon Omlor · Zhao Song · David Woodruff -
2021 Poster: Oblivious Sketching for Logistic Regression »
Alexander Munteanu · Simon Omlor · David Woodruff -
2021 Spotlight: Oblivious Sketching for Logistic Regression »
Alexander Munteanu · Simon Omlor · David Woodruff -
2018 Poster: Bayesian Optimization of Combinatorial Structures »
Ricardo Baptista · Matthias Poloczek -
2018 Oral: Bayesian Optimization of Combinatorial Structures »
Ricardo Baptista · Matthias Poloczek