Timezone: »

 
Spotlight
Zero-shot AutoML with Pretrained Models
Ekrem Öztürk · Fabio Ferreira · Hadi S Jomaa · Lars Schmidt-Thieme · Josif Grabocka · Frank Hutter

Tue Jul 19 08:40 AM -- 08:45 AM (PDT) @ Room 318 - 320
in Transfer/Multitask/Meta Learning »

Given a new dataset D and a low compute budget, how should we choose a pre-trained model to fine-tune to D, and set the fine-tuning hyperparameters without risking overfitting, particularly if D is small? Here, we extend automated machine learning (AutoML) to best make these choices. Our domain-independent meta-learning approach learns a zero-shot surrogate model which, at test time, allows to select the right deep learning (DL) pipeline (including the pre-trained model and fine-tuning hyperparameters) for a new dataset D given only trivial meta-features describing D such as image resolution or the number of classes. To train this zero-shot model, we collect performance data for many DL pipelines on a large collection of datasets and meta-train on this data to minimize a pairwise ranking objective. We evaluate our approach under the strict time limit of the vision track of the ChaLearn AutoDL challenge benchmark, clearly outperforming all challenge contenders.

Keywords: OPT: Zero-order and Black-box Optimization · MISC: Transfer, Multitask and Meta-learning

Author Information

Ekrem Öztürk (University of Freiburg)
Fabio Ferreira (University of Freiburg)
Hadi S Jomaa (Stiftung Universitat Hildesheim)
Lars Schmidt-Thieme (University of Hildesheim)
Josif Grabocka (Albert-Ludwigs-Universität Freiburg)
Frank Hutter (University of Freiburg and Bosch Center for Artificial Intelligence)
Frank Hutter

Frank Hutter is a Full Professor for Machine Learning at the Computer Science Department of the University of Freiburg (Germany), where he has been a faculty member since 2013. Before that, he was at the University of British Columbia (UBC) for eight years, for his PhD and postdoc. Frank's main research interests lie in machine learning, artificial intelligence and automated algorithm design. For his 2009 PhD thesis on algorithm configuration, he received the CAIAC doctoral dissertation award for the best thesis in AI in Canada that year, and with his coauthors, he received several best paper awards and prizes in international competitions on automated machine learning, SAT solving, and AI planning. Since 2016 he holds an ERC Starting Grant for a project on automating deep learning based on Bayesian optimization, Bayesian neural networks, and deep reinforcement learning.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors