Timezone: »

 
Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization
Sergio Izquierdo · Julia Guerrero-Viu · Sven Hauns · Guilherme Miotto · Simon Schrodi · André Biedenkapp · Thomas Elsken · Difan Deng · Marius Lindauer · Frank Hutter

@
in 8th ICML Workshop on Automated Machine Learning (AutoML 2021) »

While both neural architecture search (NAS) and hyperparameter optimization (HPO) have been studied extensively in recent years, NAS methods typically assume fixed hyperparameters and vice versa. Furthermore, NAS has recently often been framed as a multi-objective optimization problem, in order to take, e.g., resource requirements into account. In this paper, we propose a set of methods that extend current approaches to jointly optimize neural architectures and hyperparameters with respect to multiple objectives. We hope that these methods will serve as simple baselines for future research on multi-objective joint NAS + HPO.

Author Information

Sergio Izquierdo
Julia Guerrero-Viu (Universität Freiburg)
Sven Hauns
Guilherme Miotto
Simon Schrodi (CS Department, University of Freiburg, Germany, Universität Freiburg)
André Biedenkapp (University of Freiburg)

Since October 2017 I am a PhD student at the Machine Learning Group under the supervision of Frank Hutter and Marius Lindauer. Before that I completed my master and bachelor degrees in computer science at the University of Freiburg. Research Interests I am interested in all facets of artificial intelligence. My research focuses on new ways to control the behavior of algorithms online. More precisely my research areas include: Dynamic Algorithm Configuration/Algorithm Control Learning to Learn (Deep) Reinforcement Learning Bayesian Optimization Automated Hyperparameter Optimization

Thomas Elsken (Machine Learning Lab, University of Freiburg)
Difan Deng (Institut für Informationsverarbeitung, Leibniz Universität Hannover)
Marius Lindauer (Leibniz Universität Hannover)
Frank Hutter (University of Freiburg and Bosch Center for Artificial Intelligence)
Frank Hutter

Frank Hutter is a Full Professor for Machine Learning at the Computer Science Department of the University of Freiburg (Germany), where he has been a faculty member since 2013. Before that, he was at the University of British Columbia (UBC) for eight years, for his PhD and postdoc. Frank's main research interests lie in machine learning, artificial intelligence and automated algorithm design. For his 2009 PhD thesis on algorithm configuration, he received the CAIAC doctoral dissertation award for the best thesis in AI in Canada that year, and with his coauthors, he received several best paper awards and prizes in international competitions on automated machine learning, SAT solving, and AI planning. Since 2016 he holds an ERC Starting Grant for a project on automating deep learning based on Bayesian optimization, Bayesian neural networks, and deep reinforcement learning.

More from the Same Authors