HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search

Niv Nayman · Yonathan Aflalo · Asaf Noy · Lihi Zelnik

Keywords: [ AutoML ] [ Deep Learning ] [ Algorithms ] [ Deep Learning -> Biologically Plausible Deep Networks; Deep Learning ] [ CNN Architectures ]

[ Abstract ]
[ Paper ]
[ Visit Poster at Spot A1 in Virtual World ]
Tue 20 Jul 9 a.m. PDT — 11 a.m. PDT
Spotlight presentation: Auto-ML and Optimization
Tue 20 Jul 5 a.m. PDT — 6 a.m. PDT


Realistic use of neural networks often requires adhering to multiple constraints on latency, energy and memory among others. A popular approach to find fitting networks is through constrained Neural Architecture Search (NAS), however, previous methods enforce the constraint only softly. Therefore, the resulting networks do not exactly adhere to the resource constraint and their accuracy is harmed. In this work we resolve this by introducing Hard Constrained diffeRentiable NAS (HardCoRe-NAS), that is based on an accurate formulation of the expected resource requirement and a scalable search method that satisfies the hard constraint throughout the search. Our experiments show that HardCoRe-NAS generates state-of-the-art architectures, surpassing other NAS methods, while strictly satisfying the hard resource constraints without any tuning required.

Chat is not available.