Timezone: »

 
Poster
Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search
Yong Guo · Yaofo Chen · Yin Zheng · Peilin Zhao · Jian Chen · Junzhou Huang · Mingkui Tan

Wed Jul 15 08:00 AM -- 08:45 AM & Wed Jul 15 09:00 PM -- 09:45 PM (PDT) @
in Poster Session 22 »

Neural architecture search (NAS) has become an important approach to automatically find effective architectures. To cover all possible good architectures, we need to search in an extremely large search space with billions of candidate architectures. More critically, given a large search space, we may face a very challenging issue of space explosion. However, due to the limitation of computational resources, we can only sample a very small proportion of the architectures, which provides insufficient information for the training. As a result, existing methods may often produce suboptimal architectures. To alleviate this issue, we propose a curriculum search method that starts from a small search space and gradually incorporates the learned knowledge to guide the search in a large space. With the proposed search strategy, our Curriculum Neural Architecture Search (CNAS) method significantly improves the search efficiency and finds better architectures than existing NAS methods. Extensive experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of the proposed method.

Keywords: Algorithms · Architectures · Deep Learning - General

Author Information

Yong Guo (South China University of Technology)
Yaofo Chen (South China University of Technology)
Yin Zheng (Tencent)

Yin Zheng received the Ph.D degree from Tsinghua University under the supervision of Prof. Yu-Jin Zhang (Tsinghua University) and Prof. Hugo Larochelle (Google Brain). His research interest is Deep Learning, Machine Learning, Computer Vision, Artificial Intelligence and Recommender Systems. After graduation at 2015, he work as a researcher in the recommendation team of Hulu LLC. Then he joined Tencent AI Lab as a researcher in Machine Learning center.

Peilin Zhao (Artificial Intelligence Department, Ant ​Financial)
Jian Chen ("South China University of Technology, China")
Junzhou Huang (University of Texas at Arlington / Tencent AI Lab)
Mingkui Tan (South China University of Technology)

More from the Same Authors