Timezone: »

 
Spotlight
MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection
Zhenhong Sun · Ming Lin · Xiuyu Sun · Zhiyu Tan · Hao Li · rong jin

Wed Jul 20 08:30 AM -- 08:35 AM (PDT) @ Hall G
in Applications »
In object detection, the detection backbone consumes more than half of the overall inference cost. Recent researches attempt to reduce this cost by optimizing the backbone architecture with the help of Neural Architecture Search (NAS). However, existing NAS methods for object detection require hundreds to thousands of GPU hours of searching, making them impractical in fast-paced research and development. In this work, we propose a novel zero-shot NAS method to address this issue. The proposed method, named MAE-DET, automatically designs efficient detection backbones via the Maximum Entropy Principle without training network parameters, reducing the architecture design cost to nearly zero yet delivering the state-of-the-art (SOTA) performance. Under the hood, MAE-DET maximizes the differential entropy of detection backbones, leading to a better feature extractor for object detection under the same computational budgets. After merely one GPU day of fully automatic design, MAE-DET innovates SOTA detection backbones on multiple detection benchmark datasets with little human intervention. Comparing to ResNet-50 backbone, MAE-DET is $+2.0\%$ better in mAP when using the same amount of FLOPs/parameters, and is $1.54$ times faster on NVIDIA V100 at the same mAP. Code and pre-trained models are available here (https://github.com/alibaba/lightweight-neural-architecture-search).
Keywords: APP: Computer Vision · DL: Theory

Author Information

Zhenhong Sun (Alibaba Group)
Ming Lin (Alibaba Group)

I am a Staff Algorithm Engineer at DAMO Academy of Alibaba Group (U.S.). My research interests include Mathematical Foundation of Deep Learning and Statistical Machine Learning, with their applications in Neural Architecture Search, Computer Vision, Mobile AI and any other real-world problems. Before I joined Alibaba Group (U.S.), I was a Research Investigator in the Medical School of Michigan University until April 2019. I worked as a postdoctoral researcher in the School of Computer Science at Carnegie Mellon University from July 2014 to Sep 2015. I received my Ph.D. degree in computer science from Tsinghua University in 2014. During my Ph.D. study, I had been a visiting scholar in Michigan State University and in CMU from Dec 2013 to July 2014.

Xiuyu Sun (Alibaba Group)
Zhiyu Tan (Alibaba Group)
Hao Li (Alibaba Group)
rong jin (alibaba group)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors