Timezone: »

 
Poster
SDQ: Stochastic Differentiable Quantization with Mixed Precision
Xijie Huang · Zhiqiang Shen · Shichao Li · Zechun Liu · Hu Xianghong · Jeffry Wicaksana · Eric Xing · Kwang-Ting Cheng

Wed Jul 20 03:30 PM -- 05:30 PM (PDT) @ Hall E #218
in Poster Session 2 »

In order to deploy deep models in a computationally efficient manner, model quantization approaches have been frequently used. In addition, as new hardware that supports various-bit arithmetic operations, recent research on mixed precision quantization (MPQ) begins to fully leverage the capacity of representation by searching various bitwidths for different layers and modules in a network. However, previous studies mainly search the MPQ strategy in a costly scheme using reinforcement learning, neural architecture search, etc., or simply utilize partial prior knowledge for bitwidth distribution, which might be biased and sub-optimal. In this work, we present a novel Stochastic Differentiable Quantization (SDQ) method that can automatically learn the MPQ strategy in a more flexible and globally-optimized space with a smoother gradient approximation. Particularly, Differentiable Bitwidth Parameters (DBPs) are employed as the probability factors in stochastic quantization between adjacent bitwidth. After the optimal MPQ strategy is acquired, we further train our network with the entropy-aware bin regularization and knowledge distillation. We extensively evaluate our method on different networks, hardwares (GPUs and FPGA), and datasets. SDQ outperforms all other state-of-the-art mixed or single precision quantization with less bitwidth, and are even better than the original full-precision counterparts across various ResNet and MobileNet families, demonstrating the effectiveness and superiority of our method. Code will be publicly available.

Keywords: APP: Computer Vision · DL: Everything Else

Author Information

Xijie Huang (HKUST)
Zhiqiang Shen (Carnegie Mellon University)
Shichao Li (Hong Kong University of Science and Technology)

Shichao Li is a PhD candidate in the Department of Computer Science and Engineering, HKUST. He is affiliated with VSDL@HKUST and am advised by Prof. Kwang-Ting Cheng. Before his PhD study in HKUST, he obtained his B. E. degree in 2017 at Chu Kochen Honors College, Zhejiang University, where he was an undergraduate research assistant working on physics-based modeling and numerical algorithms and was supervised by Prof. Wen-Yan Yin and Prof. Sailing He.

Zechun Liu (Carnegie Mellon University)
Hu Xianghong (HKUST)
Jeffry Wicaksana (Hong Kong University of Science and Technology)
Eric Xing (Petuum Inc. and CMU)
Kwang-Ting Cheng (Hong Kong University of Science and Technology)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors