Timezone: »
Model quantization is challenging due to many tedious hyper-parameters such as precision (bitwidth), dynamic range (minimum and maximum discrete values) and stepsize (interval between discrete values). Unlike prior arts that carefully tune these values, we present a fully differentiable approach to learn all of them, named Differentiable Dynamic Quantization (DDQ), which has several benefits. (1) DDQ is able to quantize challenging lightweight architectures like MobileNets, where different layers prefer different quantization parameters. (2) DDQ is hardware-friendly and can be easily implemented using low-precision matrix-vector multiplication, making it capable in many hardware such as ARM. (3) Extensive experiments show that DDQ outperforms prior arts on many networks and benchmarks, especially when models are already efficient and compact. e.g., DDQ is the first approach that achieves lossless 4-bit quantization for MobileNetV2 on ImageNet.
Author Information
zhaoyang zhang (The Chinese University of Hong Kong)
Wenqi Shao (The Chinese University of HongKong)
Jinwei Gu (Sensebrain)
Xiaogang Wang (Chinese University of Hong Kong, Hong Kong)
Ping Luo (The University of Hong Kong)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution »
Thu. Jul 22nd 02:25 -- 02:30 PM Room
More from the Same Authors
-
2022 Poster: CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer »
Yao Mu · Shoufa Chen · Mingyu Ding · Jianyu Chen · Runjian Chen · Ping Luo -
2022 Poster: VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix »
Teng Wang · Wenhao Jiang · Zhichao Lu · Feng Zheng · Ran Cheng · chengguo yin · Ping Luo -
2022 Spotlight: VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix »
Teng Wang · Wenhao Jiang · Zhichao Lu · Feng Zheng · Ran Cheng · chengguo yin · Ping Luo -
2022 Spotlight: CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer »
Yao Mu · Shoufa Chen · Mingyu Ding · Jianyu Chen · Runjian Chen · Ping Luo -
2021 Poster: What Makes for End-to-End Object Detection? »
Peize Sun · Yi Jiang · Enze Xie · Wenqi Shao · Zehuan Yuan · Changhu Wang · Ping Luo -
2021 Spotlight: What Makes for End-to-End Object Detection? »
Peize Sun · Yi Jiang · Enze Xie · Wenqi Shao · Zehuan Yuan · Changhu Wang · Ping Luo -
2020 Poster: Channel Equilibrium Networks for Learning Deep Representation »
Wenqi Shao · Shitao Tang · Xingang Pan · Ping Tan · Xiaogang Wang · Ping Luo