Timezone: »

Up or Down? Adaptive Rounding for Post-Training Quantization
Markus Nagel · Rana Ali Amjad · Marinus van Baalen · Christos Louizos · Tijmen Blankevoort

Wed Jul 15 01:00 PM -- 01:45 PM & Thu Jul 16 12:00 AM -- 12:45 AM (PDT) @

When quantizing neural networks, assigning each floating-point weight to its nearest fixed-point value is the predominant approach. We find that, perhaps surprisingly, this is not the best we can do. In this paper, we propose AdaRound, a better weight-rounding mechanism for post-training quantization that adapts to the data and the task loss. AdaRound is fast, does not require fine-tuning of the network, and only uses a small amount of unlabelled data. We start by theoretically analyzing the rounding problem for a pre-trained neural network. By approximating the task loss with a Taylor series expansion, the rounding task is posed as a quadratic unconstrained binary optimization problem. We simplify this to a layer-wise local loss and propose to optimize this loss with a soft relaxation. AdaRound not only outperforms rounding-to-nearest by a significant margin but also establishes a new state-of-the-art for post-training quantization on several networks and tasks. Without fine-tuning, we can quantize the weights of Resnet18 and Resnet50 to 4 bits while staying within an accuracy loss of 1%.

Author Information

Markus Nagel (Qualcomm AI Research)
Rana Ali Amjad (Qualcomm)
Marinus van Baalen (Qualcomm)
Christos Louizos (Qualcomm AI Research)
Tijmen Blankevoort (Qualcomm)

More from the Same Authors