Dissecting Quantization Error: A Concentration-Alignment Perspective

Quantization can drastically increase the efficiency of large language and vision models, but typically incurs an accuracy drop. Recently, function-preserving transforms (e.g. rotations, Hadamard transform, channel-wise scaling) have been successfully applied to reduce post-training quantization error, yet a principled explanation remains elusive. We analyze linear-layer quantization via the signal-to-quantization-noise ratio (SQNR), showing that for uniform integer quantization at a fixed bit width, SQNR decomposes into (i) the concentration of weights and activations (capturing spread and outliers), and (ii) the alignment of their dominant variation directions. This provides an actionable insight: enhancing alignment between weight and activation variation directions can reduce quantization error, complementing concentration-focused approaches. Motivated by this, we introduce Concentration–Alignment Transforms (CAT), a lightweight linear transformation that uses a covariance estimate from a small calibration set to jointly improve concentration and alignment, approximately maximizing SQNR. Experiments across several LLMs show that CAT consistently matches or outperforms prior transform-based quantization methods at 4-bit precision.

Using Large Language models and other recent architectures requires extensive computational resources and memory. This can be mitigated by reducing the number of bits used to represent the neural network weights and their internal representations. However, this usually comes at the cost of worse performance. This paper aims to better understand this type of error by identifying two main causes, referred to as "Concentration" and "Alignment" and devising a simple solution to improve on both aspects by minimally changing the original neural networks. We demonstrate the effectiveness our our method against similar approaches in literature and other methods that require significantly more resources to partially restore the network accuracy.