COAT: COrrelation-Aware Orthogonal Transform for LLM Quantization

Quantization of large language models (LLMs) has become essential to mitigate memory and computational bottlenecks, however the occurrence of massive activation outliers poses the main challenge in low-bit LLM quantization. In the recent works, rotational transforms are demonstrated to effectively improve quantization performance by suppressing outliers, as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. In this paper, we demonstrate that maximizing cross-dimensional correlation provides a principled criterion for constructing quantization-friendly orthogonal transforms. We propose correlation-aware orthogonal transform (COAT) as a novel and mathematically grounded transform for post-training quantization. COAT synergistically combines a robust, data-independent rotation/Hadamard backbone with an adaptive, data-dependent component, and therefore bridges the gap between data-agnostic structured rotations and learned rotations. This approach proves to be substantially effective for post-training quantization, and numerous experiments on Llama-family models show that COAT is competitive with state-of-the-art rotation methods.