LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMs

Post-training quantization (PTQ) is a widely used approach for reducing the memory and compute costs of large language models (LLMs). Recent studies have shown that applying invertible transformations to activations can significantly improve quantization robustness by reducing activation outliers; however, existing approaches are largely restricted to rotation or Hadamard-based transformations. Moreover, most studies focused primarily on traditional quantization schemes, whereas modern hardware increasingly supports the microscaling (MX) data format. Attempts to combine both showed severe performance degradation, leading prior work to introduce assumptions on the transformations. In this work, we take a complementary perspective. First, we provide a theoretical analysis of transformations under MX quantization by deriving a bound on the quantization error. Our analysis emphasizes the importance of accounting for both the activation distribution and the underlying quantization structure. Building on this analysis, we propose LATMiX, a method that generalizes outlier reduction to learnable invertible affine transformations optimized using standard deep learning tools. Experiments show consistent improvements in average accuracy for MX low-bit quantization over strong baselines on a wide range of zero-shot benchmarks, across multiple model sizes.

Large language models have become a foundational component in many applications, including language understanding, code generation, and reasoning. However, their growing size comes with significant memory and computational cost, making efficient deployment increasingly challenging. This paper introduces LATMiX, a method to improve microscaling (MX) quantization for large language models. In very low-bit settings, model accuracy can degrade because unusually large activation values dominate the compression error. This challenge is especially important for MX quantization, where values are grouped into blocks, and each block is scaled separately. LATMiX learns a flexible transformation that redistributes these activation values while accounting for both the activation distribution and the MX block structure. The learned transformation can then be folded into the model weights, so it adds negligible cost during inference. Across several language models and evaluation tasks, LATMiX improves accuracy over strong existing methods. These results suggest that learning transformations tailored to MX quantization can help make large language models more practical to deploy.