Block Rotation is All You Need for MXFP4 Quantization

Large language models (LLMs) have achieved remarkable success, but their rapidly growing scale imposes prohibitive costs in memory, computation, and energy. Post-training quantization (PTQ) is a promising solution for efficient deployment, yet achieving accurate W4A4 quantization remains an open challenge. While most existing methods are designed for INT4 formats, the emergence of MXFP4—a new FP4 format with various hardware support (NVIDIA, AMD, Intel)—raises questions about the applicability of current techniques. In this work, we present a unified empirical comparison of representative PTQ methods under the MXFP4 format. Through systematic evaluation, we find that methods like GPTQ consistently deliver strong performance, whereas rotation-based approaches, which are widely used in state-of-the-art approaches, suffer from severe incompatibility with MXFP4. We further provide the first in-depth analysis of this conflict, tracing its root to a fundamental mismatch between MXFP4’s PoT (power-of-two) block scaling and the redistribution of outlier energy via global rotation. Building on this insight, we propose a simple yet effective block rotation strategy that adapts rotation-based methods to MXFP4, leading to substantial accuracy improvements across diverse LLMs. Our findings not only offer clear guidance for practitioners but also set a foundation for advancing PTQ research under emerging low-precision formats.

Large language models are powerful but costly to run, requiring substantial memory, computation, and energy. A promising way to reduce these costs is to use very low-bit number formats such as MXFP4, which are increasingly supported by modern AI hardware. This paper studies how existing compression methods work under MXFP4. We find that a common technique based on global rotation can unexpectedly hurt performance because it spreads large values across many small groups, making them harder to compress accurately. To address this, we propose block-wise rotation, which keeps the transformation within the same groups used by MXFP4. This simple change improves accuracy across multiple language models while reducing extra computation. Our work provides practical guidance for deploying large language models more efficiently on low-bit hardware.