WUSH: Near-Optimal Adaptive Transforms for LLM Quantization

Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors. Prior transform-based mitigations (e.g., Hadamard rotations) are fixed and data-agnostic, and their optimality for quantization has remained unclear. We derive closed-form optimal linear blockwise transforms for joint weight-activation quantization under standard RTN AbsMax-scaled block quantizers, covering both integer and floating-point formats. The resulting construction, WUSH, combines a Hadamard backbone with a data-dependent second-moment component to form a non-orthogonal transform that is provably near-optimal for FP and INT quantizers under mild assumptions while admitting an efficient fused GPU implementation. Empirically, WUSH improves W4A4 accuracy over the strongest Hadamard-based baselines (e.g., on Llama-3.1-8B-Instruct in MXFP4, it gains +2.8 average points with RTN and +0.7 with GPTQ) while delivering up to 5.8$\times$ per-layer throughput over BF16 via FP4 MatMul. Source code is available at https://github.com/IST-DASLab/WUSH.

Modern AI language models are enormous. To run them affordably, we "quantize" them: storing their internal numbers at very low precision, like rounding every price to the nearest dollar. But a few extreme values, called outliers, force everything else onto a crude scale and wreck accuracy. The usual fix is to first apply a mathematical transform that reshuffles the numbers so outliers do less harm. One fixed recipe, the Hadamard transform, works surprisingly well, yet no one knew why, or whether anything better existed. Oddly, it ignores the very numbers being compressed. We derived an exact formula for the best possible transform, one that adapts to each model's own data. Our method, WUSH, is the Hadamard transform plus a data-aware ingredient. That solves the mystery: Hadamard is just the data-blind part of the true optimum. The payoff: WUSH beats today's best methods on accuracy, runs just as fast, and lets cheap, hardware-friendly number formats hit accuracy that once needed costlier ones.