SHARP-Q: Spectral Hessian Alignment and Rectification for Post-training Quantization

Post-training quantization (PTQ) suffers from severe accuracy degradation in ultra-low-bit regimes. To address this challenge, we propose SHARP-Q, a unified framework grounded in Information Geometry that aligns the quantization objective with the intrinsic Fisher geometry. Following a "Rectify-then-Approximate" strategy, SHARP-Q first preconditions the optimization landscape via Hessian-Aware Rectification (HAR) and subsequently approximates the rectified Fisher Information Matrix through Dynamic Fisher-Subspace Compensation (DFSC). Our empirical evaluations reveal a pivotal insight: precise geometric alignment enables hardware-friendly uniform quantizers to outperform specialized non-uniform designs. Extensive experiments across representative convolutional networks, Vision Transformers, and State Space Models confirm that SHARP-Q establishes new state-of-the-art results, achieving substantial accuracy gains in the challenging W2A2 and W3A3 settings.

Artificial Intelligence (AI) models are incredibly powerful, but their massive size makes it difficult to run them efficiently on everyday devices. To solve this, researchers compress them. However, shrinking an AI model to an extreme degree usually causes a severe drop in accuracy. This happens because traditional methods blindly squeeze the model, which distorts its complex internal mathematical structure. We introduce SHARP-Q, a new method that solves this problem using a "Rectify-then-Approximate" strategy. Instead of directly forcing the model into a smaller size, SHARP-Q first "rectifies"—or smooths out—the model's internal mathematical landscape to make it naturally compression-friendly. Then, it carefully "approximates," or shrinks, the model while protecting its most sensitive parts. Extensive tests show that SHARP-Q sets a new record. It allows a wide variety of modern AI models to maintain extremely high accuracy even under extreme compression, making it much easier to deploy powerful AI in the real world.