STLA: Spatiotemporal Lookahead Alignment for Post-Training Quantization

Adaptive rounding techniques in Post-Training Quantization (PTQ) enable the efficient deployment of Large Language Models (LLMs) with low resource and data dependencies. While learning-based rounding methods are accurate yet costly, compensation-based approaches offer a highly efficient alternative. However, synergizing these two to realize their full potential is hindered by spatiotemporal misalignment in the decoupled paradigm. Key challenges include temporal parameter conflict, the invalidation of the initial Round-to-Nearest (RTN) assumption, and spatially-inconsistent optimization objectives. This paper introduces STLA, a novel rounding-optimized PTQ framework that achieves both fast and accurate LLM quantization. STLA resolves temporal inconsistency through cluster-wise integrated rounding optimization, which collocates the learning and compensation phases. STLA achieves spatial alignment through a unified global objective derived from the Schur Complement, enabling the solver to look ahead and align local rounding decisions with the optimal future compensation of remaining weights. Furthermore, we propose a Hessian-guided clustering strategy that exploits both diagonal and off-diagonal information to maximize intra-cluster error cancellation. Extensive experiments demonstrate that STLA establishes a new state-of-the-art for low-bit PTQ while maintaining high computational efficiency. The code is available at https://github.com/AI2C-Lab/STLA.

Large language models are powerful, but their enormous size makes them difficult to run on everyday devices, where memory and computing resources are limited. Post-training quantization, or PTQ, is a popular way to make these models smaller after they have already been trained. It works by representing the model’s numbers with lower precision, which can greatly reduce memory usage and improve deployment efficiency. However, this compression often reduces accuracy. Existing PTQ methods tend to prioritize either speed or accuracy, and they struggle to achieve both because of spatial and temporal conflicts in how they simplify and adjust the model. In this work, we introduce STLA, a new method that better coordinates the compression process. Instead of treating key steps separately, STLA aligns them in both space and time: it groups related parts of the model, learns how to simplify them effectively, and looks ahead to anticipate how each change will affect the rest of the model. This spatiotemporally aligned strategy helps prevent the errors that usually accumulate during compression. Our approach makes it possible to shrink large language models efficiently while preserving their performance. This can help bring advanced AI systems to resource-limited settings, such as mobile and edge devices, making AI deployment faster, cheaper, and more accessible.