LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs). Current QAT methods are mainly based on scalar quantization (SQ), which enables efficient optimization but suffers from severe performance degradation at 2-bit precision. On the other hand, vector quantization (VQ) provides substantially higher representational capacity, but its discrete codebook lookup prevents end-to-end training. We propose LC-QAT, a 2-bit weight-only VQ-QAT framework that represents quantized weights via a learned affine mapping over discrete vectors, which yields a high-quality PTQ initialization and enables fully differentiable end-to-end optimization without explicit codebook lookup in the training forward pass. This strong post-training initialization makes LC-QAT highly data-efficient. Experiments across diverse LLMs demonstrate that LC-QAT consistently outperforms state-of-the-art QAT methods while using only 0.1%–10% of the training data. Our results establish LC-QAT as a practical and scalable solution for extreme low-bit model deployment.

Large language models (LLMs) like those powering modern AI assistants are powerful but memory-hungry, making them hard to run on everyday devices. Compressing their numerical weights to just 2 bits — a technique called quantization — can shrink a model to around 15% of its original size, but it typically destroys too much information to remain useful. The best existing approach for recovering accuracy after compression, called quantization-aware training (QAT), requires retraining the model on tens of billions of text tokens, which demands enormous computational resources and time. We introduce LC-QAT, a new training framework that sidesteps this bottleneck by changing how the compressed weights are represented. Instead of independently compressing each weight value (scalar quantization), LC-QAT groups weights together and expresses them through a simple linear formula. This design lets the model start training from a much higher-quality compressed state, and allows all parameters to be updated simultaneously using standard gradient-based optimization, which could not be achieved by prior vector-quantization methods. Because LC-QAT begins from a better starting point, it recovers near-full-precision accuracy using as little as 0.1%–10% of the training data required by competing methods — a reduction of 10× to 1000×. This makes high-quality 2-bit LLM deployment dramatically more accessible, lowering the cost and energy needed to serve capable AI models on resource-constrained hardware.