Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

Vision-Language Models (VLMs) achieve strong multimodal performance but are costly to deploy, and post-training quantization often causes significant accuracy loss. Despite its potential, quantization-aware training for VLMs remains underexplored. We propose GRACE, a framework unifying knowledge distillation and QAT under the Information Bottleneck principle: quantization constrains information capacity while distillation guides what to preserve within this budget. Treating the teacher as a proxy for task-relevant information, we introduce confidence-gated decoupled distillation to filter unreliable supervision, relational centered kernel alignment to transfer visual token structures, and an adaptive controller via Lagrangian relaxation to balance fidelity against capacity constraints. Across extensive benchmarks on LLaVA and Qwen families, our INT4 models consistently outperform FP16 baselines (e.g., LLaVA-1.5-7B: 70.1 vs. 66.8 on SQA; Qwen2-VL-2B: 76.9 vs. 72.6 on MMBench), nearly matching teacher performance. Using real INT4 kernel, we achieve 3× throughput with 54\% memory reduction. This principled framework significantly outperforms existing quantization methods, making GRACE a compelling solution for resource-constrained deployment. Code and data are available at: https://github.com/ForeverBlue816/GRACE.

Modern AI systems that understand both images and text, the technology behind visual assistants and image-aware chatbots, have grown so large that running them demands expensive servers and significant energy. A common way to shrink such models is to store each internal number using fewer bits, but this typically destroys their visual reasoning ability. We developed GRACE, a method that shrinks a large vision-language model while simultaneously letting it learn from its larger "teacher" version. Most prior work treats compression and learning as two separate stages; GRACE combines them under a single principle from information theory: keep only the information needed to solve the task, and discard the rest. A confidence mechanism lets the smaller model rely on the teacher only when the teacher itself is sure, while an adaptive controller automatically balances how aggressively to compress against how much to learn. Remarkably, our 4-bit compressed model often outperforms the original full-precision version, while running faster and using a fraction of the memory. This brings powerful visual AI within reach of phones, smart glasses, robots, and other everyday devices.