Proteus: Lookup-Free Trellis-Coded Quantization by Lattice-Breaking Compute Codes for 2-Bit LLMs

Autoregressive decoding of large language models is frequently memory-traffic bound, so ultra-low-bit weight-only PTQ helps only if dequantization avoids irregular codebook or LUT access in the inner loop. Under the GPU-friendly bitshift trellis, existing 2-bit trellis-coded quantization (TCQ) pipelines either reintroduce micro-LUTs or suffer overlap-amplified artifacts because incoherence improves global Gaussianity but does not guarantee overlap-local joint geometry. We introduce Proteus a strictly lookup-free TCQ framework whose computed generator MUL-BAL uses cheap integer mixing plus a per-layer affine Gaussianizer to produce overlap-robust, near-Gaussian code values with zero runtime table loads. Proteus instantiates each layer by selecting from a tiny, pre-vetted candidate pool and then applies lightweight channel compensation and optional few-shot distillation that tune only per-layer affine statistics while keeping packed indices and the bitshift-trellis decoder fixed. On Llama 2 (7B–70B) at 2-bit PTQ, Proteus improves perplexity and zero-shot accuracy over strong TCQ/PTQ baselines and reduces end-to-end decode bandwidth at comparable throughput (e.g., 740 vs. 1020 GB/s on 70B).

Modern AI language models are built from billions of internal numbers, and what slows them down when generating text is often not raw calculation speed but the time spent shuttling those numbers between a chip's memory and its processor. A common way to ease this is to compress the numbers so each takes far less space — in our case, about two bits each instead of the usual sixteen. But a compressed number must be unpacked before it can be used. A popular shortcut keeps the possible values in a small reference table that the chip looks up over and over — and when many requests run at once, these repeated lookups clog its fastest memory and eat into the savings. We developed Proteus, a method that removes the table entirely: instead of looking each number up, it rebuilds the number on the spot from a handful of cheap arithmetic operations. The heart of Proteus is a compact "recipe" we designed so that the rebuilt numbers come out varied and evenly spread, avoiding the repetitive patterns that otherwise degrade quality when numbers are squeezed this small. The result is that heavily compressed models become both more accurate and lighter on memory than today's leading methods, making large AI models cheaper and more practical to run.