LoRA-Lens: Training Induces Spectral Compression in Low-Rank Adapters

Low-Rank Adaptation (LoRA) is widely used for parameter-efficient fine-tuning, but how training reshapes adapter spectral structure—and what this implies for rank allocation—remains poorly understood. We introduce LoRA-Lens, a diagnostic that tracks capacity utilization (CU), the fraction of rank budget carrying meaningful spectral energy. Across five models from three families, training compresses adapter spectra (CU: 1.0 → 0.43–0.83), with V-projection compressing 2–3× more than Q. This compression does not reliably indicate redundancy: on three representative models, V lora_B parameters are approximately 30–350× more Fisher-sensitive than Q across training stages, consistent with the OV circuit's role as the content channel that writes directly to the residual stream. Symmetric rank sweeps show that Q-rank reduction is safe in the tested models while V-rank reduction can be harmful. We propose FisherLoRA— preserve V, reduce Q—reducing Q/V adapter parameters by 38% relative to uniform rank allocation with worst-case degradation of 0.006 across five models, vs. 0.067 for spectral-guided allocation.