Diffract: Spectral View of LLM Domain Adaptation

We study continual pre-training (CPT) as a mechanism for adapting general-purpose large language models to specialized domains: mathematics, instruction, code, and natural text. Using singular value decomposition of weight matrices, we find that CPT leaves singular value spectra largely invariant, with adaptation driven mainly by changes in singular vectors. An analysis of attention-head projection matrices reveals strong, domain-dependent head heterogeneity, which we exploit to define a head importance criterion: up to 60% of head updates can be removed without measurable quality loss. Selectively rewinding low-importance heads to their pre-trained state improves benchmark accuracy by up to 4% versus the fully trained baseline. Finally, we identify domain connectivity—linear interpolation between CPT checkpoints yields smooth domain-quality interpolation without notable degradation on either domain—and release Diffract, an open-source toolkit for scalable spectral analysis of billion-parameter models.

Large language models like ChatGPT trained in stages: first, on a large corpus text from web pages; then, in a stage called CPT, on specialized datasets for tasks like solving math, writing code, or following instructions; the process is finished by one or several short fine-tuning stages. The second CPT stage is important for the model quality, but relatively poorly understood. We aim to answer the question - what actually changes inside the model when it learns a new domain? We took apart the billions of numbers inside the model — its weights — using a mathematical tool called singular value decomposition, that splits them into directional components. We found that during specialization the strengths of these components barely change; only their directions rotate. Only a small subset of components does most of the rotating, and they rotate differently depending on whether the model is learning math, code, or general text. By identifying which components carry the new knowledge and which don't, we can revert 60% of the changes without losing accuracy — and even gain up to 4% on math benchmarks by undoing the right ones. We can also smoothly blend two specialized models, say a math expert and a code expert, into a single model that retains both abilities. As large language models grow harder to understand, knowing where in a model new knowledge is stored becomes essential — both for safer deployment and for more efficient training. Our work shows that specialized knowledge concentrates in a small, identifiable set of internal components. We release Diffract, an open-source toolkit, so other researchers can locate and study this structure in their own models.