Rotation-Preserving Supervised Fine-Tuning

Supervised fine-tuning (SFT) improves target-task performance but can rotate pretrained representations in ways that reduce out-of-domain generalization. We propose Rotation-Preserving Supervised Fine-Tuning (RPSFT), a simple regularizer that penalizes drift in the projected top-$k$ singular-vector block of pretrained weight matrices. Across Llama and Qwen models trained on OpenR1-Math, RPSFT improves the in-domain/OOD trade-off over SFT, importance-weighted SFT, and Dynamic Fine-Tuning, and gives strong initializations for downstream RL fine-tuning. The results suggest that controlling dominant-subspace rotation is a practical way to retain pretrained structure while still allowing task adaptation.