The Geometry of Narrow Fine-Tuning Degradation: Trajectory Lock-in and Spectral Bifurcation

Magnitude-based stability proxies such as parameter drift are widely used in narrow-task fine-tuning, yet they do not reliably indicate degradation of broad capabilities. We identify trajectory lock-in: under fixed training conditions for narrow adaptation, the joint evolution of task loss and broad generalization collapses onto a shared low-dimensional degradation curve, so many stabilizers primarily change the rate of progress along this curve rather than altering the curve itself. This yields a drift paradox, in which comparable Euclidean displacement can still correspond to divergent generalization outcomes. To diagnose the underlying structure, we introduce objective-agnostic geometric probes that track the effective update subspace, together with an online harm signal that reflects curvature-dominated channeling toward directions associated with broad degradation. Finally, we show that escaping lock-in requires a spectral bifurcation, namely a qualitative reorientation of the update subspace toward softer curvature modes, thereby improving broad generalization while maintaining matched training performance. We validate these findings across model scales and modalities in narrow-task settings, and report practical deployment procedures and overhead measurements.

Large language models are often adapted to specialized tasks, but this adaptation can unintentionally weaken their broader abilities. This paper shows that such degradation is not well explained by how far the model parameters move. Instead, the direction and structure of the update trajectory are also important. We introduce simple diagnostic signals that monitor how the model changes during fine-tuning and help identify when training begins to harm broad capabilities. We also study a spectral intervention that can sometimes redirect updates, reducing broad degradation while preserving task learning. These findings may help make model adaptation more reliable, easier to monitor, and safer to deploy.