Learning What Matters First: Sequential Adaptation of Time Series Foundation Models for Robust Financial Forecasting

Foundation models like TimesFM hold strong promise for financial forecasting, yet adapting them to noisy, low-resource domains such as equities remains a challenge. Through fine-tuning on S\&P100 price data, we observe a trade-off between in-domain performance and generalization to unseen stocks. To better understand this behavior, we analyze layer-wise weight dynamics and find that early components contribute more consistently to adaptation, while later layers exhibit noisier and less effective updates. Guided by these findings, we propose a sequential fine-tuning strategy that updates one layer at a time, following an order determined by each layer’s contribution to the overall adaptation trajectory as measured by both direction and magnitude. This targeted approach improves generalization to unseen financial data while reducing the number of trainable parameters at each stage.