Are Time Series Foundation Models Ready for Zero-Shot Forecasting?

Recent successes of foundation models in various domains have spurred interest in time series foundation models (TSFMs), especially for zero-shot forecasting. We challenge the necessity of zero-shot forecasting, and demonstrate that a simpler model, PCA+Linear, can effectively serve as a TSFM. PCA+Linear uses Principal Component Analysis (PCA) as a universal feature extractor and a linear head trained specifically on each downstream dataset. Experiments show PCA+Linear achieves results competitive with state-of-the-art TSFMs. We further evaluate the robustness of transformer-encoder-based TSFMs on out-of-distribution data, highlighting the importance of the final linear layer in addition to the attention mechanism. Our findings also emphasize the effectiveness of diverse pretraining data over extensive datasets from limited sources.