Time Series Forecasting Through the Lens of Dynamics

While deep learning is facing an homogenization across modalities led by Transformers, they are still challenged by shallow linear models in the time series forecasting task. Our hypothesis is that models should learn a direct link from past to future data points, which we identify as a learning dynamics capability. We develop an original $\texttt{PRO-DYN}$ nomenclature to analyze existing models through the lens of dynamics. Two observations thus emerge: **1.** under-performing architectures learn dynamics at most partially, **2.** the location of the dynamics block at the model end is of prime importance. Our systemic and empirical studies both confirm our observations on a set of performance-varying models with diverse backbones. We propose a simple plug-and-play methodology guiding model designs and improvements.

Predicting the future may sound like science fiction, but it is already part of our everyday lives. Weather forecasts help us plan our week, traffic forecasts help us choose a route, and energy forecasts help manage electricity demand. In machine learning, this task is called time series forecasting: using past observations to predict future values. In recent years, researchers have developed increasingly complex models for this task, including models inspired by language processing. Surprisingly, very simple models often perform as well as, or even better than, these sophisticated approaches. This work investigates why. We argue that strong forecasting models are not simply better at spotting patterns in the data. Instead, their design gives them a better way to turn past observations into future predictions. We introduce a new way to analyze this past-to-future step across forecasting models. Our study shows that weaker models often handle this step only partially or indirectly, while stronger models make it more direct and easier to learn. We also show that improving this part of a model can improve its predictions. This work helps explain what makes forecasting models effective and guides the design of better ones.