Parameter Decorrelation via Transition-Variance Alignment for Multivariate Time-series Forecasting

Multivariate time-series forecasting (MTSF) learns from high-dimensional covariates with strong temporal dependence, periodic structure, and cross-variable correlations. While modern pipelines often mitigate non-stationarity through instance-wise normalization and decomposition, these interventions operate at the data level and do not directly control dependence that can emerge among the parameters during training. We study MTSF optimization from a parameter-decorrelation viewpoint. Modeling stochastic optimization as a Markov chain in parameter space and leveraging its stochastic differential equation interpretation, we use the per-step transition-variance induced by gradient noise as a tractable signal for optimization-induced dependence and update uncertainty. This signal can empirically inflate during training; we theoretically show that such inflation can degrade generalization diagnostics. Motivated by this mechanism, we propose transition-variance alignment (TVA), an architecture-agnostic procedure that regulates transition-variance by smoothly gating the step size based on the mismatch between an estimated noise scale and a chosen target. TVA maintains effective transition-variance near a prescribed scale without architectural changes, incurs negligible overhead, and integrates seamlessly with diverse methods. Across real-world multivariate benchmarks, TVA consistently improves forecasting accuracy.

Time-series forecasting helps us predict future events such as electricity demand, traffic conditions, and weather patterns. Although modern AI models can make accurate forecasts, training them is challenging because time-series data often contain repeated and closely related patterns. Most previous research has focused on improving forecasting performance by modifying model architectures or applying normalization techniques to the input data. In this work, we take a different perspective and focus on the training process itself. We developed a simple method called Transition-Variance Alignment (TVA), which helps forecasting models learn more reliably from time-series data and reduces the risk of becoming overly influenced by repetitive patterns. Across several real-world forecasting tasks, TVA consistently improved prediction accuracy across a variety of forecasting models while requiring very little extra computation. Our results suggest that helping models train more effectively can be just as important as designing better model architectures or improving the data they learn from.