Baguan-TS: dual in-context learning model for time series forecasting with covariates

Transformers enable in-context learning (ICL) for rapid, gradient-free adaptation in time series forecasting, yet most ICL-style approaches rely on tabularized, hand-crafted features, while end-to-end sequence models lack inference-time adaptation. We bridge this gap with a unified framework, Baguan-TS, which integrates the raw-sequence representation learning with ICL, instantiated by a 3D Transformer that attends jointly over temporal, variable, and context axes. To make this high-capacity model practical, we tackle two key hurdles: (i) calibration and training stability, improved with a feature-agnostic, target-space retrieval-based local calibration; and (ii) output oversmoothing, mitigated via context-overfitting strategy. On public benchmark with covariates, Baguan-TS consistently outperforms established baselines, achieving the highest win rate and significant reductions in both point and probabilistic forecasting metrics. Further evaluations across diverse real-world energy datasets demonstrate its robustness, yielding substantial improvements.

Many real-world decisions rely on forecasting data that changes over time, such as electricity demand, solar power generation, store sales, traffic, or air quality. These forecasts often depend not only on past values, but also on information known in advance, such as weather forecasts, holidays, and calendar patterns. We introduce Baguan-TS, an AI forecasting model that can learn from examples provided at prediction time. In other words, instead of needing to be retrained for every new forecasting situation, the model is given a small set of relevant past examples and uses them immediately to adapt its prediction. This ability, often called “in-context learning,” is similar to how a person might solve a new problem after seeing a few similar examples. Unlike many previous approaches that first convert time series into manually designed table features, Baguan-TS works directly with the original time-based data. It learns patterns across time, across different input variables, and across the examples provided as context. This allows it to use both historical trends and known future information more effectively. We also address two practical challenges. First, the model checks similar past patterns to better adjust its forecasts for the current situation. Second, it is trained to avoid making overly smooth predictions that miss important spikes, such as sudden demand surges. Across public benchmarks and real-world energy datasets, Baguan-TS produced more accurate forecasts than strong existing methods, showing that in-context learning can be an effective way to build adaptable time series forecasting models.