Overcoming the Modality Gap in Context-Aided Forecasting

Context-aided forecasting (CAF) holds promise for integrating domain knowledge and forward-looking information, enabling AI systems to surpass traditional statistical methods. However, recent empirical studies reveal a puzzling gap: multimodal models often fail to outperform their unimodal counterparts. We hypothesize that this underperformance stems partly from insufficiently verified context usefulness in existing datasets. To address these limitations, we introduce a semi-synthetic data augmentation method that generates contexts both descriptive of temporal dynamics and verifiably complementary to numerical histories. This approach enables massive-scale dataset creation, resulting in CAF-7M, a corpus of 7 million context-augmented time series windows, including a rigorously verified test set. We demonstrate that semi-synthetic pre-training transfers effectively to real-world evaluation, and show clear evidence of context utilization. Our results suggest that dataset quality is a major bottleneck in context-aided forecasting, and that verified context can substantially improve the usefulness of CAF training data.

Forecasting tools are used to predict future demand, prices, energy use, traffic, and many other quantities. In real life, people often also know useful background information, such as a holiday, a supply disruption, or a planned event, but today’s AI forecasting systems do not reliably use this kind of written information. Our work studies how to create better examples for teaching and evaluating AI systems on this problem. We build millions of examples by pairing real sequences of numbers over time with short, realistic written descriptions, and we carefully check a test set where the description actually helps improve the forecast. We then train a forecasting model that can read both the past numbers and the written description. Our results show that better data can help AI systems learn when written context is useful, while also showing that generated descriptions are still an imperfect substitute for real-world reports.