Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, challenging to deploy, especially in memory-constrained settings such as in edge and on-device deployment. This paper describes a simple parameter-efficient architecture for time-series foundation modeling. Our architecture consists of three components: a multi-scale input strategy where we create multiple versions of the original input via downsampling at different scales; hybrid sequence-mixing layers consisting of long convolutions and DeltaNet layers; and an attention-based decoder head. We combine our architecture with standard data augmentation recipes to train a family of parameter-efficient time series foundation models, dubbed Reverso. Reverso achieves competitive performance across zero-shot forecasting, classification and anomaly detection, pushing the performance-efficiency Pareto frontier.