Byte Pair Encoding for Efficient Time Series Forecasting

Existing time series tokenization methods predominantly encode a constant number of samples into individual tokens. This inflexible approach can generate excessive tokens for even simple patterns like extended constant values, resulting in substantial computational overhead. Inspired by the success of byte pair encoding, we propose the first pattern-centric tokenization scheme for time series analysis. Based on a discrete vocabulary of frequent motifs, our method merges samples with underlying patterns into tokens, compressing time series adaptively. Exploiting our finite set of motifs and the continuous properties of time series, we further introduce conditional decoding as a lightweight yet powerful post-hoc optimization method, which requires no gradient computation and adds no computational overhead. On recent time series foundation models, our motif-based tokenization improves forecasting performance by 40% and boosts efficiency by 2314% on average. Conditional decoding further reduces MSE by up to 48%. In an extensive analysis, we demonstrate the adaptiveness of our tokenization to diverse temporal patterns, its generalization to unseen data, and its meaningful token representations capturing distinct time series properties, including statistical moments and trends.

Many current methods for analyzing time series data break it into fixed-size chunks, even when the data contain simple or repetitive patterns. This can lead to unnecessary complexity and high computational cost. In this work, we introduce a new way of representing time series that instead focuses on recurring patterns. Inspired by text compression techniques, our method groups similar patterns into compact tokens from a learned set of motifs, allowing the representation to adapt to the structure of the data. We also propose a simple post-processing step, called conditional decoding, that further improves predictions without adding computational overhead or requiring additional training. Experiments on modern time series foundation models show that our approach substantially improves both accuracy and efficiency. Additionally, we demonstrate that the learned tokens capture meaningful properties of time series, such as trends and distributional characteristics, and generalize well to new, unseen data.