Mantis: Lightweight Foundation Model for Time Series Classification

While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce \textbf{Mantis}, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.

While large AI models have revolutionized text and vision, their application to sequential data, such as tracking a patient's heart rate or stock market trends over time, is a frontier that has only recently started to grow. Most existing models focus on predicting future trends rather than categorizing past data, making them ill-suited for critical tasks like identifying diseases or recognizing human activities. To bridge this gap, we introduce Mantis, a lightweight AI model designed specifically to categorize sequential data. A key innovation of Mantis is that it learns entirely from computer-generated, synthetic data using a technique that teaches it to distinguish distinct patterns. We also developed a novel way to break raw data into manageable pieces and an advanced method to combine insights from the model's internal layers to maximize its accuracy during testing. In extensive experiments, Mantis achieved state-of-the-art accuracy across diverse real-world fields, including neuroscience and human activity recognition. Crucially, it succeeds "out of the box" without needing expensive, real-world data to learn each new task, making powerful AI diagnostics more accessible across various industries.