FRACTAL: State Space Model with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences

Effective sequence modeling fundamentally requires balancing the retention of unbounded history with the high-resolution detection of abrupt short-term variations common in real-world phenomena. However, existing state space models (SSMs) relying on high-order polynomial projection operators (HiPPO) face a critical trade-off where uniform measures dilute recent information to maintain timescale invariance, while exponential measures sacrifice global context to capture local dynamics. This paper proposes a Fractional Recurrent Architecture for Computational Temporal Analysis of Long sequences (FRACTAL), a novel architecture integrating fractional measure theory into recursive memory updates to address this limitation. By deriving projection operators with analytically characterized spectral properties and a tunable singularity index, the proposed method amplifies sensitivity to recent signal perturbations while preserving the spectral structure that encodes scale-invariant memory dynamics. This theoretical innovation is instantiated within a simplified diagonalized state space framework by modulating input projection initialization to enable simultaneous capture of multi-scale temporal features. FRACTAL achieves an average score of 87.11\% on the Long Range Arena benchmark, including 61.85\% on the ListOps task, outperforming the S5 model.

When AI models process sequences of data such as a patient's vital signs, financial markets, or natural language, they need to do two things at once: remember patterns from the distant past, and react quickly to sudden recent changes. Current models are forced to choose one or the other. Those that remember long history tend to blur recent events; those that stay alert to recent changes tend to forget the bigger picture. We tackled this by borrowing a tool from mathematics called fractional calculus, which lets us design a memory function that does both simultaneously. Think of it like a camera with a tunable lens: one setting captures a wide panoramic view of history, and another zooms in sharply on what just happened. Our method lets the model use both lenses at the same time, with a single dial that smoothly blends between them. We tested our model, FRACTAL, on a standard benchmark designed to challenge AI with very long sequences of data. It outperformed previous models, especially on tasks that require tracking both global structure and fine-grained local detail at the same time. We hope this opens new doors for AI applications in medicine, climate science, and finance, where history and the present are equally important.