Resolving the Timestep Scaling Paradox in Spiking Neural Networks with a Timestep-Scalable Neuron Model

Spiking Neural Networks (SNNs) have garnered attention for their biological plausibility, energy efficiency, and temporal modeling capability. Due to the non-differentiability of spike generation, a widely used training method for SNNs is backpropagation through time with surrogate gradients, achieving competitive performance with few timesteps. Intuitively, scaling timesteps should improve performance by enriching temporal dynamics. However, we observe timestep scaling paradox (TSP), a counter-intuitive accuracy degradation when scaling timesteps. We investigate TSP and link it to long-term temporal gradient vanishing and weakened cross-timestep dependencies. To address this, we propose the Timestep-Scalable (TS) neuron model. It introduces long-term memory reconsolidation to enhance cross-timestep information flow and enable effective learning with more timesteps. In parallel, a temporal forgetting mechanism periodically truncates the accumulation path, suppressing excessive temporal buildup and improving training stability. Supported by theoretical analysis and extensive experiments, TS consistently improves performance when scaling timesteps. In addition, it attains state-of-the-art results on time-series and event-based tasks, while remaining strong on static image classification and object detection.

Spiking neural networks are brain-inspired models that process information as discrete spikes along the temporal dimension. Although using more timesteps should help them capture richer temporal information, we find that conventional spiking networks often suffer performance degradation instead. We call this the timestep scaling paradox. Our study shows that this happens because of long-term temporal gradient vanishing and weakened cross-timestep dependencies. To address this, we design a new spiking neuron model called TS-LIF. It introduces long-term memory reconsolidation to enhance cross-timestep information flow, together with a temporal forgetting mechanism to suppress excessive temporal buildup. This simple change makes spiking networks benefit more reliably from longer time steps. Morevoer, it improves performance on sleep-stage classification, event-based recognition, time-series forecasting, image classification, and object detection.