ParalESN: Enabling parallel information processing in Reservoir Computing

Reservoir Computing (RC) has established itself as an efficient paradigm for temporal processing. However, its scalability remains severely constrained by the need to process temporal data sequentially and the prohibitive memory footprint of high-dimensional reservoirs. To address these limitations, we revisit RC through the lens of structured operators and state space modeling, introducing Parallel Echo State Network (ParalESN). Leveraging diagonal linear recurrence in the complex domain, ParalESN enables parallel processing of temporal data and the construction of efficient, high-dimensional reservoirs. A thorough theoretical analysis demonstrates that the Echo State Property and the universality guarantees of traditional Echo State Networks are preserved, while also admitting an equivalent representation of arbitrary linear reservoirs in the complex diagonal form. Empirically, ParalESN achieves competitive predictive accuracy with traditional RC and with fully trainable sequence models, while delivering computational savings by orders of magnitude. Overall, ParalESN offers a scalable and principled pathway for integrating RC within the deep learning landscape.

Reservoir Computing (RC) is a popular framework for designing neural networks that process sequential data, such as time series or text, while remaining efficient in terms of energy consumption and CO2 emissions. To keep costs low, most of the network is randomly initialized and never trained, relying solely on untrained dynamics. However, traditional RC suffers from two critical bottlenecks. First, data must be processed one step at a time, much like reading a book word by word, which is inherently slow. Second, scaling these systems to the large sizes, often needed for complex tasks, is impractical due to prohibitive memory requirements. We introduce Parallel Echo State Networks (ParalESN), a redesigned RC approach that uses memory efficient matrix parameterizations to overcome both limitations at once: entire sequences can be processed in parallel, and memory usage is drastically reduced. Theoretically, we prove that ParalESN is just as powerful as traditional approaches. Empirically, we demonstrated that ParalESN trains faster, consumes less energy, and produces lower CO2 emissions, while also being more accurate in terms of predictions. Our work paves the way for more sustainable and resource-efficient artificial intelligence, reducing energy consumption while maintaining strong performance.