Biologically plausible heavy-tailed connectivity enhances generalizations on cognitive tasks in recurrent neural networks

While heavy-tailed synaptic weight distributions are pervasive in biological neural networks, their computational role---particularly in relation to generalization---remains poorly understood. To address this, we develop a novel optimal-transport-based optimization algorithm that incorporates key biological constraints, including Dale’s principle and heavy-tailed synaptic statistics, to train recurrent neural networks (RNNs) on a wide range of cognitive tasks. We show that these biologically constrained, heavy-tailed RNNs exhibit substantially improved generalization, which we further characterize within a PAC-Bayes framework. Our theoretical analysis and numerical experiments reveal two complementary mechanisms underlying this generalization enhancement. Topologically, heavy-tailed connectivity induces an effectively low-rank structure, which in turn yields low-dimensional neural dynamics. Geometrically, heavy-tailed connectivity intrinsically shapes task variable representations to lie near a linear manifold, thereby improving generalization for a linear readout strategy. Together, these results identify heavy-tailed connectivity as a biologically grounded intrinsic mechanism that promotes low-rank structure and favorable representational geometry, leading to improved generalization in flexible cognitive tasks.

Although brains naturally feature heavy-tailed network connection, which means a rare few neurons share exceptionally strong links while most are weakly connected, the computational benefit of this structure remains mystery. To investigate this, we designed a training method to build recurrent neural networks (RNNs) that mimic these brain-like wiring statistics. We discovered that these brain-inspired networks generalize significantly better when solving diverse cognitive tasks. Our analysis reveals two reasons for this: first, topologically, the heavy-tailed connectivity leads to an effectively low-rank network structure, which in turn yields low-dimensional neural dynamics; second, geometrically, it shapes how information is stored, forcing task data to align along nearly straight paths. This cooperative structure allows a simple linear readout mechanism to easily decode and predict new scenarios, proving that the brain's unique connection patterns serve as a fundamental blueprint for efficient learning and generalization.