A$^2$SG: Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks

Training deep spiking neural networks (SNNs) remains challenging due to sharp loss landscapes and temporal inconsistency caused by surrogate gradients. To address these challenges, we propose a unified framework: adaptive and asymmetric surrogate gradients (A$^2$SG). The adaptive gradients adjust an effective window for spatio-temporal adaptation, reducing spatial gradient variation and maintaining directional consistency of gradients over time. The asymmetric gradients reflect neuronal dynamics by assigning larger gradients to neurons with higher membrane potentials, and we prove that they yield lower variation than symmetric surrogates. Our analysis further establishes a direct connection between local gradient variation and the curvature of the loss landscape, providing a principled explanation for how A$^2$SG promotes convergence to flatter minima and improves generalization. We conduct extensive experiments on diverse models, including CNN-based and Transformer-based SNNs, across various tasks such as image classification using both static and neuromorphic datasets, as well as segmentation. The results demonstrate that A$^2$SG consistently improves accuracy and energy efficiency, establishing it as a general and reliable solution for training deep SNNs.Our code is available at https://github.com/KIST-NCL/A2SG.git.

Spiking neural networks are a brain-inspired form of AI that communicate through brief electrical pulses, making them remarkably energy-efficient. But these abrupt pulses break the DNN-like training method, so researchers rely on an approximate guide called a "surrogate gradient." Existing guides tend to steer networks into rugged terrain that hurts performance, and they give inconsistent signals over time. We introduce A²SG, which fixes this in two ways: it continuously tunes the guiding signal to keep it stable across time, and it gives each neuron guidance in proportion to its membrane potential. Together, these steer the network toward smoother regions that generalize better. Across many models and tasks — including image recognition and segmentation — A²SG consistently improved both accuracy and energy efficiency, offering a general and reliable way to train powerful, low-power AI.