Trajectory-Aware Spiking DiTs Conversion via Membrane Potential Error-Feedback

Diffusion Transformers (DiTs) have achieved state-of-the-art generative performance, yet their iterative denoising process remains computationally expensive and energy-intensive. Spiking Neural Networks (SNNs) offer a promising neuromorphic alternative for energy efficiency; however, the non-differentiable nature of spiking neurons makes direct training difficult, positioning ANN-to-SNN conversion as a more practical, training-free solution. In this paper, we identify a critical challenge unique to converting DiTs: standard fixed-scale spiking neurons fail to accommodate the highly dynamic activation ranges inherent across denoising steps. This mismatch leads to cumulative errors that significantly degrade generation fidelity. To resolve this, we propose a novel conversion framework featuring Multi-Threshold (MT) neurons and a Membrane Potential Error-Feedback (MPEF) mechanism. MT neurons expand the expressive capacity of discrete spikes by employing a multi-level firing strategy. Concurrently, MPEF exploits the temporal correlation between successive denoising steps to recycle residual membrane potential, effectively compensating for information loss and mitigating distribution shifts without retraining. Extensive experiments on ImageNet demonstrate that our framework achieves competitive generative quality with superior energy efficiency, establishing a new performance benchmark for spiking Diffusion Transformers.

Generating high-quality images with AI often requires the model to refine a noisy picture step by step, much like a painter repeatedly correcting a canvas. While effective, this process consumes a huge amount of energy, limiting practical use. Brain-inspired, energy-efficient “spiking” networks are a promising alternative, but they are difficult to train from scratch. A common shortcut is to convert a well-trained model into a spiking version without retraining, yet we discovered a fundamental problem when applying this to modern image generators: the strength of the internal signals swings across refinement steps, and standard spiking units, which act like simple on/off switches, cannot adapt. This mismatch causes errors to pile up and the final image to degrade. We solved this with a two-part conversion toolkit. First, we redesigned the spiking unit to act like a dimmer rather than a mere switch, allowing it to signal at multiple levels and carry richer information. Second, we introduced a feedback mechanism that carries any leftover signal from one refinement step into the next, much like recycling leftover paint for the next brushstroke. This recovers lost detail without any retraining. In large-scale experiments, our approach matches the image quality of the original power-hungry model while slashing energy consumption. It sets a new bar for high-quality, energy-efficient image generation with spiking technology.