DiscoForcing: A Unified Framework for Real-Time Audio-Driven Character Control with Diffusion Forcing

We study real-time audio-responsive character control as a deployment-faithful problem: strictly causal, bounded-latency streaming that must generate coherent full-body motion at interactive frame rates while the audio condition can change abruptly (tempo shifts, drops, or user edits). Prior music-to-motion systems are largely optimized for offline generation with global context, and degrade in streaming rollouts where conditioning history becomes stale or unreliable. We introduce DiscoForcing, a streaming audio-driven diffusion framework that combines a causal music encoder that captures rhythmic structure and phase dynamics with a diffusion-forcing sequence model trained under heterogeneous noise levels across the temporal horizon. Building on this, we design a hybrid temporal schedule and a history-guided streaming sampler to explicitly trade off responsiveness against long-horizon consistency under non-stationary audio. Implemented in an end-to-end real-time interactive system with online avatar playback and humanoid deployment workflows, DiscoForcing delivers more stable long-horizon rollouts and sharper audio–motion alignment than prior baselines under matched causality and latency constraints while maintaining real-time throughput.Project Page: https://discoforcing.github.io

Imagine an online concert, game, or robot show where a digital character should dance as soon as the music changes. This is hard because most motion-generation systems plan with the whole song in advance, while real interactive systems only hear the music moment by moment. We introduce DiscoForcing, a system that listens to a live audio stream and continuously creates full-body dance motion in real time. It is designed to react quickly to sudden changes such as silence, beat drops, tempo shifts, or user edits, while still keeping the motion smooth over long performances. The same generated motion can drive an animated avatar and can also be converted into commands for a humanoid robot. In experiments, DiscoForcing produced more stable motion and better music–motion timing than prior methods under realistic real-time constraints. This work can support more responsive avatars, virtual performances, and embodied robots.