STEP: Warm-Started Visuomotor Policies with Spatiotemporal Consistency Prediction

Diffusion policies have recently emerged as a powerful paradigm for visuomotor control in robotic manipulation due to their ability to model the distribution of action sequences and capture multimodality. However, iterative denoising leads to substantial inference latency, limiting control frequency in real-time closed-loop systems. Existing acceleration methods either reduce sampling steps, bypass diffusion through direct prediction, or reuse past actions, but often struggle to jointly preserve action quality and achieve consistently low latency. In this work, we propose STEP, a lightweight spatiotemporal consistency prediction mechanism to construct high-quality warm-start actions that are both distributionally close to the target action and temporally consistent, without compromising the generative capability of the original diffusion policy. Then, we propose a velocity-aware perturbation injection mechanism that adaptively modulates actuation excitation based on temporal action variation to prevent execution stall especially for real-world tasks. We further provide a theoretical analysis showing that the proposed prediction induces a locally contractive mapping, ensuring convergence of action errors during diffusion refinement. Extensive evaluations on nine simulated benchmarks and two real-world tasks show that STEP with 2 steps can achieve an average 21.6\% and 27.5\% higher success rate than BRIDGER and DDIM on the RoboMimic benchmark and real-world tasks, respectively. The code is available at https://github.com/Kimho666/STEP.

Robots capable of visuomotor control commonly rely on diffusion policies to determine action sequences. This approach excels at modeling potential action possibilities and adapting to diverse scenarios but suffers from significant latency, hindering real-time closed-loop control. Existing acceleration techniques either reduce sampling steps, bypass diffusion, or reuse prior actions, yet often fail to balance action quality and low latency. In this work, we develop a lightweight mechanism to generate high-quality, temporally consistent initial actions without sacrificing the original diffusion policy’s generative advantages, alongside a velocity-aware feature that modulates actuation to prevent execution stagnation. We provide theoretical validation that our method ensures diminishing action errors during diffusion refinement. Evaluations across nine simulated benchmarks and two real-world tasks show our 2-step approach achieves 21.6% and 27.5% higher success rates than BRIDGER and DDIM, outperforming existing methods in balancing latency and performance.