Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model

Augmenting Vision-Language-Action models (VLAs) with world models is promising for robotic policy learning but faces challenges in jointly predicting states and actions due to the modality gap. To address this, we propose DUal-STream diffusion (DUST), a world-model augmented VLA framework featuring a multimodal diffusion transformer that maintains separate modality streams while enabling cross-modal knowledge sharing. In addition, DUST utilizes independent noise perturbations and a decoupled flow matching loss to learn cross-modal causal relationships. We further introduce an asynchronous sampling method for action and vision tokens that enhances performance through inference-time scaling. Experimental results on simulated benchmarks like RoboCasa and GR-1 show that DUST achieves up to 6\% gains over state-of-the-art VLA and world-modeling baselines, with inference-time scaling providing an additional 2–5\% improvement. In real-world tasks using the Franka Research 3, DUST outperforms baselines by 10\% in success rate. Finally, we demonstrate that DUST enables effective transfer learning through both pretraining on action-free videos and joint-training with heterogeneous robot and human datasets.

Robots that can follow natural language instructions such as "pick up the mug and place it on the shelf" need to both understand what they see and plan precise movements. Recent AI models combine vision and language understanding with action generation, but they typically act without imagining what will happen next, much like navigating a room with your eyes closed between glances. Adding the ability to predict future scenes (a "world model") helps, but existing approaches struggle because visual predictions and physical movements are fundamentally different kinds of information. Visual information is rich and high-dimensional, while action information is compact and smooth. We propose DUST (Dual-Stream Diffusion), a framework that maintains two separate but communicating processing streams: one for predicting future scenes and one for generating robot actions. By letting each stream operate at its own pace taking more refinement steps for complex visual predictions and fewer for simpler action signals, DUST captures the interplay between seeing and acting without forcing them into a single mold. In experiments on both simulated and real robotic tasks, DUST outperforms existing methods by up to 10%, and can learn from videos of humans performing tasks, even without robot action labels, opening a path to training robots from widely available internet video.