DriveWorld-VLA: Unified Latent-Space World Modeling with Vision–Language–Action for Autonomous Driving
Abstract
End-to-end (E2E) autonomous driving has recently attracted increasing interest in unifying Vision–Language–Action (VLA) with World Models to enhance decision-making and forward-looking imagination. However, existing methods fail to effectively unify future scene evolution and action planning within a single architecture due to inadequate sharing of latent states, limiting the impact of visual imagination on action decisions. To address this limitation, we propose DriveWorld-VLA, a novel framework that unifies world modeling and planning within a latent space by tightly integrating VLA and world models at the representation level, which enables the VLA planner to benefit directly from holistic scene-evolution modeling and reducing reliance on dense annotated supervision. Additionally, DriveWorld-VLA incorporates the latent states of the world model as core decision-making states for the VLA planner, facilitating the planner to assess how candidate actions impact future scene evolution. By conducting world modeling entirely in the latent space, DriveWorld-VLA supports controllable, action-conditioned imagination at the feature level, avoiding expensive pixel-level rollouts. Extensive open-loop and closed-loop evaluations demonstrate the effectiveness of DriveWorld-VLA, which achieves state-of-the-art performance with 91.3 PDMS on NAVSIMv1, 86.8 EPDMS on NAVSIMv2, and 0.16 3-second average collision rate on nuScenes. Code and models are released at https://github.com/liulin815/DriveWorld-VLA.
Lay Summary
Self-driving cars need to “see, understand, and plan ahead” safely, but current systems struggle to link what they perceive with how their actions shape the future. Existing methods either separate scene prediction from decision-making or fail to reason about what will happen if the car takes a certain action. We created DriveWorld‑VLA, a unified AI system that combines visual perception, language understanding, and driving control into one shared thinking space. It uses a world model to imagine future road scenes conditioned on different driving actions, letting the car proactively evaluate risks and choose safer trajectories. Tests on standard driving benchmarks show our approach outperforms state‑of‑the‑art methods in safety and planning accuracy, making self-driving more reliable, forward‑thinking, and ready for real‑world roads.