World Guidance: World Modeling in Condition Space for Action Generation

Leveraging future observation modeling to facilitate action generation presents a promising avenue for enhancing the capabilities of Vision-Language-Action (VLA) models. However, existing approaches struggle to strike a balance between maintaining efficient, predictable future representations and preserving sufficient fine-grained information to guide precise action generation. To address this limitation, we propose WoG (World Guidance), a framework that maps future observations into compact conditions by injecting them into the action inference pipeline. The VLA is then trained to simultaneously predict these compressed conditions alongside future actions, thereby achieving effective world modeling within the condition space for action inference. We demonstrate that modeling and predicting this condition space not only facilitates fine-grained action generation but also exhibits superior generalization capabilities. Moreover, it learns effectively from substantial human manipulation videos. Extensive experiments across both simulation and real-world environments validate that WoG significantly outperforms existing methods based on future prediction. Project page is available at: https://selen-suyue.github.io/WoGNet/.

Robots often struggle to make reliable decisions in complex real-world environments, especially when they need to anticipate how a scene may change in the near future. Existing approaches either predict too much visual information, making them inefficient, or oversimplify the future, reducing control accuracy. We propose WoG (World Guidance), a new method that helps robots focus only on the future information most relevant for taking actions. Instead of predicting full future images or videos, WoG learns compact future guidance signals that improve decision-making while remaining efficient and robust. Experiments in both simulation and real-world robot tasks show that WoG enables more accurate and adaptable manipulation, including under changes in lighting, backgrounds, and objects. Our method can also learn effectively from large collections of human demonstration videos, suggesting a scalable path toward more capable and generalizable robotic systems.