RoboOmni: Actions Are Just Another Modality for Vision-Language Models

Integrating Vision-Language Models (VLMs) into robotics has facilitated the development of generalizable Vision-Language Action (VLA) policies. However, unified discrete frameworks lag behind decoupled continuous designs due to limitations in action chunking and temporal modeling. To address this, we introduce **RoboOmni**, a unified multi-modal next-token prediction framework. Challenging the assumption that continuous modeling is essential for high-performance manipulation, **RoboOmni** demonstrates that *actions are just another modality* capable of being effectively modeled discretely. At the core of our method is Multi-Token Action Prediction (MTAP), which integrates action chunking directly into the discrete tokenizer. This design resolves temporal modeling bottlenecks and significantly reduces distribution shift between training and inference. By preserving the native VLM training and inference pipeline, **RoboOmni** naturally benefits from large-scale multimodal co-training and modern decoding optimizations. Extensive evaluations on the CALVIN, SimplerEnv, and real-world platforms confirm that **RoboOmni** establishes new state-of-the-art performance, significantly outperforming diffusion-based baselines such as $\pi_0$. Notably, combining our proposed MTAP with the FAST tokenizer achieves a 94.4\% average success rate on CALVIN, while the Bin tokenizer implementation attains a 27$\times$ inference speedup compared to OpenVLA.

Modern AI can read text and understand images remarkably well, yet making a robot act in the physical world — pick up a cup, fold a shirt, plug in a cable — is still very hard. A common recipe today is to use a large vision-and-language model as the robot's "eyes and brain," then attach a separate motor module that turns understanding into movements. This split forces perception and action to live in two different worlds, limiting how well a robot generalizes to new tasks. We propose RoboOmni, a single model that treats physical actions the way it treats words and pixels — as another stream of tokens it can directly generate. Instead of producing one tiny motion at a time, our method predicts a short sequence of motions in one shot, like a writer drafts a phrase rather than one letter. Because the model never leaves its native pipeline, it inherits the speed and scaling tricks behind today's large language models. On standard robot manipulation benchmarks, RoboOmni sets new state-of-the-art success rates and runs about 27× faster than prior unified approaches. By closing this perception-action gap inside one model, we bring general-purpose, easy-to-deploy robot assistants meaningfully closer to reality.