DexScale: Automating Data Scaling for Sim2Real Generalizable Robot Control

A critical prerequisite for achieving generalizable robot control is the availability of a large-scale robot training dataset. Due to the expense of collecting realistic robotic data, recent studies explored simulating and recording robot skills in virtual environments. While simulated data can be generated at higher speeds, lower costs, and larger scales, the applicability of such simulated data remains questionable due to the gap between simulated and realistic environments. To advance the Sim2Real generalization, in this study, we present DexScale, a data engine designed to perform automatic skills simulation and scaling for learning deployable robot manipulation policies. Specifically, DexScale ensures the usability of simulated skills by integrating diverse forms of realistic data into the simulated environment, preserving semantic alignment with the target tasks. For each simulated skill in the environment, DexScale facilitates effective Sim2Real data scaling by automating the process of domain randomization and adaptation. Tuned by the scaled dataset, the control policy achieves zero-shot Sim2Real generalization across diverse tasks, multiple robot embodiments, and widely studied policy model architectures, highlighting its importance in advancing Sim2Real embodied intelligence.

Training robots to perform real-world tasks usually requires huge amounts of data collected from actual robots. This process is slow, expensive, and difficult to scale. A faster alternative is to train robots in virtual simulations. But here’s the problem: what works in simulation often fails in reality because the virtual world doesn’t fully reflect the complexity of the real one.To solve this, we developed DexScale, a tool that generates more useful and realistic robot training data in simulation. DexScale mixes in elements from real-world experiences and automatically adjusts the virtual training environment to semantically align with real-life conditions. Besides, DexScale designs automatic domain randomization and adaptation methods, which make simulations more robust and diverse.What makes DexScale especially powerful is its ability to achieve zero-shot Sim2Real transfer. This means a robot trained only in simulation, without any additional real-world fine-tuning, can successfully perform complex tasks in the real world right away. DexScale works across a variety of tasks, robot types, and learning models, making it a significant step toward building general-purpose, real-world-ready robots.