A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

Co-training, which combines limited in-domain real-world data with abundant surrogate data such as simulation or cross-embodiment robot data, is widely used for training generative robot policies. Despite its empirical success, the mechanisms that determine when and why co-training is effective remain poorly understood. We investigate the mechanism of sim-and-real co-training through theoretical analysis and empirical study, and identify two intrinsic effects governing performance. The first, "structured representation alignment", reflects a balance between cross-domain representation alignment and domain discernibility, and plays a primary role in downstream performance. The second, the "importance reweighting effect", arises from domain-dependent modulation of action weighting and operates at a secondary level. We validate these effects with controlled experiments on a toy model and extensive sim-and-sim and sim-and-real robot manipulation experiments. Our analysis offers a unified interpretation of recent co-training techniques and motivates a simple method that consistently improves upon prior approaches. More broadly, our aim is to examine the inner workings of co-training and to facilitate research in this direction.

Robots often need large amounts of data to learn useful skills, but collecting real-world robot data is expensive and time-consuming. A common workaround is co-training: combining a small amount of real-world data with much larger amounts of related data, such as simulation data or data from other robots. Although this strategy often improves robot learning, it has not been clear why it works well in some cases but not others. In this work, we study the inner mechanisms behind sim-and-real co-training for generative robot policies. We find that performance is mainly governed by how the policy represents the two data sources: the simulation and real-world data should be aligned enough to share useful structure, but still distinguishable enough for the robot to adapt its actions to the real world. We call this effect structured representation alignment. We also identify a second effect, importance reweighting, where the training process gives different amounts of influence to different domains depending on the data and mixing ratio. We validate these ideas through theory, controlled toy experiments, and robot manipulation experiments. Our findings help explain several existing co-training methods and lead to a simple improved method for training robot policies with mixed real and surrogate data.