DyGRO-VLA: Cross-Task Scaling of Vision–Language–Action Models via Dynamic Grouped Residual Optimization

Recent progress in Reinforcement Learning (RL) provides a principled approach to optimizing Vision-Language-Action (VLA) models, facilitating a shift from trajectory imitation to active learning in the task environment. Despite improvements in control precision, most RL optimizers remain task-specific, which reduces VLA models from generalist controllers to policies that overfit to a narrow set of tasks. In this study, we conduct an in-depth analysis of this phenomenon and highlight the importance of cross-task feature representations for improving the generalizability of VLA models. Motivated by this finding, we introduce DyGRO-VLA, a two-stage optimization framework that 1) effectively captures cross-task latent representations based on information-theoretic principles, and 2) dynamically refines policy optimization via a mixture-of-RL-residuals. DyGRO-VLA enables the RL optimizer to exploit task-relevant latent information while strategically mitigating adverse interference on the learned representations throughout the optimization process. We evaluate our approach on LIBERO, RoboTwin2 benchmarks, and further validate it on real world, demonstrating consistent improvements over strong baselines under multi-task training and distribution shift. Our project page is available at DyGRO-VLA.

A major goal in robotics is to build robots that can understand images and language instructions and then perform many different tasks in the real world. Existing training methods can improve a robot on specific tasks, but they may also cause the robot to forget or perform worse on other tasks. This paper presents DyGRO-VLA, a new training approach that helps robot models improve across multiple tasks while preserving the shared knowledge they have already learned. Instead of changing the whole model for every task, DyGRO-VLA keeps the general robot knowledge stable and adds small, task-specific improvements when needed. Tests in simulated environments and real-world robot experiments show that this approach improves manipulation performance. This work is a step toward more reliable general-purpose robots that can adapt to diverse situations.