Zero-Shot Utility and Efficient Adaptation in Vision-Language Multi-Agent Control

Vision-language models (VLMs) provide a promising starting point for multi-agent control because they can act before environment-specific training. However, it remains unclear how far this zero-shot advantage extends in partially observable cooperative settings, and how adapted VLM agents compare with specialized multi-agent reinforcement learning (MARL) policies. This work studies this question in a cooperative pursuit benchmark with controlled distribution shifts spanning visual appearance, semantic remapping, observation layout, agent counts, and environment scale. We compare zero-shot VLMs, LoRA-adapted VLMs, cold-start MARL, and fully trained IPPO/MAPPO baselines under matched local-observation constraints. Results show that zero-shot VLMs provide useful behavior where cold-start RL is nearly non-functional, and that supervised adaptation with 1k--25k expert demonstrations rapidly produces competitive long-horizon controllers. Adapted VLMs are especially robust to visual and semantic shifts, while trained MARL remains strongest on the in-distribution task and some coordination-heavy variants. Overall, the results reveal complementary scaling behavior: VLMs adapt rapidly from limited expert data and generalize better across visual-semantic shifts, while MARL remains stronger when extensive task-specific interaction is available.