Distilling Task-Level Coordination Policies for Generalizable Multi-Agent Cooperation

Large language models have shown strong reasoning abilities and are increasingly explored as high-level coordinators for multi-agent systems. However, directly deploying LLMs for coordination remains challenging, as effective policies often fail to reliably emerge at the low-level control stage, and inference costs limit scalability. We propose SynCoord (Synthetic Coordination Distillation), a self-supervised pipeline that distills task-level decision-making for cooperation from high-capacity reasoning models into lightweight agent policies. Our approach does not rely on explicit supervision or handcrafted coordination rules. Instead, we define a set of task-level tool interfaces that constrain LLM interaction and enable the collection of interaction trajectories, which are then used to train compact coordinated policies. This distillation process transfers coordination behaviors that are difficult to elicit through prompting alone, while substantially reducing inference overhead at execution time. We evaluate our method on cooperative multi-agent benchmarks including Overcooked-AI and Level-Based Foraging (LBF), under varying team sizes and environment scales. Experimental results show that the distilled policies achieve success rates and execution efficiency comparable to reinforcement learning–based methods, while exhibiting fewer erroneous or redundant actions. Moreover, the learned task-level coordination policy generalizes effectively to unseen team compositions and larger layouts without retraining.

Modern AI systems are becoming increasingly capable of reasoning and planning, but enabling multiple AI agents to cooperate effectively remains a major challenge. In real-world scenarios such as household robotics, autonomous delivery, or virtual assistants, agents must not only complete their own tasks, but also coordinate with others under changing environments and limited information. Traditional multi-agent reinforcement learning methods can achieve strong performance in fixed environments, but they often struggle to generalize when the environment changes or when the number of agents increases. On the other hand, large language models (LLMs) show impressive high-level reasoning abilities, yet directly deploying them for real-time multi-agent control is computationally expensive and often unreliable at low-level execution. In this work, we introduce SynCoord, a framework that combines the strengths of both approaches. Instead of using large language models directly during deployment, we use them offline as “Thinking Agents” that generate examples of effective coordination strategies. These strategies are then distilled into smaller and faster “Distilled Agents” that can act efficiently in real time. We evaluate SynCoord in cooperative game-like environments where agents must work together to complete tasks such as preparing recipes or collecting resources. Our experiments show that the distilled agents achieve strong coordination performance while being significantly more efficient than directly using large language models. Importantly, the learned coordination behaviors also generalize to unseen tasks, larger environments, and different team sizes without retraining. Our results suggest that high-level cooperative reasoning can be transferred from powerful reasoning models into lightweight agents, providing a scalable path toward practical multi-agent AI systems. We believe this direction may contribute to future applications in robotics, human-AI collaboration, and autonomous multi-agent systems.