CooT: Learning to Coordinate In-Context with Coordination Transformers

Effective coordination among unfamiliar partners remains a major challenge in multi-agent systems. Existing approaches, such as population-based methods, improve robustness through diversity but often lack mechanisms for efficient adaptation beyond the training distribution. Fine-tuning is also impractical for few-shot learning because it requires a large number of interactions for meaningful improvement. To address these limitations, we propose Coordination Transformers (CooT), a framework that leverages in-context learning (ICL) for real-time partner adaptation. Unlike prior ICL approaches that focus on task generalization, CooT is designed to generalize across diverse partner behaviors. Trained on trajectories from behavior-preferring agents, it learns to align actions with partner intentions purely through observation. We evaluate CooT on two challenging multi-agent benchmarks: Overcooked and Google Research Football. Results show that CooT consistently outperforms population-based methods, gradient-based fine-tuning, and Meta-RL baselines, achieving stable and rapid adaptation without parameter updates. Human evaluations also identify CooT as a preferred collaborator, and our ablations confirm its ability to adapt quickly to new partners and remain stable under sudden partner changes, making it reliable for real-world human-AI collaboration.

(1) Problem. AI agents trained to cooperate often master teamwork only with their training partners. Pairing them with an unfamiliar AI or human, and coordination falls apart. (2) Solution. We developed CooT (Coordination Transformers), which borrows the "in-context learning" trick from large language models: instead of retraining for each new partner, the agent watches a few rounds of play and adapts on the spot. We trained it on interactions between agents with deliberately mismatched preferences, teaching it the general skill of reading a partner's style. (3) Impact. In the Overcooked game and Google Research Football, CooT outperformed existing methods with unfamiliar AI partners, and human players ranked it their favorite teammate — a step toward AI that can adapt to real people without retraining.