Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly and selects them adaptively based on a learnability criterion, removing the need for pre-trained partner populations or manual parameter tuning. We show that this simple mechanism enables effective partner diversity and can be extended to joint partner-environment selection when a procedural level generator is available. Across Level-Based Foraging, Overcooked-AI, and the Overcooked Generalisation Challenge, UPD consistently achieves strong performance compared to both population-based and population-free baselines. In a human-AI user study, agents trained with UPD achieve higher returns and are rated as more adaptive, more human-like, and less frustrating than all evaluated baseline methods.

AI agents often struggle to cooperate with unfamiliar partners because they are typically trained with a fixed set of teammates. This can make them brittle when paired with new AI agents or humans. This paper introduces Unsupervised Partner Design (UPD), a new training method for cooperative AI. Instead of relying on a pre-trained population of training partners, UPD continuously generates new partner behaviours automatically during training. The key idea is simple: generate a large number of possible partner behaviours and train the AI using partners that are neither too easy nor too difficult to cooperate with. This creates an adaptive curriculum over partner behaviours that changes together with the learning progress of the AI agent. The method is simple, does not require manually curated training populations, and naturally extends to settings where both the environment and the cooperation partner change over time. Experiments in cooperative game environments show that UPD improves coordination with unseen AI agents and human players, while also scaling to procedurally generated environments where previous approaches struggle.