Offline Multi-agent Continual Cooperation via Skill Partition and Reuse

Extracting skills from multi-agent offline dataset improves learning efficiency via sharing task-invariant coordination skills among tasks. In settings where tasks occur sequentially and the space of skills grows exponentially, existing approaches that rely on heuristically designed and fixed-sized skill libraries struggle to resolve the problem of distributional shift and interference, facing catastrophic forgetting and plasticity loss. To address this problem and endow agents with the ability to continually discover and reuse coordination skills in open-environment, we propose COMAD, a principled framework for Continual Offline Multi-agent Skill Discovery via Skill Partition and Reuse. We first discover skills from mixed multi-agent behavior data with an auto-encoder to transform coordination knowledge into reusable coordination skills. Then we construct a skill-augmented policy learning objective with multi-head architectures, explicitly guiding the advantage function with reusable skills identified via a density-based reusability estimator. Theoretical analysis shows our method approximates the optimum of a continual skill discovery problem. Empirical results across diverse MARL benchmarks show that COMAD continually expands its skill library to mitigate interference, achieving superior forward and backward transfer for task streams compared to multiple baselines.

AI agents often need to work together as a team, for example in games, robots, or traffic systems. In practice, they may face a sequence of new tasks, but only have access to previously collected data rather than the ability to freely try new actions in the real world. This makes learning difficult: the agents must acquire new ways to cooperate without forgetting useful skills from earlier tasks. We introduce COMAD, a method that helps a team of agents continually discover, store, and reuse coordination skills from offline data. When a new task arrives, COMAD decides whether existing skills are relevant or whether new skills should be added. It then uses reusable skills to guide learning on the new task while keeping older skills separated. Our results show that this approach helps agents adapt to new cooperative tasks while retaining past knowledge. This is a step toward multi-agent AI systems that can build up useful teamwork skills over time, instead of starting over or forgetting whenever their environment changes.