Decentralized and Disentangled Task–Role Representation Learning for Generalizable Offline Multi-Agent Meta Reinforcement Learning

Offline meta reinforcement learning (RL) enables agents to learn a unified policy from multi-task offline data to support generalization in out-of-distribution (OOD) tasks. Recent approaches in single-agent RL tackle this by learning an efficient task representation to distinguish between tasks, showing promising adaptation ability. However, when extended to multi-agent settings, these methods struggle with decentralized task identification due to limited global information, and suffer from inefficient knowledge transfer in the absence of role information. To address this, we propose D$^2$TR, a novel context-based meta RL framework with efficient decentralized and disentangled task-role identification. Specifically, D$^2$TR first introduces mutual information knowledge distillation to align decentralized task representations with centralized task representations inferred from global trajectories, enabling efficient decentralized team-centric information identification. Next, D$^2$TR leverages a large language model to assign semantic roles to trajectories in offline data, and achieves effective individual-centric information inference by learning decentralized role representations. Extensive experiments conducted on commonly used multi-agent environments, including CN, SMAC, and SMACv2, demonstrate that D$^2$TR exhibits strong generalization performance to unseen tasks, outperforming prior multi-agent multi-task and context-based meta RL baselines.

Teams of AI agents are increasingly expected to cooperate in new situations, but real-world practice can be expensive or risky, so they may have to learn only from past records. This is hard because each agent sees only part of the situation, and the data usually does not say which teammate was playing which role. We propose D2TR, a method that helps agents infer both the team’s current goal and each agent’s individual job. D2TR first learns from full team histories, then teaches each agent to recognize the task from its own limited history. It also uses a large language model to turn behavior patterns into role labels, such as covering a target or fighting from a particular position. These task and role cues guide the agents’ decisions, helping them reuse and recombine learned behaviors on new tasks. In navigation and StarCraft-based benchmarks, D2TR performs better on unseen tasks than prior multi-task and meta-learning methods. This suggests that cooperative AI can become more adaptable when it understands both what the team is trying to do and who should do which job, which may help future robot teams and game agents handle changing goals and teammates.