Timezone: »
Ad hoc teamwork requires an agent to cooperate with unknown teammates without prior coordination. Many works propose to abstract teammate instances into high-level representation of types and then pre-train the best response for each type. However, most of them do not consider the distribution of teammate instances within a type. This could expose the agent to the hidden risk of type confounding. In the worst case, the best response for an abstract teammate type could be the worst response for all specific instances of that type. This work addresses the issue from the lens of causal inference. We first theoretically demonstrate that this phenomenon is due to the spurious correlation brought by uncontrolled teammate distribution. Then, we propose our solution, CTCAT, which disentangles such correlation through an instance-wise teammate feedback rectification. This operation reweights the interaction of teammate instances within a shared type to reduce the influence of type confounding. The effect of CTCAT is evaluated in multiple domains, including classic ad hoc teamwork tasks and real-world scenarios. Results show that CTCAT is robust to the influence of type confounding, a practical issue that directly hazards the robustness of our trained agents but was unnoticed in previous works.
Author Information
Dong Xing (Zhejiang University)
Pengjie Gu (Nanyang Technological University)
Qian Zheng (Zhejiang University)
Xinrun Wang (Nanyang Technological University)
Shanqi Liu (Zhejiang University)
Longtao Zheng (Nanyang Technological University)
Bo An (Nanyang Technological University)
Gang Pan (Zhejiang University)
More from the Same Authors
-
2023 Poster: Mitigating Memorization of Noisy Labels by Clipping the Model Prediction »
Hongxin Wei · HUIPING ZHUANG · RENCHUNZI XIE · Lei Feng · Gang Niu · Bo An · Sharon Li -
2023 Poster: Weakly Supervised Regression with Interval Targets »
Xin Cheng · Yuzhou Cao · Ximing Li · Bo An · Lei Feng -
2023 Poster: Rethinking Visual Reconstruction: Experience-Based Content Completion Guided by Visual Cues »
Jiaxuan Chen · Yu Qi · Gang Pan -
2022 Poster: Learning Pseudometric-based Action Representations for Offline Reinforcement Learning »
Pengjie Gu · Mengchen Zhao · Chen Chen · Dong Li · Jianye Hao · Bo An -
2022 Spotlight: Learning Pseudometric-based Action Representations for Offline Reinforcement Learning »
Pengjie Gu · Mengchen Zhao · Chen Chen · Dong Li · Jianye Hao · Bo An -
2020 Poster: Converging to Team-Maxmin Equilibria in Zero-Sum Multiplayer Games »
Youzhi Zhang · Bo An