ICML 2023

Timezone: »

Poster

Provably Invariant Learning without Domain Information

Xiaoyu Tan · Yong LIN · Shengyu Zhu · Chao Qu · Xihe Qiu · Xu Yinghui · Peng Cui · Yuan Qi

Tue Jul 25 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #702

in Poster Session 2 »

Typical machine learning applications always assume the data follows independent and identically distributed (IID) assumptions. In contrast, this assumption is frequently violated in real-world circumstances, leading to the Out-of-Distribution (OOD) generalization problem and a major drop in model robustness. To mitigate this issue, the invariant learning technique is leveraged to distinguish between spurious features and invariant features among all input features and to train the model purely on the basis of the invariant features. Numerous invariant learning strategies imply that the training data should contain domain information. Such information includes the environment index or auxiliary information acquired from prior knowledge. However, acquiring these information is typically impossible in practice. In this study, we present TIVA for environment-independent invariance learning, which requires no environment-specific information in training data. We discover and prove that, given certain mild data conditions, it is possible to train an environment partitioning policy based on attributes that are independent of the targets and then conduct invariant risk minimization. We examine our method in comparison to other baseline methods, which demonstrate superior performance and excellent robustness under OOD, using multiple benchmarks.

[ PDF] [ Poster]

Author Information

Xiaoyu Tan (INF Technology (Shanghai) Co., Ltd.)

Yong LIN (The Hong Kong University of Science and Technology)

Shengyu Zhu (Huawei Noah's Ark Lab)

Chao Qu (Inftech)

Xihe Qiu (Shanghai University of Engineering Science)

Xu Yinghui (Fudan University)

Peng Cui (Tsinghua University)

Peng Cui is an Associate Professor in Tsinghua University. He got his PhD degree from Tsinghua University in 2010. His research interests include causal inference and stable learning, network representation learning, and human behavioral modeling. He has published more than 100 papers in prestigious conferences and journals in data mining and multimedia. His recent research won the IEEE Multimedia Best Department Paper Award, SIGKDD 2016 Best Paper Finalist, ICDM 2015 Best Student Paper Award, SIGKDD 2014 Best Paper Finalist, IEEE ICME 2014 Best Paper Award, ACM MM12 Grand Challenge Multimodal Award, and MMM13 Best Paper Award. He is the Associate Editors of IEEE TKDE, IEEE TBD, ACM TIST, and ACM TOMM etc. He has served as program co-chair and area chair of several major machine learning and artificial intelligence conferences, such as IJCAI, AAAI, ACM CIKM, ACM Multimedia etc.