Timezone: »
Despite the success of invariant risk minimization (IRM) in tackling the Out-of-Distribution generalization problem, IRM can compromise the optimality when applied in practice. The practical variants of IRM, e.g., IRMv1, have been shown to have significant gaps with IRM and thus could fail to capture the invariance even in simple problems. Moreover, the optimization procedure in IRMv1 involves two intrinsically conflicting objectives, and often requires careful tuning for the objective weights. To remedy the above issues, we reformulate IRM as a multi-objective optimization problem, and propose a new optimization scheme for IRM, called PAreto Invariant Risk Minimization (PAIR). PAIR can adaptively adjust the optimization direction under the objective conflicts. Furthermore, we show PAIR can empower the practical IRM variants to overcome the barriers with the original IRM when provided with proper guidance. We conduct experiments with ColoredMNIST to confirm our theory and the effectiveness of PAIR.
Author Information
Yongqiang Chen (The Chinese University of Hong Kong)
Kaiwen Zhou (The Chinese University of Hong Kong)
Yatao Bian (Tencent AI Lab)
Binghui Xie (The Chinese University of Hong Kong)
Kaili MA (CUHK)
Yonggang Zhang (Hong Kong Baptist University)
Han Yang (The Chinese University of Hong Kong)
Bo Han (HKBU / RIKEN)
James Cheng (CUHK)
More from the Same Authors
-
2022 : Invariance Principle Meets Out-of-Distribution Generalization on Graphs »
Yongqiang Chen · Yonggang Zhang · Yatao Bian · Han Yang · Kaili MA · Binghui Xie · Tongliang Liu · Bo Han · James Cheng -
2023 Poster: A Universal Unbiased Method for Classification from Aggregate Observations »
Zixi Wei · LEI FENG · Bo Han · Tongliang Liu · Gang Niu · Xiaofeng Zhu · Heng Tao Shen -
2023 Poster: Detecting Out-of-distribution Data through In-distribution Class Prior »
Xue JIANG · Feng Liu · zhen fang · Hong Chen · Tongliang Liu · Feng Zheng · Bo Han -
2023 Poster: Which is Better for Learning with Noisy Labels: The Semi-supervised Method or Modeling Label Noise? »
Yu Yao · Mingming Gong · Yuxuan Du · Jun Yu · Bo Han · Kun Zhang · Tongliang Liu -
2023 Poster: On Strengthening and Defending Graph Reconstruction Attack with Markov Chain Approximation »
Zhanke Zhou · Chenyu Zhou · Xuan Li · Jiangchao Yao · QUANMING YAO · Bo Han -
2023 Poster: Diversity-enhancing Generative Network for Few-shot Hypothesis Adaptation »
Ruijiang Dong · Feng Liu · Haoang Chi · Tongliang Liu · Mingming Gong · Gang Niu · Masashi Sugiyama · Bo Han -
2023 Poster: Detecting Adversarial Data by Probing Multiple Perturbations Using Expected Perturbation Score »
Shuhai Zhang · Feng Liu · Jiahao Yang · 逸凡 杨 · Changsheng Li · Bo Han · Mingkui Tan -
2023 Poster: Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability »
Jianing Zhu · Hengzhuang Li · Jiangchao Yao · Tongliang Liu · Jianliang Xu · Bo Han -
2023 Poster: Exploring Model Dynamics for Accumulative Poisoning Discovery »
Jianing Zhu · Xiawei Guo · Jiangchao Yao · Chao Du · LI He · Shuo Yuan · Tongliang Liu · Liang Wang · Bo Han -
2023 Poster: Moderately Distributional Exploration for Domain Generalization »
Rui Dai · Yonggang Zhang · zhen fang · Bo Han · Xinmei Tian -
2022 : DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations »
Yatao Bian -
2022 : Hypergraph Convolutional Networks via Equivalence Between Hypergraphs and Undirected Graphs »
Jiying Zhang · fuyang li · Xi Xiao · Tingyang Xu · Yu Rong · Junzhou Huang · Yatao Bian -
2022 Poster: Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network »
Shuo Yang · Erkun Yang · Bo Han · Yang Liu · Min Xu · Gang Niu · Tongliang Liu -
2022 Poster: Contrastive Learning with Boosted Memorization »
Zhihan Zhou · Jiangchao Yao · Yan-Feng Wang · Bo Han · Ya Zhang -
2022 Poster: Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning »
Zhenheng Tang · Yonggang Zhang · Shaohuai Shi · Xin He · Bo Han · Xiaowen Chu -
2022 Spotlight: Contrastive Learning with Boosted Memorization »
Zhihan Zhou · Jiangchao Yao · Yan-Feng Wang · Bo Han · Ya Zhang -
2022 Spotlight: Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning »
Zhenheng Tang · Yonggang Zhang · Shaohuai Shi · Xin He · Bo Han · Xiaowen Chu -
2022 Spotlight: Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network »
Shuo Yang · Erkun Yang · Bo Han · Yang Liu · Min Xu · Gang Niu · Tongliang Liu -
2022 Poster: $p$-Laplacian Based Graph Neural Networks »
Guoji Fu · Peilin Zhao · Yatao Bian -
2022 Poster: Understanding Robust Overfitting of Adversarial Training and Beyond »
Chaojian Yu · Bo Han · Li Shen · Jun Yu · Chen Gong · Mingming Gong · Tongliang Liu -
2022 Poster: Modeling Adversarial Noise for Adversarial Training »
Dawei Zhou · Nannan Wang · Bo Han · Tongliang Liu -
2022 Poster: Improving Adversarial Robustness via Mutual Information Estimation »
Dawei Zhou · Nannan Wang · Xinbo Gao · Bo Han · Xiaoyu Wang · Yibing Zhan · Tongliang Liu -
2022 Spotlight: Understanding Robust Overfitting of Adversarial Training and Beyond »
Chaojian Yu · Bo Han · Li Shen · Jun Yu · Chen Gong · Mingming Gong · Tongliang Liu -
2022 Spotlight: $p$-Laplacian Based Graph Neural Networks »
Guoji Fu · Peilin Zhao · Yatao Bian -
2022 Spotlight: Improving Adversarial Robustness via Mutual Information Estimation »
Dawei Zhou · Nannan Wang · Xinbo Gao · Bo Han · Xiaoyu Wang · Yibing Zhan · Tongliang Liu -
2022 Spotlight: Modeling Adversarial Noise for Adversarial Training »
Dawei Zhou · Nannan Wang · Bo Han · Tongliang Liu -
2022 Poster: On the Finite-Time Complexity and Practical Computation of Approximate Stationarity Concepts of Lipschitz Functions »
Lai Tian · Kaiwen Zhou · Anthony Man-Cho So -
2022 Poster: Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack »
Ruize Gao · Jiongxiao Wang · Kaiwen Zhou · Feng Liu · Binghui Xie · Gang Niu · Bo Han · James Cheng -
2022 Spotlight: Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack »
Ruize Gao · Jiongxiao Wang · Kaiwen Zhou · Feng Liu · Binghui Xie · Gang Niu · Bo Han · James Cheng -
2022 Spotlight: On the Finite-Time Complexity and Practical Computation of Approximate Stationarity Concepts of Lipschitz Functions »
Lai Tian · Kaiwen Zhou · Anthony Man-Cho So -
2021 Poster: Towards Defending against Adversarial Examples via Attack-Invariant Features »
Dawei Zhou · Tongliang Liu · Bo Han · Nannan Wang · Chunlei Peng · Xinbo Gao -
2021 Spotlight: Towards Defending against Adversarial Examples via Attack-Invariant Features »
Dawei Zhou · Tongliang Liu · Bo Han · Nannan Wang · Chunlei Peng · Xinbo Gao -
2020 Poster: Dual-Path Distillation: A Unified Framework to Improve Black-Box Attacks »
Yonggang Zhang · Ya Li · Tongliang Liu · Xinmei Tian -
2020 Poster: From Sets to Multisets: Provable Variational Inference for Probabilistic Integer Submodular Models »
Aytunc Sahin · Yatao Bian · Joachim Buhmann · Andreas Krause -
2019 Poster: Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference »
Yatao Bian · Joachim Buhmann · Andreas Krause -
2019 Oral: Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference »
Yatao Bian · Joachim Buhmann · Andreas Krause -
2018 Poster: A Distributed Second-Order Algorithm You Can Trust »
Celestine Mendler-Dünner · Aurelien Lucchi · Matilde Gargiani · Yatao Bian · Thomas Hofmann · Martin Jaggi -
2018 Oral: A Distributed Second-Order Algorithm You Can Trust »
Celestine Mendler-Dünner · Aurelien Lucchi · Matilde Gargiani · Yatao Bian · Thomas Hofmann · Martin Jaggi -
2018 Poster: A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates »
Kaiwen Zhou · Fanhua Shang · James Cheng -
2018 Oral: A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates »
Kaiwen Zhou · Fanhua Shang · James Cheng -
2017 Poster: Guarantees for Greedy Maximization of Non-submodular Functions with Applications »
Yatao Bian · Joachim Buhmann · Andreas Krause · Sebastian Tschiatschek -
2017 Talk: Guarantees for Greedy Maximization of Non-submodular Functions with Applications »
Yatao Bian · Joachim Buhmann · Andreas Krause · Sebastian Tschiatschek