Timezone: »
Meta-learning has emerged as a potent paradigm for quick learning of few-shot tasks, by leveraging the meta-knowledge learned from meta-training tasks. Well-generalized meta-knowledge that facilitates fast adaptation in each task is preferred; however, recent evidence suggests the undesirable memorization effect where the meta-knowledge simply memorizing all meta-training tasks discourages task-specific adaptation and poorly generalizes. There have been several solutions to mitigating the effect, including both regularizer-based and augmentation-based methods, while a systematic understanding of these methods in a single framework is still lacking. In this paper, we offer a novel causal perspective of meta-learning. Through the lens of causality, we conclude the universal label space as a confounder to be the causing factor of memorization and frame the two lines of prevailing methods as different deconfounder approaches. Remarkably, derived from the causal inference principle of front-door adjustment, we propose two frustratingly easy but effective deconfounder algorithms, i.e., sampling multiple versions of the meta-knowledge via Dropout and grouping the meta-knowledge into multiple bins. The proposed causal perspective not only brings in the two deconfounder algorithms that surpass previous works in four benchmark datasets towards combating memorization, but also opens a promising direction for meta-learning.
Author Information
Yinjie Jiang (Zhejiang University)
Zhengyu Chen (Zhejiang University)
Kun Kuang (Zhejiang University)

Kun Kuang is an Associate Professor at the College of Computer Science and Technology, Zhejiang University. He received his Ph.D. in the Department of Computer Science and Technology at Tsinghua University in 2019. He was a visiting scholar with Prof. Susan Athey's Group at Stanford University. His main research interests include Causal Inference, Data Mining, and Causality Inspired Machine Learning. He has published over 70 papers in prestigious conferences and journals in data mining and machine learning, including TKDE, TPAMI, ICML, NeurIPS, KDD, ICDE, WWW, MM, DMKD, Engineering, etc. He received ACM SIGAI China Rising Star Award in 2022.
Luotian Yuan (Zhejiang University)
Xinhai Ye (Zhejiang University)
Zhihua Wang (Shanghai Institute for Advanced Study of Zhejiang University)
Fei Wu (Zhejiang University, China)
Ying WEI (City University of Hong Kong)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: The Role of Deconfounding in Meta-learning »
Tue. Jul 19th through Wed the 20th Room Hall E #501
More from the Same Authors
-
2022 : Towards Multi-level Fairness and Robustness on Federated Learning »
Fengda Zhang · Kun Kuang · Yuxuan Liu · Long Chen · Jiaxun Lu · Yunfeng Shao · Fei Wu · Chao Wu · Jun Xiao -
2023 Poster: Stable Estimation of Heterogeneous Treatment Effects »
Anpeng Wu · Kun Kuang · Ruoxuan Xiong · Bo Li · Fei Wu -
2023 Poster: Causal Structure Learning for Latent Intervened Non-stationary Data »
Chenxi Liu · Kun Kuang -
2022 Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward »
Huaxiu Yao · Hugo Larochelle · Percy Liang · Colin Raffel · Jian Tang · Ying WEI · Saining Xie · Eric Xing · Chelsea Finn -
2022 Poster: Frustratingly Easy Transferability Estimation »
Long-Kai Huang · Junzhou Huang · Yu Rong · Qiang Yang · Ying WEI -
2022 Poster: Instrumental Variable Regression with Confounder Balancing »
Anpeng Wu · Kun Kuang · Bo Li · Fei Wu -
2022 Poster: Deconfounded Value Decomposition for Multi-Agent Reinforcement Learning »
Jiahui Li · Kun Kuang · Baoxiang Wang · Furui Liu · Long Chen · Changjie Fan · Fei Wu · Jun Xiao -
2022 Spotlight: Deconfounded Value Decomposition for Multi-Agent Reinforcement Learning »
Jiahui Li · Kun Kuang · Baoxiang Wang · Furui Liu · Long Chen · Changjie Fan · Fei Wu · Jun Xiao -
2022 Spotlight: Instrumental Variable Regression with Confounder Balancing »
Anpeng Wu · Kun Kuang · Bo Li · Fei Wu -
2022 Spotlight: Frustratingly Easy Transferability Estimation »
Long-Kai Huang · Junzhou Huang · Yu Rong · Qiang Yang · Ying WEI -
2021 Poster: KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation »
Haozhe Feng · Zhaoyang You · Minghao Chen · Tianye Zhang · Minfeng Zhu · Fei Wu · Chao Wu · Wei Chen -
2021 Poster: Improving Generalization in Meta-learning via Task Augmentation »
Huaxiu Yao · Long-Kai Huang · Linjun Zhang · Ying WEI · Li Tian · James Zou · Junzhou Huang · Zhenhui (Jessie) Li -
2021 Spotlight: Improving Generalization in Meta-learning via Task Augmentation »
Huaxiu Yao · Long-Kai Huang · Linjun Zhang · Ying WEI · Li Tian · James Zou · Junzhou Huang · Zhenhui (Jessie) Li -
2021 Spotlight: KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation »
Haozhe Feng · Zhaoyang You · Minghao Chen · Tianye Zhang · Minfeng Zhu · Fei Wu · Chao Wu · Wei Chen -
2021 Poster: Meta-learning Hyperparameter Performance Prediction with Neural Processes »
Ying WEI · Peilin Zhao · Junzhou Huang -
2021 Spotlight: Meta-learning Hyperparameter Performance Prediction with Neural Processes »
Ying WEI · Peilin Zhao · Junzhou Huang -
2021 Poster: Explainable Automated Graph Representation Learning with Hyperparameter Importance »
Xin Wang · Shuyi Fan · Kun Kuang · Wenwu Zhu -
2021 Spotlight: Explainable Automated Graph Representation Learning with Hyperparameter Importance »
Xin Wang · Shuyi Fan · Kun Kuang · Wenwu Zhu -
2020 Poster: Description Based Text Classification with Reinforcement Learning »
Duo Chai · Wei Wu · Qinghong Han · Fei Wu · Jiwei Li -
2019 Poster: Hierarchically Structured Meta-learning »
Huaxiu Yao · Ying WEI · Junzhou Huang · Zhenhui (Jessie) Li -
2019 Oral: Hierarchically Structured Meta-learning »
Huaxiu Yao · Ying WEI · Junzhou Huang · Zhenhui (Jessie) Li -
2019 Poster: Disentangled Graph Convolutional Networks »
Jianxin Ma · Peng Cui · Kun Kuang · Xin Wang · Wenwu Zhu -
2019 Oral: Disentangled Graph Convolutional Networks »
Jianxin Ma · Peng Cui · Kun Kuang · Xin Wang · Wenwu Zhu -
2018 Poster: Transfer Learning via Learning to Transfer »
Ying WEI · Yu Zhang · Junzhou Huang · Qiang Yang -
2018 Oral: Transfer Learning via Learning to Transfer »
Ying WEI · Yu Zhang · Junzhou Huang · Qiang Yang