Timezone: »

 
Spotlight
The Role of Deconfounding in Meta-learning
Yinjie Jiang · Zhengyu Chen · Kun Kuang · Luotian Yuan · Xinhai Ye · Zhihua Wang · Fei Wu · Ying WEI

Tue Jul 19 01:35 PM -- 01:40 PM (PDT) @ Ballroom 3 & 4
in MISC: Causality »

Meta-learning has emerged as a potent paradigm for quick learning of few-shot tasks, by leveraging the meta-knowledge learned from meta-training tasks. Well-generalized meta-knowledge that facilitates fast adaptation in each task is preferred; however, recent evidence suggests the undesirable memorization effect where the meta-knowledge simply memorizing all meta-training tasks discourages task-specific adaptation and poorly generalizes. There have been several solutions to mitigating the effect, including both regularizer-based and augmentation-based methods, while a systematic understanding of these methods in a single framework is still lacking. In this paper, we offer a novel causal perspective of meta-learning. Through the lens of causality, we conclude the universal label space as a confounder to be the causing factor of memorization and frame the two lines of prevailing methods as different deconfounder approaches. Remarkably, derived from the causal inference principle of front-door adjustment, we propose two frustratingly easy but effective deconfounder algorithms, i.e., sampling multiple versions of the meta-knowledge via Dropout and grouping the meta-knowledge into multiple bins. The proposed causal perspective not only brings in the two deconfounder algorithms that surpass previous works in four benchmark datasets towards combating memorization, but also opens a promising direction for meta-learning.

Keywords: OPT: Bilevel optimization · MISC: Causality · MISC: Transfer, Multitask and Meta-learning

Author Information

Yinjie Jiang (Zhejiang University)
Zhengyu Chen (Zhejiang University)
Kun Kuang (Zhejiang University)
Kun Kuang

Kun Kuang is an Associate Professor at the College of Computer Science and Technology, Zhejiang University. He received his Ph.D. in the Department of Computer Science and Technology at Tsinghua University in 2019. He was a visiting scholar with Prof. Susan Athey's Group at Stanford University. His main research interests include Causal Inference, Data Mining, and Causality Inspired Machine Learning. He has published over 70 papers in prestigious conferences and journals in data mining and machine learning, including TKDE, TPAMI, ICML, NeurIPS, KDD, ICDE, WWW, MM, DMKD, Engineering, etc. He received ACM SIGAI China Rising Star Award in 2022.

Luotian Yuan (Zhejiang University)
Xinhai Ye (Zhejiang University)
Zhihua Wang (Shanghai Institute for Advanced Study of Zhejiang University)
Fei Wu (Zhejiang University, China)
Ying WEI (City University of Hong Kong)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors