Timezone: »

 
Poster
KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation
Haozhe Feng · Zhaoyang You · Minghao Chen · Tianye Zhang · Minfeng Zhu · Fei Wu · Chao Wu · Wei Chen

Thu Jul 22 09:00 PM -- 11:00 PM (PDT) @ Virtual
in Poster Session 6 »

Conventional unsupervised multi-source domain adaptation (UMDA) methods assume all source domains can be accessed directly. However, this assumption neglects the privacy-preserving policy, where all the data and computations must be kept decentralized. There exist three challenges in this scenario: (1) Minimizing the domain distance requires the pairwise calculation of the data from the source and target domains, while the data on the source domain is not available. (2) The communication cost and privacy security limit the application of existing UMDA methods, such as the domain adversarial training. (3) Since users cannot govern the data quality, the irrelevant or malicious source domains are more likely to appear, which causes negative transfer. To address the above problems, we propose a privacy-preserving UMDA paradigm named Knowledge Distillation based Decentralized Domain Adaptation (KD3A), which performs domain adaptation through the knowledge distillation on models from different source domains. The extensive experiments show that KD3A significantly outperforms state-of-the-art UMDA approaches. Moreover, the KD3A is robust to the negative transfer and brings a 100x reduction of communication cost compared with other decentralized UMDA methods.

Keywords: Algorithms · Multitask, Transfer, and Meta Learning
[ Slides] 

Author Information

Haozhe Feng (State Key Lab of CAD&CG, Zhejiang University)

I am studying for a Ph.D. in the State Key Lab of CAD&CG at Zhejiang University. I received my BS degree in Mathematics Statistics from Zhejiang University. My research interest includes machine learning / deep learning / probabilistic model / representation learning / federated learning. ## Education Zhejiang University Bachelor of Mathematics & Statistics • 2014 — 2018 Zhejiang University Ph.D. of Computer Science • 2018 — ## Projections and Publications (Selected) ### Federated Transfer Learning (2020.8 - Present) Publication: [FengHZ, You Z, et al. KD3A: Unsupervised Multi-Source Decentralized Domain Adaptation via Knowledge Distillation. (Accepted at ICML 2021).](https://arxiv.org/abs/2011.09757) ### Semi-supervised Representation Learning (2019.10-2020.6) Publication: [Feng HZ, Kong K, et al. SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations. (Accepted at AAAI 2021.)](https://arxiv.org/abs/2011.10684) ### Representation Learning for Basic Visualization Charts (2018.10-2019.7) Publication: Zhang T, Feng HZ, et al. ChartSeer: An Interactive Pattern Identification and Annotation Framework for Charts. (First co-author, Under Review of IEEE TKDE). ### Unsupervised Medical Image Generator Model Analysis (2018.3-2018.9) Publication: Feng HZ, Zhang P, Xu Xinnan, et al. An Unsupervised Suggestive Annotation Algorithm for 3D CT Image Processing [J]. Journal of CAD, 2019, 31(2): 183-189. (EI)

Zhaoyang You (Zhejiang University)
Minghao Chen (Zhejiang University)
Tianye Zhang (Zhejiang University)
Minfeng Zhu (State Key Lab of CAD&CG, Zhejiang University)
Fei Wu (Zhejiang University, China)
Chao Wu (Zhejiang University)
Wei Chen (State Key Lab of CAD&CG, Zhejiang University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors