Timezone: »
Model fusion without accessing training data in machine learning has attracted increasing interest due to the practical resource-saving and data privacy issues. During the training process, the neural weights of each model can be randomly permuted, and we have to align the channels of each layer before fusing them. Regrading the channels as nodes and weights as edges, aligning the channels to maximize weight similarity is a challenging NP-hard assignment problem. Due to its quadratic assignment nature, we formulate the model fusion problem as a graph matching task, considering the second-order similarity of model weights instead of previous work merely formulating model fusion as a linear assignment problem. For the rising problem scale and multi-model consistency issues, we propose an efficient graduated assignment-based model fusion method, dubbed GAMF, which iteratively updates the matchings in a consistency-maintaining manner. We apply GAMF to tackle the compact model ensemble task and federated learning task on MNIST, CIFAR-10, CIFAR-100, and Tiny-Imagenet. The performance shows the efficacy of our GAMF compared to state-of-the-art baselines.
Author Information
Chang Liu (Shanghai Jiao Tong University)
Chenfei Lou (Shanghai Jiao Tong University)
Runzhong Wang (Shanghai Jiao Tong University)
Alan Yuhan Xi (university of wisconsin madison)
Li Shen (JD Explore Academy)
Junchi Yan (Shanghai Jiao Tong University)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Spotlight: Deep Neural Network Fusion via Graph Matching with Applications to Model Ensemble and Federated Learning »
Tue. Jul 19th 06:25 -- 06:30 PM Room Hall F
More from the Same Authors
-
2023 : Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning »
Guozheng Ma · · Haoyu Wang · Lu Li · Zilin Wang · Zhen Wang · Li Shen · Xueqian Wang · Dacheng Tao -
2023 Oral: Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape »
Yan Sun · Li Shen · Shixiang Chen · Liang Ding · Dacheng Tao -
2023 Poster: Towards Quantum Machine Learning for Constrained Combinatorial Optimization: a Quantum QAP Solver »
Xinyu Ye · Ge Yan · Junchi Yan -
2023 Poster: Patch-level Contrastive Learning via Positional Query for Visual Pre-training »
Shaofeng Zhang · Qiang Zhou · Zhibin Wang · Fan Wang · Junchi Yan -
2023 Poster: Quantum 3D Graph Learning with Applications to Molecule Embedding »
Ge Yan · Huaijin Wu · Junchi Yan -
2023 Poster: QAS-Bench: Rethinking Quantum Architecture Search and A Benchmark »
Xudong Lu · Kaisen Pan · Ge Yan · Jiaming Shan · Wenjie Wu · Junchi Yan -
2023 Poster: Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective »
Liangliang Shi · Gu Zhang · Haoyu Zhen · Jintao Fan · Junchi Yan -
2023 Poster: Are Large Kernels Better Teachers than Transformers for ConvNets? »
Tianjin Huang · Lu Yin · Zhenyu Zhang · Li Shen · Meng Fang · Mykola Pechenizkiy · Zhangyang “Atlas” Wang · Shiwei Liu -
2023 Poster: LinSATNet: The Positive Linear Satisfiability Neural Networks »
Runzhong Wang · Yunhao Zhang · Ziao Guo · Tianyi Chen · Xiaokang Yang · Junchi Yan -
2023 Poster: Improving the Model Consistency of Decentralized Federated Learning »
Yifan Shi · Li Shen · Kang Wei · Yan Sun · Bo Yuan · Xueqian Wang · Dacheng Tao -
2023 Poster: QuantumDARTS: Differentiable Quantum Architecture Search for Variational Quantum Algorithms »
Wenjie Wu · Ge Yan · Xudong Lu · Kaisen Pan · Junchi Yan -
2023 Poster: Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape »
Yan Sun · Li Shen · Shixiang Chen · Liang Ding · Dacheng Tao -
2023 Poster: Learning to Learn from APIs: Black-Box Data-Free Meta-Learning »
Zixuan Hu · Li Shen · Zhenyi Wang · Baoyuan Wu · Chun Yuan · Dacheng Tao -
2023 Poster: CoCo: A Coupled Contrastive Framework for Unsupervised Domain Adaptive Graph Classification »
Nan Yin · Li Shen · Mengzhu Wang · Long Lan · Zeyu Ma · Chong Chen · Xian-Sheng Hua · Xiao Luo -
2022 : Paper 12: SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Driving »
· Li Shen · Bo Yuan · Xueqian Wang -
2022 Poster: Understanding Robust Overfitting of Adversarial Training and Beyond »
Chaojian Yu · Bo Han · Li Shen · Jun Yu · Chen Gong · Mingming Gong · Tongliang Liu -
2022 Poster: DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training »
Rong Dai · Li Shen · Fengxiang He · Xinmei Tian · Dacheng Tao -
2022 Spotlight: Understanding Robust Overfitting of Adversarial Training and Beyond »
Chaojian Yu · Bo Han · Li Shen · Jun Yu · Chen Gong · Mingming Gong · Tongliang Liu -
2022 Spotlight: DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training »
Rong Dai · Li Shen · Fengxiang He · Xinmei Tian · Dacheng Tao -
2022 Poster: On Collective Robustness of Bagging Against Data Poisoning »
Ruoxin Chen · Zenan Li · Jie Li · Junchi Yan · Chentao Wu -
2022 Poster: Improving Task-free Continual Learning by Distributionally Robust Memory Evolution »
Zhenyi Wang · Li Shen · Le Fang · Qiuling Suo · Tiehang Duan · Mingchen Gao -
2022 Poster: GNNRank: Learning Global Rankings from Pairwise Comparisons via Directed Graph Neural Networks »
Yixuan He · Quan Gan · David Wipf · Gesine Reinert · Junchi Yan · Mihai Cucuringu -
2022 Spotlight: GNNRank: Learning Global Rankings from Pairwise Comparisons via Directed Graph Neural Networks »
Yixuan He · Quan Gan · David Wipf · Gesine Reinert · Junchi Yan · Mihai Cucuringu -
2022 Spotlight: On Collective Robustness of Bagging Against Data Poisoning »
Ruoxin Chen · Zenan Li · Jie Li · Junchi Yan · Chentao Wu -
2022 Spotlight: Improving Task-free Continual Learning by Distributionally Robust Memory Evolution »
Zhenyi Wang · Li Shen · Le Fang · Qiuling Suo · Tiehang Duan · Mingchen Gao -
2021 Poster: Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach »
Qitian Wu · Hengrui Zhang · Xiaofeng Gao · Junchi Yan · Hongyuan Zha -
2021 Poster: Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation »
Chao Chen · Haoyu Geng · Nianzu Yang · Junchi Yan · Daiyue Xue · Jianping Yu · Xiaokang Yang -
2021 Spotlight: Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach »
Qitian Wu · Hengrui Zhang · Xiaofeng Gao · Junchi Yan · Hongyuan Zha -
2021 Spotlight: Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation »
Chao Chen · Haoyu Geng · Nianzu Yang · Junchi Yan · Daiyue Xue · Jianping Yu · Xiaokang Yang -
2021 Poster: Deep Latent Graph Matching »
Tianshu Yu · Runzhong Wang · Junchi Yan · baoxin Li -
2021 Spotlight: Deep Latent Graph Matching »
Tianshu Yu · Runzhong Wang · Junchi Yan · baoxin Li -
2021 Poster: Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss »
Xue Yang · Junchi Yan · Qi Ming · Wentao Wang · xiaopeng zhang · Qi Tian -
2021 Spotlight: Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss »
Xue Yang · Junchi Yan · Qi Ming · Wentao Wang · xiaopeng zhang · Qi Tian -
2020 Poster: Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks »
Zhishuai Guo · Mingrui Liu · Zhuoning Yuan · Li Shen · Wei Liu · Tianbao Yang -
2018 Poster: An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method »
Li Shen · Peng Sun · Yitong Wang · Wei Liu · Tong Zhang -
2018 Oral: An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method »
Li Shen · Peng Sun · Yitong Wang · Wei Liu · Tong Zhang -
2017 Poster: GSOS: Gauss-Seidel Operator Splitting Algorithm for Multi-Term Nonsmooth Convex Composite Optimization »
Li Shen · Wei Liu · Ganzhao Yuan · Shiqian Ma -
2017 Talk: GSOS: Gauss-Seidel Operator Splitting Algorithm for Multi-Term Nonsmooth Convex Composite Optimization »
Li Shen · Wei Liu · Ganzhao Yuan · Shiqian Ma