Timezone: »

 
Poster
Model Fusion with Kullback--Leibler Divergence
Sebastian Claici · Mikhail Yurochkin · Soumya Ghosh · Justin Solomon

Thu Jul 16 08:00 AM -- 08:45 AM & Thu Jul 16 08:00 PM -- 08:45 PM (PDT) @
in Poster Session 43 »

We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors and proceeds using a simple assign-and-average approach. The components of the dataset posteriors are assigned to the proposed global model components by solving a regularized variant of the assignment problem. The global components are then updated based on these assignments by their mean under a KL divergence. For exponential family variational distributions, our formulation leads to an efficient non-parametric algorithm for computing the fused model. Our algorithm is easy to describe and implement, efficient, and competitive with state-of-the-art on motion capture analysis, topic modeling, and federated learning of Bayesian neural networks.

Keywords: Approximate Inference · Bayesian Methods · Parallel and Distributed Learning · Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods

Author Information

Sebastian Claici (MIT)
Mikhail Yurochkin (IBM Research AI)
Soumya Ghosh (IBM Research)
Justin Solomon (MIT)

More from the Same Authors