Multiobjective distribution matching

Distribution matching is a key technique in machine learning, with applications in generative models, domain adaptation, and algorithmic fairness. A related but less explored challenge is generating a distribution that aligns with multiple underlying distributions, often with conflicting objectives, known as a Pareto optimal distribution.In this paper, we develop a general theory based on information geometry to construct the Pareto set and front for the entire exponential family under KL and inverse KL divergences. This formulation allows explicit derivation of the Pareto set and front for multivariate normal distributions, enabling applications like multiobjective variational autoencoders (MOVAEs) to generate interpolated image distributions.Experimental results on real-world images demonstrate that both algorithms can generate high-quality interpolated images across multiple distributions.

We study how to generate a distribution, which resembles to multiple distributions at the same time in this paper. We formulate this problem as an optimization problem defined in the dually flat manifold. In this way, we could derive the explicit formulation for the PS for the exponational family. Multiobjective distribution matching is a fundamental research problem in machine learning, which is highly related with group DRO, algorithmic fairness, and generative models.