Mixtures Closest To A Given Measure: A Semidefinite Programming Approach

Mixture models, such as Gaussian mixture models (GMMs), are widely used in machine learning to represent complex data distributions. A key challenge, especially in high-dimensional settings, is to determine the mixture order and estimate the mixture parameters. We study the problem of approximating a target measure, available only through finitely many of its moments, by a mixture of distributions from a parametric family (e.g., Gaussian, exponential, Poisson), with approximation quality measured by the 2-Wasserstein ($\operatorname{W_2}$) or the total variation ($\operatorname{TV}$) distance. Unlike many existing approaches, the parameter set is not assumed to be finite; it is modeled as a compact basic semi-algebraic set. We introduce a hierarchy of semidefinite relaxations with asymptotic convergence to the desired optimal value. In addition, when a certain rank condition is satisfied, the convergence is even finite and recovery of an optimal mixing measure is obtained. We also present an application to clustering, where our framework serves either as a stand-alone method or as a preprocessing step that yields both the number of clusters and strong initial parameter estimates, thereby accelerating convergence of standard (local) clustering algorithms

Many real-world datasets are made of several hidden groups: for example, a collection of images, measurements, or signals may come from several different sources. A common way to model such data is to fit a mixture of simple probability distributions, but two hard questions are how many components to use and what their parameters should be. Existing methods often rely on local search or on first choosing a fixed number of components, which can make the result sensitive to initialization. We propose a different approach: instead of directly guessing the components, we use a small set of statistical summaries of the data, called moments, and formulate the search for the closest mixture as an optimization problem that can be solved systematically. Our method can compare distributions using either Wasserstein distance, which measures how much probability mass must be moved, or total variation distance, which measures how different two distributions are overall. Under suitable mathematical conditions, the resulting sequence of optimization problems converges to the best mixture, and in some cases the mixture can be recovered exactly. In clustering experiments, the method can estimate the number of clusters and provide strong initial centers for standard algorithms such as k-means and EM. This makes mixture learning more principled and less dependent on fragile initialization choices.