Sharp Inequalities between Total Variation and Hellinger Distances for Gaussian Mixtures

We study the relation between the total variation (TV) and Hellinger distances between two Gaussian location mixtures. Our first result establishes a general upper bound: for any two mixing distributions supported on a compact set, the Hellinger distance between the two mixtures is controlled by the TV distance raised to a power $1-o(1)$, where the $o(1)$ term is of order $1/\log\log(1/\mathrm{TV})$. We also construct two sequences of mixing distributions that demonstrate the sharpness of this bound. Taken together, our results resolve an open problem raised in Jia et al. (2023) and thus lead to an entropic characterization of learning Gaussian mixtures in total variation. Our inequality also yields optimal robust estimation of Gaussian mixtures in Hellinger distance, which has a direct implication for bounding the minimax regret of empirical Bayes under Huber contamination.

Many machine learning methods rely on comparing probability distributions, but different ways of measuring how different two distributions are can behave very differently. In this paper, we study two widely used measures — total variation distance and Hellinger distance — for Gaussian mixture models, which are fundamental tools in statistics, clustering, and Bayesian learning. We prove a sharp mathematical relationship between these two distances. We show that if two Gaussian mixtures are close in total variation distance, then they must also be close in Hellinger distance, up to a correction term that is essentially the best possible. This resolves an open problem posed in earlier work. Beyond the theoretical result itself, our work has practical implications for robust machine learning. In settings where data may contain corrupted observations or outliers, our inequality leads to optimal guarantees for estimating Gaussian mixtures under contamination. These results also improve the theoretical understanding of empirical Bayes methods, which are widely used for large-scale statistical inference and denoising problems. Overall, the paper strengthens the theoretical foundations of robust learning and statistical estimation for Gaussian mixture models.