Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

Learning conditional distributions $\pi^\star(\cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) \sim \pi^\star$. However, acquiring paired data samples is often challenging, especially in problems such as domain translation. This necessitates the development of *semi-supervised* models that utilize both limited paired data and additional unpaired i.i.d. samples $x \sim \pi^\star_x$ and $y \sim \pi^\star_y$ from the marginal distributions. The usage of such combined data is complex and often relies on heuristic approaches. To tackle this issue, we propose a new learning paradigm that integrates both paired and unpaired data seamlessly using data likelihood maximization techniques. We demonstrate that our approach also connects intriguingly with inverse entropic optimal transport (OT). This finding allows us to apply recent advances in computational OT to establish an *end-to-end* learning algorithm to get $\pi^\star(\cdot|x)$. In addition, we derive the universal approximation property, demonstrating that our approach can theoretically recover true conditional distributions with arbitrarily small error. Finally, we demonstrate through empirical tests that our method effectively learns conditional distributions using paired and unpaired data simultaneously.

Machine learning systems often need to learn relationships between two types of data. For example, a system may need to translate images between styles, predict weather conditions from sensor measurements, or generate outputs based on limited examples. Most existing methods require large collections of perfectly matched input-output pairs for training. However, in many real-world applications, such paired data is expensive or difficult to obtain, while separate collections of inputs and outputs are much easier to collect. In this work, we introduce a new method that can learn from both a small amount of paired data and a large amount of unpaired data at the same time. Our approach is based on maximizing how well the model explains the observed data, while also drawing inspiration from optimal transport, a mathematical framework for comparing and transforming probability distributions. We show theoretically that our method can recover complex relationships between domains and provide an efficient algorithm for training it in practice. In experiments on synthetic tasks, weather prediction, image translation, and classification, our approach consistently outperforms existing semi-supervised methods, especially when only a limited number of paired examples are available. For example, our method can learn realistic image translations and accurate probabilistic weather forecasts while requiring substantially less paired supervision than standard approaches. Our findings suggest that combining paired and unpaired data through principled likelihood-based learning can significantly improve data efficiency and reliability in machine learning systems.