Magnitude Distance: A Geometric Measure of Dataset Similarity

Quantifying the distance between datasets is a fundamental question in mathematics and machine learning. We propose *magnitude distance*, a novel distance metric defined on finite datasets using the notion of the *magnitude* of a metric space. The proposed distance incorporates a tunable scaling parameter, $t$, that controls the sensitivity to global structure (small $t$) and finer details (large $t$). We prove several theoretical properties of magnitude distance, including its limiting behavior across scales and conditions under which it satisfies key metric properties. In contrast to classical distances, we show that magnitude distance remains discriminative in high-dimensional settings when the scale is appropriately tuned. We further demonstrate how magnitude distance can be used as a training objective for push-forward generative models. Our experimental results support our theoretical analysis and demonstrate that magnitude distance provides meaningful signals, comparable to established distance-based generative approaches.

We introduce a new method to measure the distance and similarity between two sets of data points. To do this, we use a mathematical tool called "Metric Magnitude," which can naturally capture the geometric structure of data across different scales. Intuitively, Magnitude can zoom in or out of the data space: a large zoom focuses more on fine local details, while a small zoom captures the global structure. The distance defined using magnitude can be correspondingly scaled for a local or global view. We looked at fundamental properties of this distance, including its behavior in high-dimensional settings (where data has thousands of features). While standard distances tend to lose their ability to distinguish differences in high dimensions, magnitude distance remains highly effective. Also, we show that under mild assumptions, our measure is robust to outliers: adding or removing a datapoint does not cause extreme changes. We also explain how magnitude distance behaves across different scaling regimes. As a proof of concept, we used the distance aggregated across different scales as a core guiding signal (the loss function) to train a new generative AI model called MagGN. By learning to generate new images with metric magnitude values and geometric properties similar to those of the training points, MagGN shows strong performance—in some cases, better than existing methods.