Tail Annealing for Heavy-Tailed Flow Matching

Standard generative models struggle with heavy-tailed data: Lipschitz architectures cannot produce power-law tails from Gaussian noise, and interpolating between heavy-tailed data and Gaussians is ill-posed. We propose a simple fix: apply the soft-log transform $\phi(x) = \mathrm{sign}(x) \cdot \log(1 + |x|)$ coordinate-wise to data before training, then exponentiate samples after generation. A Hill diagnostic decides per-coordinate whether to transform, leaving light-tailed margins untouched at no added complexity. This compresses heavy tails into a range where standard flow matching succeeds, without heavy-tailed base distributions or architectural modifications. We provide theoretical intuition for why this works: the log-transform maps Pareto tails to exponentials, and the induced dynamics implement a form of tail annealing via power transformations. On a 144-configuration multivariate benchmark (3 copulas, $d$ up to 100, 4 tail indices), Log-FM dominates specialized baselines on $W_1$, CVaR$_{99}$, and extreme-quantile metrics, and is the only method with zero severe divergences across 2{,}880 runs.

Heavy-tailed data, where extreme events like financial crashes or massive insurance claims happen far more often than the bell curve predicts, are not taken into account by modern generative AI models. These models are built from smooth components that mathematically can't stretch standard noise into the rare and huge values such data contains, and forcing them to try tends to break. We propose a simple fix: before training, squeeze the data through a logarithm-like transform that compresses huge values, then reverse it after generating samples. A quick statistical check applies this only where needed, leaving well-behaved variables alone. This compresses the extremes into a range standard models handle well, with no exotic architectures or special noise sources. Mathematically, the transform turns the worst tails into milder ones, and generation gradually warms up to the true heaviness rather than all at once. Across 144 configurations spanning different dependency structures, up to 100 dimensions, and varying tail heaviness, our method beats specialized competitors on every measure of extreme-event accuracy. It was also the only one that never catastrophically failed across nearly 3,000 runs.