Phase-Type Variational Autoencoders for Heavy-Tailed Data

Heavy-tailed distributions are ubiquitous in real-world data, where rare but extreme events dominate risk and variability. However, standard Variational Autoencoders (VAEs) employ simple decoder distributions (e.g., Gaussian) that fail to capture heavy-tailed behavior, while existing heavy-tail-aware extensions remain restricted to predefined parametric families whose tail behavior is fixed a priori. We propose the Phase-Type Variational Autoencoder (PH-VAE), whose decoder distribution is a latent-conditioned Phase-Type (PH) distribution—defined as the absorption time of a continuous-time Markov chain (CTMC). This formulation composes multiple exponential time scales, yielding a flexible, analytically tractable decoder that adapts its finite-range tail behavior directly from the observed data. Experiments on synthetic and real-world benchmarks demonstrate that PH-VAE accurately approximates diverse heavy-tailed distributions, significantly outperforming Gaussian, Student-t, and extreme-value-based VAE decoders in modeling observed tail behavior and extreme quantiles. In multivariate settings, PH-VAE captures realistic cross-dimensional tail dependence through its shared latent representation. To our knowledge, this is the first work to integrate Phase-Type distributions into deep generative modeling, bridging applied probability and representation learning.

Rare and extreme events play an important role in many real-world systems, from financial crashes and insurance losses to internet traffic and word usage patterns. However, many modern AI generative models struggle to accurately represent these rare events because they are designed under simpler statistical assumptions that underestimate extreme behavior. In this work, we introduce a new generative AI model called the Phase-Type Variational Autoencoder (PH-VAE). Instead of relying on standard assumptions about data distributions, our model learns more flexible patterns that can better represent both common observations and rare extreme events. The method is inspired by stochastic processes and Markov chains, mathematical tools used to model evolving systems under uncertainty, allowing the model to adapt flexibly to different data behaviors. We evaluate the proposed approach on both synthetic and real-world datasets involving heavy-tailed phenomena, including financial data, insurance losses, and word-frequency statistics. Our experiments show that PH-VAE captures extreme events and complex dependencies more accurately than existing VAE-based methods while remaining computationally efficient. This work provides a new bridge between probabilistic modeling and deep generative AI, opening the door to more reliable machine learning systems for domains where rare events matter most.