Likelihood over Estimation: Robust Quadratic Discriminant Analysis for Heavy-Tailed Distributions with Theory and Evidence

Quadratic Discriminant Analysis (QDA) assumes Gaussian class-conditional distributions, causing systematic misclassification when data exhibit heavy tails. We propose Stable-QDA, which replaces the Gaussian likelihood with a symmetric $\alpha$-stable likelihood that decays polynomially rather than exponentially in Mahalanobis distance. Crucially, we find that correcting likelihood misspecification yields larger gains than robustifying parameter estimation: standard estimators (sample mean, Ledoit--Wolf covariance) often outperform robust alternatives when class heteroscedasticity is discriminative. We provide consistency guarantees under infinite-variance regimes, data-driven diagnostics for estimator selection, and demonstrate 15--53\% error reduction on real-world heavy-tailed benchmarks.

Machine learning classifiers like Quadratic Discriminant Analysis (QDA) assume your data looks like a bell curve (Gaussian). But real-world data — fraud transactions, pulsar signals, network intrusions — has heavy tails: extreme outliers are far more common than a bell curve predicts. When you apply classical QDA to this kind of data, it fails to classify outliers correctly. What's the core insight? There are two ways the existing QDA can go wrong with heavy-tailed data: 1. Wrong model — the Gaussian likelihood underweights outliers, so the classifier is systematically wrong even if you had perfect knowledge of the parameters. 2. Wrong estimation — the standard covariance matrix estimator becomes unreliable (effectively blows up) when data has infinite variance. The key finding is counterintuitive: fixing the likelihood model (problem #1) matters more than fixing the estimator (problem #2). Stable-QDA replaces the Gaussian likelihood with a multivariate elliptical $\alpha$-stable distribution , which naturally handles heavy tails — and this drives most of the improvement. What is an $\alpha$-stable distribution? Think of it as a generalization of the bell curve. Just as the Central Limit Theorem says averages of many random variables converge to a Gaussian, the Generalized CLT says that sums of heavy-tailed variables converge to an $\alpha$-stable distribution. So $\alpha$-stable is the "right" model for heavy-tailed phenomena by the same foundational logic that makes Gaussian the right model for ordinary data. What did the paper prove? Stable-QDA is Bayes consistent — given enough data, it converges to the theoretically best possible classifier. It works very well even if you don't know the exact tail heaviness ($\alpha$= 1.5 works universally as a default). Classical QDA under heavy tails converges to the wrong answer permanently, no matter how much data you have. To make this work accessible and reproducible, we've released the full implementation of Stable-QDA along with all experimental code on GitHub. This includes the classifier itself and scripts to reproduce every result in the paper — so researchers can build on this work or apply Stable-QDA directly to their own heavy-tailed classification problems.