Exponential Family Discriminant Analysis: Generalizing LDA-Style Generative Classification to Non-Gaussian Models

We introduce Exponential Family Discriminant Analysis (EFDA), a unified framework that extends classical Linear Discriminant Analysis (LDA) generative classification beyond the Gaussian setting to any member of the exponential family. Under the assumption that each class–conditional density belongs to the same exponential family, EFDA derives closed-form or semi-closed-form maximum likelihood estimators for all natural parameters, and yields an explicit expression for the log-odds ratio as an linear function of the sufficient statistic. We demonstrate the method on the Weibull distribution, showing that EFDA accurately models the nonlinear log-odds ratio, which logistic regression and LDA are unable to capture. Finally, to demonstrate EFDA's practicality, we provide closed-form EFDA derivations for four additional exponential family distributions.