Pairwise Maximum Likelihood For Multi-Class Logistic Regression Model With Multiple Rare Classes

We study in this work the problem of multi-class logistic regression with one major class and multiple rare classes, which is motivated by a real application in TikTok live stream data. The model is inspired by the two-class logistic regression model of Wang (2020) but with surprising theoretical findings, which in turn motivate new estimation methods with excellent statistical and computational efficiency. Specifically, since rigorous theoretical analysis suggests that the resulting maximum likelihood estimators of different rare classes should be asymptotically independent, we consider to solve multiple pairwise two-class logistic regression problems instead of optimizing the joint log-likelihood function with computational challenge in multi-class problem, which are computationally much easier and can be conducted in a fully parallel way. To further reduce the computation cost, a subsample-based pairwise likelihood estimator is developed by down-sampling the major class. We show rigorously that the resulting estimators could be as asymptotically efficient as the global maximum likelihood estimator under appropriate regularity conditions. Extensive simulation studies are presented to support our theoretical findings and a TikTok live stream dataset is analyzed for illustration purpose.

In real-world applications like car plate recognition in TikTok live streams, the multi-class classification task often faces the challenge of class imbalance, where multiple rare classes (e.g., unique license plates) are vastly outnumbered by a major class (background). We propose a new method based on logistic regression. Specifically, the multi-class problem is decomposed into multiple independent binary subproblems, where each rare class is paired separately with the major class. Theoretical analysis shows that this decomposition maintains classification validity while enabling independent optimization for rare classes. To further reduce the computational cost, we reduce the size of the major class through random subsampling. By enabling parallel training, this method offers an efficient and effective solution to imbalanced classification.