Trust Region Newton Metho ds for Large-Scale Logistic Regression

Trust Region Newton Metho ds for Large-Scale Logistic Regression
Chih-Jen Lin - National Taiwan University, Taiwan Ruby Chiu-Hsing Weng - National Chengche University, Taiwan Sathiya Keerthi - Yahoo Research, USA
Large-scale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the log-likelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also compare it with linear SVM implementations.

Chih-Jen Lin - National Taiwan University, Taiwan
Ruby Chiu-Hsing Weng - National Chengche University, Taiwan
Sathiya Keerthi - Yahoo Research, USA

Large-scale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the log-likelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also compare it with linear SVM implementations.