Poster
Lossless or Quantized Boosting with Integer Arithmetic
Richard Nock · Robert C Williamson

Wed Jun 12th 06:30 -- 09:00 PM @ Pacific Ballroom #194
in Posters Wed »

In supervised learning, efficiency often starts with the choice of a good loss: support vector machines popularised Hinge loss, Adaboost popularised the exponential loss, etc. Recent trends in machine learning have highlighted the necessity for training routines to meet tight requirements on communication, bandwidth, energy, operations, encoding, among others. Fitting the often decades-old state of the art training routines into these new constraints does not go without pain and uncertainty or reduction in the original guarantees. Our paper starts with the design of a new strictly proper canonical, twice differentiable loss called the Q-loss. Importantly, its mirror update over (arbitrary) rational inputs uses only integer arithmetics -- more precisely, the sole use of $+, -, /, \times, |.|$. We build a learning algorithm which is able, under mild assumptions, to achieve a lossless boosting-compliant training. We give conditions for a quantization of its main memory footprint, weights, to be done while keeping the whole algorithm boosting-compliant. Experiments display that the algorithm can achieve a fast convergence during the early boosting rounds compared to AdaBoost, even with a weight storage that can be 30+ times smaller. Lastly, we show that the Bayes risk of the Q-loss can be used as node splitting criterion for decision trees and guarantees optimal boosting convergence.

Keywords: Convex Optimization · Ensemble Methods · Statistical Learning Theory

Author Information

Richard Nock (Data61, The Australian National University and the University of Sydney)
Bob C Williamson (ANU)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors