Timezone: »

Online Platt Scaling with Calibeating
Chirag Gupta · Aaditya Ramdas

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #611

We present an online post-hoc calibration method, called Online Platt Scaling (OPS), which combines the Platt scaling technique with online logistic regression. We demonstrate that OPS smoothly adapts between i.i.d. and non-i.i.d. settings with distribution drift. Further, in scenarios where the best Platt scaling model is itself miscalibrated, we enhance OPS by incorporating a recently developed technique called calibeating to make it more robust. Theoretically, our resulting OPS+calibeating method is guaranteed to be calibrated for adversarial outcome sequences. Empirically, it is effective on a range of synthetic and real-world datasets, with and without distribution drifts, achieving superior performance without hyperparameter tuning. Finally, we extend all OPS ideas to the beta scaling method.

Author Information

Chirag Gupta (Carnegie Mellon University)
Aaditya Ramdas (Carnegie Mellon University)

More from the Same Authors