Timezone: »

The Statistical Scope of Multicalibration
Georgy Noarov · Aaron Roth

Wed Jul 26 02:00 PM -- 03:30 PM (PDT) @ Exhibit Hall 1 #738
We make a connection between multicalibration and property elicitation and show that (under mild technical conditions) it is possible to produce a multicalibrated predictor for a continuous scalar property $\Gamma$ if and only if $\Gamma$ is *elicitable*. On the negative side, we show that for non-elicitable continuous properties there exist simple data distributions on which even the true distributional predictor is not calibrated. On the positive side, for elicitable $\Gamma$, we give simple canonical algorithms for the batch and the online adversarial setting, that learn a $\Gamma$-multicalibrated predictor. This generalizes past work on multicalibrated means and quantiles, and in fact strengthens existing online quantile multicalibration results. To further counter-weigh our negative result, we show that if a property $\Gamma^1$ is not elicitable by itself, but *is* elicitable *conditionally* on another elicitable property $\Gamma^0$, then there is a canonical algorithm that *jointly* multicalibrates $\Gamma^1$ and $\Gamma^0$; this generalizes past work on mean-moment multicalibration. Finally, as applications of our theory, we provide novel algorithmic and impossibility results for fair (multicalibrated) risk assessment.

Author Information

Georgy Noarov (University of Pennsylvania)

4th year PhD student at the University of Pennsylvania advised by Aaron Roth and Michael Kearns. Hold a Bachelor's in Mathematics from Princeton University. Work on Machine Learning topics including uncertainty quantification, fairness, online learning, and game theory. Recent works on fair and robust uncertainty quantification include: - The Statistical Scope of Multicalibration [ICML'23] - Batch Multivalid Conformal Prediction [ICLR'23] - Practical Adversarial Multivalid Conformal Prediction [NeurIPS'22, Oral] - Online Minimax Multiobjective Optimization [NeurIPS'22, Oral]

Aaron Roth (University of Pennsylvania)

More from the Same Authors