Timezone: »

 
Poster
The Statistical Scope of Multicalibration
Georgy Noarov · Aaron Roth

Wed Jul 26 02:00 PM -- 03:30 PM (PDT) @ Exhibit Hall 1 #738
in Poster Session 3 »
We make a connection between multicalibration and property elicitation and show that (under mild technical conditions) it is possible to produce a multicalibrated predictor for a continuous scalar property $\Gamma$ if and only if $\Gamma$ is *elicitable*. On the negative side, we show that for non-elicitable continuous properties there exist simple data distributions on which even the true distributional predictor is not calibrated. On the positive side, for elicitable $\Gamma$, we give simple canonical algorithms for the batch and the online adversarial setting, that learn a $\Gamma$-multicalibrated predictor. This generalizes past work on multicalibrated means and quantiles, and in fact strengthens existing online quantile multicalibration results. To further counter-weigh our negative result, we show that if a property $\Gamma^1$ is not elicitable by itself, but *is* elicitable *conditionally* on another elicitable property $\Gamma^0$, then there is a canonical algorithm that *jointly* multicalibrates $\Gamma^1$ and $\Gamma^0$; this generalizes past work on mean-moment multicalibration. Finally, as applications of our theory, we provide novel algorithmic and impossibility results for fair (multicalibrated) risk assessment.
[ PDF] 

Author Information

Georgy Noarov (University of Pennsylvania)

4th year PhD student at the University of Pennsylvania advised by Aaron Roth and Michael Kearns. Hold a Bachelor's in Mathematics from Princeton University. Work on Machine Learning topics including uncertainty quantification, fairness, online learning, and game theory. Recent works on fair and robust uncertainty quantification include: - The Statistical Scope of Multicalibration [ICML'23] - Batch Multivalid Conformal Prediction [ICLR'23] - Practical Adversarial Multivalid Conformal Prediction [NeurIPS'22, Oral] - Online Minimax Multiobjective Optimization [NeurIPS'22, Oral]

Aaron Roth (University of Pennsylvania)

More from the Same Authors