Timezone: »
On the Calibration of Learning to Defer to Multiple Experts
Rajeev Verma
We study the calibration properties of multi-expert learning to defer (L2D). In particular, we study the framework's ability to estimate $\mathbb{P}(\rsm_{j} = \ry | \vx)$, the probability that the $j$th expert will correctly predict the label for $\vx$. We compare softmax- and one-vs-all-parameterized L2D, finding the former causes mis-calibration to propagate between the estimates of expert correctness while the latter's parameterization does not.
Author Information
Rajeev Verma (University of Amsterdam)
More from the Same Authors
-
2022 Poster: Calibrated Learning to Defer with One-vs-All Classifiers »
Rajeev Verma · Eric Nalisnick -
2022 Spotlight: Calibrated Learning to Defer with One-vs-All Classifiers »
Rajeev Verma · Eric Nalisnick