Rethinking Calibration for Early-Exit Neural Networks

Early-exit neural networks~(EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enough. Most methods rely on confidence thresholds for exiting, and consequently, classifier calibration is widely assumed to improve performance. In this work, we challenge this assumption and show that calibration is often not suitable for EENNs through a detailed theoretical study. To address the limitations of calibration, we introduce Early-Exit Failure Prediction~(EEFP), which accounts for both prediction correctness and the cost of further computation. We also propose a lightweight, EEFP-motivated procedure to improve the intermediate classifiers, which can directly replace calibration in EENNs. Extensive experiments demonstrate that our approach achieves superior cost–accuracy trade-offs than calibration and discuss how EEFP measures more reliably reflect overall EENN performance.