Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models

Recent advances in vision-language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification in sequential tasks. However, assessing and improving calibration in such settings remains mostly unexplored, especially when only partial trajectories are observed. In this work, we formulate sequential calibration for episodic tasks, where task-success confidence is produced along an episode, while success is determined at the end of it. We introduce a sequential extension of the Brier score and show that, for binary outcomes, its risk minimizer coincides with the VLA policy’s value function. This connection bridges uncertainty calibration and reinforcement learning, enabling the use of temporal-difference (TD) value estimation as a principled calibration mechanism over time. We empirically show that TD calibration improves performance relative to the state-of-the-art on simulated and real-robot data. Interestingly, we show that when calibrated using TD, the VLA's single-step action probabilities can yield competitive uncertainty estimates, in contrast to recent findings that employed different calibration techniques.

Recent advances in vision-language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification and calibration in sequential tasks. However, assessing and improving calibration in such settings remains mostly unexplored, especially when only partial trajectories are observed. Our work provides the first formal framework for calibration in sequential tasks. We establish a novel connection between sequential calibration and RL value function prediction, enabling use of value estimation methods. We demonstrate that black-box calibration, using only action probabilities output by a VLA model, can match or outperform white-box methods, that require access to model's internal features, making it useful for proprietary models. The approach requires no VLA fine-tuning and operates as a lightweight MLP on top of the model's output. Finally, we show the added value of the calibrated success predictor both for early stopping and for improving the task success rates.