Cooperative Variance Estimation and Bayesian Neural Networks for Disentangling Aleatoric and Epistemic Uncertainties

Real-world data contains aleatoric uncertainty - irreducible noise arising from imperfect measurements or from incomplete knowledge about the data generation process. Mean-variance estimation networks can learn this type of uncertainty but require ad-hoc regularization strategies to avoid overfitting and are unable to predict epistemic uncertainty (model uncertainty). Conversely, Bayesian neural networks predict epistemic uncertainty but are notoriously difficult to train due to the approximate nature of Bayesian inference. We propose to cooperatively train a variance estimation network with a Bayesian neural network and empirically demonstrate that the resulting model disentangles aleatoric and epistemic uncertainties while improving the mean estimation. We demonstrate the effectiveness and scalability of this method across a diverse range of datasets, including a time-dependent heteroscedastic regression dataset we created where the aleatoric uncertainty is known. The proposed method is straightforward to implement, robust, and adaptable to various model architectures.

Neural networks are usually trained for deterministic predictions. Methods that do quantify uncertainty conflate two distinct sources: the model's ignorance where data is sparse (epistemic), and the irreducible noise in the data-generating process (aleatoric). Disentangling them is essential, epistemic uncertainty tells us where to acquire more data; aleatoric uncertainty tells us what variability we must design around. Yet recent benchmarks show that no existing method reliably disentangles the two in practice. This paper presents Variance estimation Bayesian Neural Networks (VeBNN), which resolve this through a three-step cooperative training strategy: a deterministic mean network, a variance network trained on residuals via a Gamma likelihood with well-behaved gradients, and Bayesian inference with the aleatoric variance held fixed. The method is architecture-agnostic and demonstrated across UCI regression, ResNet-34 image regression under distribution shift, and a new plasticity constitutive model learning problem — possibly the first open dataset with known ground-truth aleatoric uncertainty.