Timezone: »

 
Neural Network-based Estimation of the MMSE
Mario Diaz · Peter Kairouz · Lalitha Sankar

@
in Information-Theoretic Methods for Rigorous, Responsible, and Reliable Machine Learning (ITR3) »
The minimum mean-square error (MMSE) achievable by optimal estimation of a random variable $S$ given another random variable $T$ is of much interest in a variety of statistical contexts. Motivated by a growing interest in auditing machine learning models for unintended information leakage, we propose a neural network-based estimator of this MMSE. We derive a lower bound for the MMSE based on the proposed estimator and the Barron constant associated with the conditional expectation of $S$ given $T$. Since the latter is typically unknown in practice, we derive a general bound for the Barron constant that produces order optimal estimates for canonical distribution models.

Author Information

Mario Diaz (Universidad Nacional Autónoma de México)
Peter Kairouz (Google)
Lalitha Sankar (Arizona State University)

More from the Same Authors