Timezone: »

 
Oral
Minimal Achievable Sufficient Statistic Learning
Milan Cvitkovic · Günther Koliander

Wed Jun 12 04:30 PM -- 04:35 PM (PDT) @ Grand Ballroom
in Representation Learning »

We introduce Minimal Achievable Sufficient Statistic (MASS) Learning, a training objective for machine learning models whose minimizers are minimal sufficient statistics with respect to the class of functions being optimized over (e.g. deep networks). In deriving MASS Learning, we also introduce Conserved Differential Information (CDI), an information-theoretic quantity that - unlike standard mutual information - can be usefully applied to deterministically-dependent continuous random variables like the input and output of a deep network. In a series of experiments, we show that deep networks trained with MASS Learning match state–of–the–art performance on supervised learning, uncertainty quantification, and adversarial robustness benchmarks.

[ Slides]  [ Video] 

Author Information

Milan Cvitkovic (California Institute of Technology)
Günther Koliander (Austrian Academy of Sciences)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors