Timezone: »

 
Morse Neural Networks for Uncertainty Quantification
Benoit Dherin · Huiyi Hu · JIE REN · Michael Dusenberry · Balaji Lakshminarayanan

@
in Structured Probabilistic Inference and Generative Modeling »
Event URL: https://openreview.net/forum?id=qJtr9ThuVs »

We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points. Fitting the Morse neural network via a KL-divergence loss yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a calibration temperature, 4) a generative sampler, along with in the supervised case 5) a distance aware-classifier. The Morse network can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data. Because of its versatility, the Morse neural networks unifies many techniques: e.g., the Entropic Out-of-Distribution Detector of (Macêdo et al., 2021) in OOD detection, the one class Deep Support Vector Description method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One Class classifier in continuous learning (Sun et al., 2021).The Morse neural network has connections to sup-port vector machines, kernel methods, and Morse theory in topology.

Keywords: Deep Generative Models · uncertainty quantification · out-of-distribution detection · Anomaly detection · generative probabilistic model

Author Information

Benoit Dherin (Google)
Huiyi Hu (Google DeepMind)
JIE REN (Google DeepMind)
Michael Dusenberry (Google)
Balaji Lakshminarayanan (Google Brain)

More from the Same Authors