Timezone: »

 
Spotlight
Supervised Learning with General Risk Functionals
Liu Leqi · Audrey Huang · Zachary Lipton · Kamyar Azizzadenesheli

Wed Jul 20 02:20 PM -- 02:25 PM (PDT) @ Room 310
in Deep Learning/MISC »

Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class. The emergence of risk-sensitive learning requires generalization guarantees for functionals of the loss distribution beyond the expectation. While prior works specialize in uniform convergence of particular functionals, our work provides uniform convergence for a general class of H\"older risk functionals for which the closeness in the Cumulative Distribution Function (CDF) entails closeness in risk. We establish the first uniform convergence results for estimating the CDF of the loss distribution, which yield uniform convergence guarantees that hold simultaneously both over a class of H\"older risk functionals and over a hypothesis class. Thus licensed to perform empirical risk minimization, we develop practical gradient-based methods for minimizing distortion risks (widely studied subset of H\"older risks that subsumes the spectral risks, including the mean, conditional value at risk, cumulative prospect theory risks, and others) and provide convergence guarantees. In experiments, we demonstrate the efficacy of our learning procedure, both in settings where uniform convergence results hold and in high-dimensional settings with deep networks.

Keywords: T: Learning Theory · MISC: Supervised Learning

Author Information

Liu Leqi (Carnegie Mellon University)
Audrey Huang (University of Illinois Urbana-Champaign)
Zachary Lipton (Carnegie Mellon University)
Kamyar Azizzadenesheli (Purdue University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors