Poster
On Dropout and Nuclear Norm Regularization
Poorya Mianjy · Raman Arora

Tue Jun 11th 06:30 -- 09:00 PM @ Pacific Ballroom #79

We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer is composed of an $\ell_2$-path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regularizer is the squared nuclear norm of the network map, and (c) for a sufficiently large dropout rate, we characterize the global optima of the dropout objective. We validate our theoretical findings with empirical results.

Author Information

Poorya Mianjy (Johns Hopkins University)
Raman Arora (Johns Hopkins University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors