Timezone: »

On Dropout and Nuclear Norm Regularization
Poorya Mianjy · Raman Arora

Tue Jun 11 03:10 PM -- 03:15 PM (PDT) @ Grand Ballroom
We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with the squared loss. We show that (a) the explicit regularizer is composed of an $\ell_2$-path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regularizer is the squared nuclear norm of the network map, and (c) for a sufficiently large dropout rate, we characterize the global optima of the dropout objective. We validate our theoretical findings with empirical results.

Author Information

Poorya Mianjy (Johns Hopkins University)
Raman Arora (Johns Hopkins University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors