Timezone: »

On the Implicit Bias of Dropout
Poorya Mianjy · Raman Arora · Rene Vidal

Fri Jul 13 09:15 AM -- 12:00 PM (PDT) @ Hall B #69

Algorithmic approaches endow deep learning systems with implicit bias that helps them generalize even in over-parametrized settings. In this paper, we focus on understanding such a bias induced in learning through dropout, a popular technique to avoid overfitting in deep learning. For single hidden-layer linear neural networks, we show that dropout tends to make the norm of incoming/outgoing weight vectors of all the hidden nodes equal. In addition, we provide a complete characterization of the optimization landscape induced by dropout.

Author Information

Poorya Mianjy (Johns Hopkins University)
Raman Arora (Johns Hopkins University)
Rene Vidal (Johns Hopkins University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors