Skip to yearly menu bar Skip to main content


Poster

Convolutional and Residual Networks Provably Contain Lottery Tickets

Rebekka Burkholz

Hall E #304

Keywords: [ DL: Theory ]


Abstract:

The Lottery Ticket Hypothesis continues to have a profound practical impact on the quest for small scale deep neural networks that solve modern deep learning tasks at competitive performance. These lottery tickets are identified by pruning large randomly initialized neural networks with architectures that are as diverse as their applications. Yet, theoretical insights that attest their existence have been mostly focused on deed fully-connected feed forward networks with ReLU activation functions. We prove that also modern architectures consisting of convolutional and residual layers that can be equipped with almost arbitrary activation functions can contain lottery tickets with high probability.

Chat is not available.