Timezone: »

Proving the Lottery Ticket Hypothesis: Pruning is All You Need
Eran Malach · Gilad Yehudai · Shai Shalev-Schwartz · Ohad Shamir

Wed Jul 15 11:00 AM -- 11:45 AM & Wed Jul 15 10:00 PM -- 10:45 PM (PDT) @ None #None

The lottery ticket hypothesis (Frankle and Carbin, 2018), states that a randomly-initialized network contains a small subnetwork such that, when trained in isolation, can compete with the performance of the original network. We prove an even stronger hypothesis (as was also conjectured in Ramanujan et al., 2019), showing that for every bounded distribution and every target network with bounded weights, a sufficiently over-parameterized neural network with random weights contains a subnetwork with roughly the same accuracy as the target network, without any further training.

Author Information

Eran Malach (Hebrew University Jerusalem Israel)
Gilad Yehudai (Weizmann Institute of Science)
Shai Shalev-Schwartz (Hebrew University of Jerusalem)
Ohad Shamir (Weizmann Institute of Science)

More from the Same Authors