Poster
On Connected Sublevel Sets in Deep Learning
Quynh Nguyen
This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large global valley.
Author Information
Quynh Nguyen (Saarland University)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: On Connected Sublevel Sets in Deep Learning »
Wed Jun 12th 11:20 -- 11:25 AM Room Room 104
More from the Same Authors
-
2018 Poster: Optimization Landscape and Expressivity of Deep CNNs »
Quynh Nguyen · Matthias Hein -
2018 Oral: Optimization Landscape and Expressivity of Deep CNNs »
Quynh Nguyen · Matthias Hein -
2018 Poster: Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions »
Quynh Nguyen · Mahesh Mukkamala · Matthias Hein -
2018 Oral: Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions »
Quynh Nguyen · Mahesh Mukkamala · Matthias Hein -
2017 Poster: The loss surface of deep and wide neural networks »
Quynh Nguyen · Matthias Hein -
2017 Talk: The loss surface of deep and wide neural networks »
Quynh Nguyen · Matthias Hein
