Timezone: »

Essentially No Barriers in Neural Network Energy Landscape
Felix Draxler · Kambis Veschgini · Manfred Salmhofer · Fred Hamprecht

Wed Jul 11 08:00 AM -- 08:20 AM (PDT) @ K1 + K2

Training neural networks involves finding minima of a high-dimensional non-convex loss function. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFAR100. Surprisingly, the paths are essentially flat in both the training and test landscapes. This implies that minima are perhaps best seen as points on a single connected manifold of low loss, rather than as the bottoms of distinct valleys.

Author Information

Felix Draxler (Heidelberg University)
Kambis Veschgini (University of Heidelberg)
Manfred Salmhofer (Heidelberg University)
Fred Hamprecht (Heidelberg Collaboratory for Image Processing)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors