Processing math: 100%
Skip to yearly menu bar Skip to main content


Poster

A Law of Robustness beyond Isoperimetry

Yihan Wu · Heng Huang · Hongyang Zhang

Exhibit Hall 1 #331
[ ]
[ PDF [ Poster

Abstract: We study the robust interpolation problem of arbitrary data distributions supported on a bounded space and propose a two-fold law of robustness. Robust interpolation refers to the problem of interpolating n noisy training data points in Rd by a Lipschitz function. Although this problem has been well understood when the samples are drawn from an isoperimetry distribution, much remains unknown concerning its performance under generic or even the worst-case distributions. We prove a Lipschitzness lower bound Ω(n/p) of the interpolating neural network with p parameters on arbitrary data distributions. With this result, we validate the law of robustness conjecture in prior work by Bubeck, Li and Nagaraj on two-layer neural networks with polynomial weights. We then extend our result to arbitrary interpolating approximators and prove a Lipschitzness lower bound Ω(n1/d) for robust interpolation. Our results demonstrate a two-fold law of robustness: a) we show the potential benefit of overparametrization for smooth data interpolation when n=poly(d), and b) we disprove the potential existence of an O(1)-Lipschitz robust interpolating function when n=exp(ω(d)).

Chat is not available.