Timezone: »

 
Poster
Does Data Augmentation Lead to Positive Margin?
Shashank Rajput · Zhili Feng · Zachary Charles · Po-Ling Loh · Dimitris Papailiopoulos

Wed Jun 12 06:30 PM -- 09:00 PM (PDT) @ Pacific Ballroom #155
in Posters Wed »

Data augmentation (DA) is commonly used during model training, as it significantly improves test error and model robustness. DA artificially expands the training set by applying random noise, rotations, crops, or even adversarial perturbations to the input data. Although DA is widely used, its capacity to provably improve robustness is not fully understood. In this work, we analyze the robustness that DA begets by quantifying the margin that DA enforces on empirical risk minimizers. We first focus on linear separators, and then a class of nonlinear models whose labeling is constant within small convex hulls of data points. We present lower bounds on the number of augmented data points required for non-zero margin, and show that commonly used DA techniques may only introduce significant margin after adding exponentially many points to the data set.

Keywords: Adversarial Examples · Deep Learning Theory · Information Theory and Estimation · Robust Statistics and Machine Learning · Statistical Learning Theory

Author Information

Shashank Rajput (University of Wisconsin - Madison)
Shashank Rajput

I am a 5th year graduate student in the CS department at UW-Madison. I am advised by Prof. Dimitris Papailiopoulos. I am interested in sparsity in Deep Learning and Distributed Optimization.

Zhili Feng (University of Wisconsin-Madison)
Zachary Charles (University of Wisconsin-Madison)
Po-Ling Loh (UW-Madison)
Dimitris Papailiopoulos (University of Wisconsin-Madison)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors