Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward

Training strategies with unlabeled and few labeled examples under 1-pixel attack by combining supervised and self-supervised learning

Gabriel Biscaro Cavallari · Moacir Ponti


Self-supervised learning pre-training exhibited excellent performance on feature learning by using only unlabeled examples. Still, it is not clear how different self-supervised tasks perform under distinct image domains and there are still training issues to be tackled under scenarios of limited labeled data. We investigate two self-supervised tasks: rotation and Barlow Twins, on three distinct image domains, exploring a combination of supervised and self-supervised learning. Our motivation is to work on scenarios where the proportion of labeled data with respect to unlabeled data is small, as well as investigate the model's robustness to 1-pixel attacks. The models that combine supervised with self-supervised tasks can take advantage of the unlabeled data to improve the learned representation in terms of the linear discrimination, as well as allowing learning even under attack.

Chat is not available.