Semi-Supervised Noise Adaptation: Transferring Knowledge from Noise Domain

Transfer learning aims to facilitate the learning of a target domain by transferring knowledge from a source domain. The source domain typically contains semantically meaningful samples (e.g., images) to facilitate effective knowledge transfer. However, a recent study observes that the noise domain constructed from simple distributions (e.g., Gaussian distributions) can serve as a surrogate source domain in the semi-supervised setting, where only a small proportion of target samples are labeled while most remain unlabeled. Based on this surprising observation, we formulate a novel problem termed Semi-Supervised Noise Adaptation (SSNA), which aims to leverage a synthetic noise domain to improve the generalization of the target domain. To address this problem, we first establish a generalization bound characterizing the effect of the noise domain on generalization, based on which we propose a Noise Adaptation Framework (NAF). Extensive experiments demonstrate that NAF effectively leverages the noise domain to tighten the generalization bound of the target domain, leading to improved performance. The codes are available at https://github.com/AIResearch-Group/SSNA.

Transfer learning helps machine learning models adapt knowledge from one domain to another, especially when labeled samples are limited. Existing methods usually rely on meaningful source samples, such as images or text, to support this transfer process. However, recent studies suggest that even randomly generated noise may surprisingly contain useful transferable knowledge. Motivated by this observation, we study whether synthetic noise can improve learning when only a small portion of the target samples are labeled. We formulate a new problem called Semi-Supervised Noise Adaptation (SSNA), where a model learns from both limited labeled samples and artificially generated noise. To address this problem, we develop a theoretical framework to analyze how noise influences model generalization and propose a method called Noise Adaptation Framework (NAF). Extensive experiments show that properly structured noise can help models learn more discriminative representations and achieve better performance across different tasks. Our work offers a new perspective on transfer learning, suggesting that useful transferable knowledge may not always need to come from semantically meaningful samples.