FoMu-SSL: Foundation Model-Guided Multi-Sensor Self-Supervised Learning for Remote Sensing

The application of remote sensing in computer vision struggles with domain shifts among datasets, where models trained on one satellite dataset may not generalize well to others due to diverse geographic and environmental conditions. These differences hinder the self-supervised representation learning, hence this paper introduces an innovative strategy that employs the ImageNet-pretrained foundation model as a guide to enhance the semantic feature extraction process. We also incorporate radar sensor to complement optical sensor inputs, without additional training. Our approach significantly improves performances in segmentation, detection, and classification tasks, offering a robust and efficient method for self-supervised learning in remote sensing.