Timezone: »

 
Poster
CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations
Gengchen Mai · Ni Lao · Yutong He · Jiaming Song · Stefano Ermon

Tue Jul 25 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #137
in Poster Session 2 »

Geo-tagged images are publicly available in large quantities, whereas labels such as object classes are rather scarce and expensive to collect. Meanwhile, contrastive learning has achieved tremendous success in various natural image and language tasks with limited labeled data. However, existing methods fail to fully leverage geospatial information, which can be paramount to distinguishing objects that are visually similar. To directly leverage the abundant geospatial information associated with images in pre-training, fine-tuning, and inference stages, we present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images. We use a dual-encoder to separately encode the images and their corresponding geo-locations, and use contrastive objectives to learn effective location representations from images, which can be transferred to downstream supervised tasks such as image classification. Experiments show that CSP can improve model performance on both iNat2018 and fMoW datasets. Especially, on iNat2018, CSP significantly boosts the model performance with 10-34% relative improvement with various labeled training data sampling ratios.

[ PDF]  [ Poster] 

Author Information

Gengchen Mai (University of Georgia, GA, USA)
Gengchen Mai

Dr. Gengchen Mai is an Assistant Professor (since Aug. 1st 2022) at the Department of Geography at UGA, and a Graduate Program Faculty of the School of Computing, UGA. His research areas are Spatially Explicit Artificial Intelligence, Geographic Knowledge Graphs, Geographic Question Answering, and so on. He is the receipt of many prestigious awards including AAG 2021 Dissertation Research Grants, AAG 2022 William L. Garrison Award for Best Dissertation in Computational Geography, AAG 2023 J. Warren Nystrom Dissertation Award, Top 10 WGDC 2022 Global Young Scientist Award, the The Jack and Laura Dangermond Graduate Fellowship, and so on.

Ni Lao (Google)
Yutong He (School of Computer Science, Carnegie Mellon University)
Jiaming Song (NVIDIA)
Stefano Ermon (Stanford University)

More from the Same Authors