FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model

Fine-grained vision-language understanding requires precise alignment between visual content and linguistic descriptions, a capability that remains limited in current models, particularly in non-English settings. While models like CLIP perform well on global alignment, they often struggle to capture fine-grained details in object attributes, spatial relations, and linguistic expressions, with limited support for bilingual comprehension. To address these challenges, we introduce FG-CLIP 2, a bilingual vision-language model designed to advance fine-grained alignment for both English and Chinese. Our approach leverages rich fine-grained supervision, including region-text matching and long-caption modeling, alongside multiple discriminative objectives. We further introduce the Textual Intra-modal Contrastive (TIC) loss to better distinguish semantically similar captions. Trained on a carefully curated mixture of large-scale English and Chinese data, including a newly released 12M Chinese region-text dataset, FG-CLIP 2 achieves powerful bilingual performance. To enable rigorous evaluation, we present a new benchmark for Chinese multimodal understanding, featuring long-caption retrieval and bounding box classification. Extensive experiments on 29 datasets across 8 tasks show that FG-CLIP 2 outperforms existing methods, achieving state-of-the-art results in both languages. We release the model, code, and benchmark to facilitate future research on bilingual fine-grained vision-language alignment.

We teach computers to understand pictures together with their text descriptions. Existing systems like CLIP can tell that an image shows "a dog in a park," but they often miss the fine details — such as the color of the dog's collar, where it is standing, or how it differs from another similar dog nearby. They also work much better in English than in Chinese. In this work, we build FG-CLIP 2, a new image-and-text model that pays attention to these fine-grained details and works equally well in both English and Chinese. We train it on large collections of images paired with both short and detailed descriptions, and teach it to tell apart sentences that look similar but mean different things. To support research on Chinese vision-language understanding, we also release a new large-scale Chinese dataset and a new test suite with more challenging tasks. Across many evaluations, FG-CLIP 2 understands images and text more accurately than previous methods. We share our model, data, and code freely so others can build on this work.