PartCo: Part-Level Correspondence Priors Enhance Category Discovery

Generalized Category Discovery (GCD) aims to identify both known and novel categories within unlabeled data by leveraging a set of labeled examples from known categories. Existing GCD methods primarily depend on semantic labels and global image representations, often overlooking the detailed part-level cues that are crucial for distinguishing closely related categories. In this paper, we introduce PartCo, short for Part-Level Correspondence Prior, a novel framework that enhances category discovery by incorporating part-level visual feature correspondences. By leveraging part-level relationships, PartCo captures finer-grained semantic structures, enabling a more nuanced understanding of category relationships. Importantly, PartCo seamlessly integrates with existing GCD methods without requiring significant modifications. Our extensive experiments on multiple benchmark datasets demonstrate that PartCo significantly improves the performance of current GCD approaches, outperforming most existing methods by bridging the gap between semantic labels and part-level visual compositions, thereby setting new benchmarks for GCD.

Modern AI systems can recognize image categories they have been trained on, but they often struggle when they encounter new kinds of categories / objects in the real world. This is a problem because practical vision systems should be able to organize both familiar and unfamiliar images without needing humans to label every example. Our work studies how to make this process more reliable by looking beyond the whole image and paying attention to object parts, such as a bird’s head, body, or tail. Existing methods usually compare images using a single overall representation, which can miss small but important visual details that separate closely related categories. We introduce PartCo, a method that teaches category discovery models to use part-level correspondences: visual patterns that recur across images even when parts change in size, pose, or visibility. PartCo can be added to existing category discovery methods without changing how they make predictions at test time. Across several image benchmarks, it consistently improves the ability of current methods to discover both known and new categories. This suggests that part-level visual structure is a simple and practical cue for building AI systems that better adapt to unfamiliar visual concepts.