Are Object-Centric Representations Better at Compositional Generalization?

Compositional generalization, the ability to reason about novel combinations of familiar concepts, is fundamental to human cognition and a critical challenge for machine learning. Object-centric (OC) representations, which encode a scene as a set of objects, are often argued to support such generalization, but systematic evidence in visually rich settings is limited. We introduce a Visual Question Answering benchmark across three controlled visual worlds (CLEVRTex, Super-CLEVR, and MOVi-C) to measure how well vision encoders, with and without object-centric biases, generalize to unseen combinations of object properties. To ensure a fair and comprehensive comparison, we carefully account for training data diversity, sample size, representation size, downstream model capacity, and compute. We use DINOv2 and SigLIP2, two widely used vision encoders, as the foundation models and their OC counterparts. Our key findings reveal that (1) OC approaches are superior in harder compositional generalization settings; (2) original dense representations surpass OC only on easier settings and typically require substantially more downstream compute; and (3) OC models are more sample efficient, achieving stronger generalization with fewer images, whereas dense encoders catch up or surpass them only with sufficient data and diversity. Overall, object-centric representations offer stronger compositional generalization when any one of dataset size, training data diversity, or downstream compute is constrained.

Imagine seeing a red cube for the first time after only ever encountering red spheres and blue cubes; most people instantly recognize it. This kind of recombining familiar properties into new arrangements, called compositional generalization, is something even today's strongest vision models still struggle with. Some researchers argue that giving a model an explicit "slot" for each object in a scene should help, but systematic evidence has been thin. To test this, we built a benchmark of rendered 3D scenes across three visually rich worlds, withheld 20% of all object–property combinations from training, and asked models to answer questions about scenes containing those held-out objects. We compared two widely used vision foundation models with their object-centric counterparts, carefully matching training data, model size, and compute to ensure a fair comparison. The object-centric models clearly win whenever data, training diversity, or computational budget is limited; standard models catch up only by spending substantially more of all three. For practitioners working under realistic resource constraints, an object-centric representation offers a more efficient path to robust compositional reasoning.