Timezone: »

 
Spotlight
COAT: Measuring Object Compositionality in Emergent Representations
Sirui Xie · Ari Morcos · Song-Chun Zhu · Shanmukha Ramakrishna Vedantam

Wed Jul 20 11:15 AM -- 11:20 AM (PDT) @ Hall G
in MISC: Representation Learning/Causality »

Learning representations that can decompose a multi-object scene into its constituent objects and recompose them flexibly is desirable for object-oriented reasoning and planning. Built upon object masks in the pixel space, existing metricsfor objectness can only evaluate generative models with an object-specific “slot” structure. We propose to directly measure compositionality in the representation space as a form of objections, making such evaluations tractable for a widerclass of models. Our metric, COAT (Compositional Object Algebra Test), evaluates if a generic representation exhibits certain geometric properties that underpin object compositionality beyond what is already captured by the raw pixel space. Our experiments on the popular CLEVR (Johnson et.al., 2018) domain reveal that existing disentanglement-based generative models are not as compositional as one might expect, suggesting room for further modeling improvements. We hope our work allows for a unified evaluation of object-centric representations, spanning generative as well as discriminative, self-supervised models.

Keywords: DL: Generative Models and Autoencoders · DL: Self-Supervised Learning · MISC: Representation Learning · MISC: Unsupervised and Semi-supervised Learning

Author Information

Sirui Xie (UCLA)
Ari Morcos (Facebook AI Research (FAIR))
Song-Chun Zhu (UCLA)
Shanmukha Ramakrishna Vedantam (Facebook AI Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors