Multiview Self-Representation Learning across Heterogeneous Views

Features of the same sample generated by different pretrained models often exhibit inherently distinct feature distributions. Learning invariant representations from large-scale unlabeled visual data in a fully unsupervised transfer manner remains a significant challenge. In this paper, we propose a multiview self-representation learning (MSRL) method in which invariant representations are learned by exploiting the self-representation property of features across heterogeneous views. The features are derived from large-scale unlabeled visual data through transfer learning with various pretrained models and are referred to as heterogeneous multiview data. We introduce an information-passing mechanism that relies on self-representation learning to support feature aggregation over the outputs of the linear model. Moreover, an assignment probability distribution consistency scheme is presented to guide multiview self-representation learning by exploiting complementary information across different views. Consequently, representation invariance across different linear models is enforced through this scheme. Additionally, we provide a theoretical analysis of the assignment probability distribution consistency and the incremental views. Extensive experiments demonstrate that the proposed MSRL method consistently outperforms several state-of-the-art approaches.

When different artificial intelligence (AI) models look at the same image, they often focus on entirely different details. This paper addresses a fundamental question: how can we combine the knowledge of multiple AI models to construct a unified understanding of images, without relying on any human-provided labels? We introduce Multiview Self-Representation Learning (MSRL) to tackle this challenge. A key insight is that images belonging to the same category tend to have descriptions that can be expressed as combinations of one another. The descriptions of the same image produced by all models are encouraged to agree on a common interpretation. The proposed MSRL method selects the most relevant neighboring images to refine each representation, making the final descriptions more discriminative. We provide mathematical guarantees showing that as more high-quality AI models are incorporated, the combined understanding becomes increasingly reliable. Experiments across eight image datasets show that MSRL consistently outperforms existing methods, which demonstrates that combining the complementary strengths of multiple AI models is a powerful strategy for unsupervised image understanding.