Structure-aware Granular-Ball based Information Bottleneck for Multi-modal Clustering

Multi-modal clustering, which integrates information from diverse sources and feature modalities, has shown great potential in data mining and computer vision. However, existing methods relying on single-granularity relationships often struggle with complex data distributions, leading to limited performance, as fine-grained features are prone to local heterogeneity and redundant perturbations while coarse-grained representations tend to lose local structural information. To address these limitations, we introduce granular-balls (GBs), adaptive multi-granularity hyperspheres that enclose similar samples, and propose the Structure-aware Granular-Ball based Information Bottleneck (SGB-IB) algorithm. This method initializes the dataset as a single GB and recursively splits GBs based on a purity metric, which quantifies the average mutual information between sample features and K-means-derived pseudo-labels across all modalities. It also balances local structure preservation and global redundancy suppression through a structure-aware objective function. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches, validating the effectiveness of fusing GB structures with information-theoretic principles.

Imagine you have a massive pile of mixed-up items, such as photos, text documents, and audio recordings. Your goal is to teach a computer to sort them into groups automatically, like separating family photos from work files or identifying different types of animals in pictures. Current computer programs often struggle because they look at the data in only one rigid way. They either focus too much on tiny details, i.e., getting distracted by irrelevant differences, or too much on the big picture, i.e., missing important clues. We developed a new approach that mimics how humans naturally organize things. Instead of using fixed boxes, our method creates flexible "balls" of data. It starts with everything in one big ball and then intelligently splits it into smaller balls whenever the items inside are too different from each other. This allows the computer to understand both the fine details and the overall structure at the same time. When tested on standard challenges, our method consistently outperformed existing technologies. By helping computers better understand complex, mixed-type information, this research could lead to smarter AI assistants, more accurate medical scans, and better self-driving cars.