Multi-Label Learning with Contrastive Cluster Self-Supervision for 3D Hierarchical Semantic Segmentation

3D hierarchical semantic segmentation (3DHS) is crucial for embodied intelligence that demands the coarse-to-fine grained and multi-hierarchy understanding of 3D scenes. 3DHS tasks can be addressed by multi-label learning, but facing two issues: I) learning multiple labels for each point with a shared model can lead to multi-hierarchy conflicts in cross-hierarchy optimization, and II) the class imbalance issue is inevitable across multiple hierarchies of 3D scenes, making the model easily be dominated by major classes. To address these issues, we propose a novel multi-label learning with contrastive cluster self-supervision framework for 3DHS. Specifically, we propose a late-decoupled multi-label learning 3DHS network which employs decoupled decoders with the coarse-to-fine hierarchical consistency guidance. This late-decoupled model architecture can mitigate the underfitting and overfitting conflicts among multiple hierarchies and also constrain the class imbalance problem within each individual hierarchy. Moreover, we introduce a 3DHS-oriented contrastive cluster self-supervision learning method, which learns cluster-wise point cloud features with contrastive loss and produces self-supervised information to enhance the class-imbalance segmentation. Extensive experiments on multiple datasets and backbones demonstrate that our approach promotes the multi-hierarchy balance and mitigates the class imbalance issue in 3DHS tasks.

In this paper, we focus on addressing two persistent but overlooked challenges for 3D hierarchical semantic segmentation: the multi-hierarchy conflict between learning multiple labels and a parameter-sharing model, and the class imbalance issue hindering performance on minority classes. To this end, we propose ML3DHS, a novel late-decoupled multi-label learning framework with the contrastive cluster self-supervision method. The late-decoupled 3DHS architecture can mitigate the multi-hierarchy conflicts and the auxiliary contrastive discrimination branch can improve the segmentation ability for point clouds with class imbalance. Extensive experiments on multiple datasets and backbones demonstrate that our approach achieves state-of-the-art 3DHS performance, whose core components can also be used as a plug-and-play enhancement to improve previous 3DHS methods.