CauseCollab: Causal Unified and Modality-Agnostic Network for Heterogeneous Collaborative Perception

Collaborative perception enhances environment understanding through multi-agent information sharing, but its performance in real-world scenarios is constrained by heterogeneous sensor modalities and model architectures. Recent protocol-based two-stage methods alleviate this problem by mapping heterogeneous features into a shared protocol space; however, independently trained modality-specific converters often generate modality-specific pseudo-protocol distributions, leading to semantic inconsistency and error accumulation, which is particularly pronounced in scenarios with large modality discrepancies. To address this issue, we propose CauseCollab, a causal unified and modality-agnostic network. CauseCollab formulates representation learning in the protocol space from a causal perspective, explicitly disentangling semantic factors from modality-specific statistical confounders via causal metric learning. Meanwhile, CauseCollab adopts context-guided Unified Converter for heterogeneous modalities to ensure cross-modal semantic consistency. In addition, integrating new modalities only requires training adapters with minimal parameters. Extensive experiments on the OPV2V and DAIR-V2X datasets demonstrate that CauseCollab achieves state-of-the-art performance, with more significant gains in scenarios involving large modality gaps.

Autonomous vehicles can better understand their surroundings when they share perception information with nearby vehicles and roadside sensors. However, in practice, different vehicles may use different sensors, such as cameras or LiDAR, and different perception models. This makes it difficult for them to directly understand and combine each other's information. This paper proposes CauseCollab, a framework that helps different perception systems communicate through a shared representation. It reduces the influence of sensor-specific differences and keeps the shared information focused on useful scene semantics. As a result, new sensing modalities can be added with only a small amount of extra training and computation. Experiments show that the method improves collaborative perception under heterogeneous settings, which may support more reliable multi-agent perception systems for autonomous driving and intelligent transportation.