One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception

By sharing intermediate features, collaborative perception extends each agent's sensing beyond standalone limits, but real-world feature modality heterogeneity remains a key barrier to effective fusion. Most existing methods, including direct adaption and protocol-based transforma-tion, typically rely on training adapters for newly emerging feature modalities and often require additional retraining or fine-tuning. Such repeated training is costly and is often infeasible across manufacturers due to model and data privacy con-straints, limiting real-world scalability. To address this issue, we propose UniTrans, a universal any-to-any feature modality translation model that instantiates translators on the fly for arbitrary modalities. UniTrans pretrains a bank of translator expert parameters and learns their combination coefficients as a function of source-to-target modality mapping. The mapping is measured in a modality-intrinsic latent space, where an intrinsic encoder extracts modality-specific yet scene-invariant codes from single-frame intermediate features, enabling UniTrans to instantiate translators in a zero-shot manner. Experiments on OPV2V-H and DAIR-V2X demonstrate that UniTrans consistently outperforms state-of-the-art methods in both simulated and real-world set-tings, enabling efficient any-to-any translation through a universal model. Code will be made available.

Self-driving vehicles can see more reliably when they share information with nearby vehicles or roadside units. In practice, however, different vehicles may use different sensors and perception systems, making their shared information hard to combine. This paper presents UniTrans, a general translation model that allows different vehicles to understand each other's shared information without retraining whenever a new vehicle type appears. Experiments on simulated and real-world driving datasets show that this approach improves collaborative perception and supports more scalable deployment of connected autonomous driving systems.