API: Adaptive Prototype Imputation for Incomplete Multimodal Sentiment Analysis

Multimodal sentiment analysis aims to infer human emotions by integrating signals from diverse modalities. However, missing modalities are common in real-world applications due to sensor failure, data corruption, or privacy concerns. Existing approaches typically follow two main paradigms: recovery-based and non-recovery-based methods. This dichotomy results in two critical limitations: I) computational inefficiency and semantic inconsistency (recovery-based methods rely on heavy generators that incur prohibitive inference latency and risk semantic drift due to lack of class-level priors); II) lack of instance specificity (non-recovery-based methods rely on static global mappings that fail to capture sample-specific affective cues). To address these gaps, we propose Adaptive Prototype Imputation (API). To mitigate I), we introduce Semantic-anchored Class-Temporal Prototype Estimation (SCOPE) to construct non-trainable prototypes as stable semantic anchors, promoting semantic reliability. To resolve II), we design Directional Instance-Adaptive Affine Modulation (DIAM) to dynamically modulate these anchors via direction-specific affine transformations, capturing instance-unique affective characteristics without generative overhead. Experimental results on CMU-MOSI and CMU-MOSEI demonstrate that API outperforms state-of-the-art baselines, establishing a robust and lightweight prototype-centric paradigm for multimodal sentiment analysis. The code is publicly available at https://github.com/KX-yolo/API.

Many AI systems try to understand human sentiment by combining what people say, how they look, and how they sound. In real-world settings, however, some of these signals may be missing because of sensor failures, poor transmission, or privacy restrictions. Existing methods either try to reconstruct the missing information, which can be slow and sometimes unreliable, or avoid reconstruction but may miss details that are specific to each person or situation. We propose Adaptive Prototype Imputation, a lightweight method for handling missing information in multimodal sentiment analysis. Instead of generating an entire missing signal, our method uses typical sentiment patterns learned from the training data as stable references, and then adjusts them according to the information that is still available for the current sample. This helps the model keep its predictions consistent while still capturing individual differences. Experiments on standard sentiment analysis benchmarks show that our method is both accurate and efficient, making it more suitable for real-world applications where data is often incomplete.