PRIM：Cooperative Dynamic Token Compression for Efficient Large Multimodal Models

Large multimodal models (LMMs) have shown strong capabilities in audio-visual understanding by jointly reasoning over visual, auditory, and linguistic inputs. However, processing long-form audio-visual content often requires a large number of tokens, leading to substantial computational and memory overhead during inference. Existing efficiency-oriented methods typically apply uniform compression or pruning strategies, which overlook modality-specific characteristics and instruction-dependent reasoning behaviors in multimodal models. In this work, we present PRIM, an inference framework for efficient multimodal reasoning that systematically compresses audio-visual representations based on attention dynamics and instruction relevance. Motivated by an attention-based analysis revealing modality imbalance and layer-wise redundancy in LMMs, PRIM introduces a cooperative compression pipeline that spans both multimodal encoders and the language model. Specifically, PRIM performs early text-conditioned audio-visual fusion to externalize cross-modal interactions, applies attention-guided and frequency-aware strategies to remove redundant audio and video tokens, and further adapts token retention inside the language model according to task demands. Extensive experiments on multiple audio-visual benchmark datasets demonstrate that PRIM consistently achieves stable and superior efficiency--accuracy trade-offs across diverse tasks and datasets. These results demonstrate that PRIM, a multimodal cooperative compression approach, provides an effective pathway toward scalable and efficient audio-visual reasoning.

Large multimodal models can understand videos by combining visual, audio, and language information, but they often require a very large amount of computation when processing long audio-visual content. This is because videos and audio streams are converted into many small units for the model to analyze, and many of these units may be redundant or only weakly related to the user’s question. In this work, we propose PRIM, a framework that makes large multimodal models more efficient by selectively keeping the most useful audio and video information. Instead of treating all visual and audio information equally, PRIM looks at how the model pays attention to different parts of the input and uses this information to remove less important content. It also considers the user’s instruction, so the model can preserve information that is more relevant to the specific task. PRIM combines several ideas: it first helps audio, video, and text information interact before entering the main language model; then it removes redundant audio and video tokens using attention-based and signal-aware strategies; finally, it further adapts the amount of information kept inside the language model according to the difficulty of the instruction. Experiments on multiple audio-visual understanding benchmarks show that PRIM can reduce computational cost and memory usage while maintaining strong accuracy. These results suggest that large multimodal models can become more practical for long-video understanding and real-world deployment by using cooperative and task-aware compression.