World-Model Inspired Emotion-aware Token Refinement for Training-Free Multimodal Emotion Recognition

Multimodal Large Language Models (MLLMs) show promise for Multimodal Emotion Recognition (MER) but often remain unreliable because sparse emotional cues could be easily overwhelmed and affected by redundant context. While fine-tuning is effective, it is usually costly when using large models. Training-free methods like chain-of-thought reasoning provide a practical alternative, but they mostly rely on heuristic prompting to influence the model behaviors and do not explicitly focus on emotion relevant tokens internally, which would allow decision-relevant emotional tokens to be diluted by environmental noise, resulting in unstable predictions. To address this limitation without training, we rethink MER from a world-model perspective that treats emotion as a latent state inferred from noisy and redundant multimodal observations. Under frozen parameters, this view suggests that robustness depends on constraining why and how tokens contribute to inference. Based on this insight, we propose WETR (World-Model inspired Emotion-aware Token Refinement), a training-free, plug-and-play regulator that reshapes token usage through two mechanisms: Noise-suppressed Token Selection (NTS), which suppresses redundant intra-modal noise, and State-strengthened Token Reweighting (STR), which amplifies decision-relevant emotional tokens. Experiments on multiple MER benchmarks demonstrate that WETR consistently improves accuracy and stability under frozen parameters, which also improves token-level interpretability.

Recognizing emotions from videos, images, and text is useful for applications such as human-computer interaction, education, and mental health support. However, even powerful multimodal language models can make unstable predictions because emotional clues are often subtle, while videos and surrounding context contain many irrelevant details. Fine-tuning large models can help, but it is expensive and not always practical. We propose WETR, a training-free method that improves emotion recognition without changing the model’s parameters. Our idea is inspired by how a world model infers a hidden emotional state from noisy observations. WETR guides the model to focus less on redundant or distracting tokens and more on tokens that are truly useful for deciding the emotion. It does this through noise-suppressed token selection and emotion-aware token reweighting. Across several emotion recognition benchmarks, WETR improves both accuracy and prediction stability, while also making the model’s token-level decision process easier to interpret.