VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning

With the rapid advancement of large multimodal models (LMMs), inference-time overhead has become a key bottleneck for real-world deployment. Existing methods typically prune visual tokens at prefill, assuming the required visual evidence remains static during reasoning. However, we empirically show that visual evidence is strongly step-dependent: only a sparse subset of visual tokens is critical at each decoding step, and the critical set evolves across reasoning. Furthermore, we identify a coupled bottleneck where redundant visual context can steer the model toward query-irrelevant regions, lengthening the reasoning trace. Guided by these insights, we propose VisionPulse, a step-wise visual token pruning framework during reasoning. VisionPulse computes a lightweight visual attention mass to estimate the step-wise retention budget by exploiting its strong positive correlation with LMMs' effective visual token usage and retain only the most critical tokens under this budget. By enforcing visual sparsity during reasoning, VisionPulse filters redundant visual context while preserving relevant visual evidence, shortening reasoning traces naturally. Extensive experiments show that VisionPulse only retains 5\% of visual tokens per step with reasoning traces shortened by 11.2\%, while keeping accuracy almost unchanged.

Large AI systems that can understand both images and text are increasingly used to answer complex visual questions, such as reading charts, interpreting scenes, or solving problems from images. However, these systems are expensive to run because they often process all visual details at every step, even when only a few parts of the image are actually needed. In this paper, we introduce VisionPulse, a method that makes multimodal reasoning more efficient. The key idea is that an AI model would not look at the same image regions throughout the whole reasoning process. Instead, different parts of an image become useful at different moments. VisionPulse therefore lets the model keep only the most relevant visual information at each reasoning step, while ignoring unnecessary visual details. This not only reduces computation, but can also make the model’s reasoning shorter and more focused. Experiments show that VisionPulse can keep only about 5% of visual information per step, shorten reasoning by about 11%, and maintain almost the same accuracy as using all visual information. These results suggest that dynamic visual selection is a practical way to make multimodal AI systems faster and more efficient without sacrificing performance.