Localize and Neutralize: Gradient-Guided Token Suppression Against Visual Prompt Injection Attack

Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the underlying mechanisms and struggle to balance efficiency and fidelity. In this work, we show that successful adversarial attacks do not rely on the entire image uniformly but instead depend on a small subset of critical image tokens. Based on this insight, we propose a defense that first localizes these critical tokens via gradient analysis and then neutralizes them through masking. We show that attribution based on output probabilities fails when adversarial attacks preserve the predicted token. To overcome this limitation, we introduce the Hidden-State Gradient Norm score for adversarial behavior attribution and prove that its ranking is consistent with that of the full adversarial loss gradient, providing a theoretical guarantee for accurate localization. GTM requires only a single forward–backward pass to identify and zero out a small number of high-scoring tokens, effectively disrupting the adversarial attack path. Extensive experiments on prompt injection and multimodal jailbreak attacks demonstrate that our approach reduces attack success rates (ASR) to near zero while preserving model utility with negligible computational overhead. The code is available at: https://github.com/fish883/GTM-Defense.

Images can be used to secretly manipulate multimodal AI assistants: a picture may contain hidden visual cues that make the model follow an attacker’s instructions instead of the user’s. Existing defenses often either slow the system down or reduce the model’s normal ability to understand images. We found that these attacks usually do not depend on the whole image; instead, they rely on a small number of internal visual tokens, which are compact units the model uses to represent an image. In experiments involving hidden-instruction and safety-bypass attacks, our approach reduced attack success to nearly zero while preserving the model’s usefulness.