Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation

Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics. This paper investigates a novel threat: Semantic-Aware Hijacking. We explore the feasibility of hijacking multiple stateless decisions simultaneously using a single universal perturbation. We introduce the Semantic-Aware Universal Perturbation (SAUP), which acts as a semantic router, "actively" perceiving input semantics and routing them to distinct, attacker-defined targets. To achieve this, we conduct a theoretical and empirical analysis on the geometric properties in the latent space. Guided by these insights, we propose the Semantic-Oriented (SORT) optimization strategy and annotate a new dataset with fine-grained semantics to evaluate performance. Extensive experiments on three representative MLLMs demonstrate the fundamental feasibility of this attack, achieving a 66% attack success rate over five targets using a single frame against Qwen.

Current image hijacking attacks use adversarial perturbations to force Multimodal Large Language Models (MLLMs) to produce a single, fixed output. We investigated a novel security threat: is it possible to create a universal perturbation that steers an MLLM to map different images to entirely different, predefined outputs? We optimized a single perturbation across image clusters with varying semantics. We discovered that such a perturbation acts like a "semantic router," forcing the model to generate distinct outputs based on visual semantics, and it generalizes well to unseen images. By proving the feasibility of this semantic-aware attack, we hope our findings serve as a starting point to motivate future research on image hijacking in dynamic environments.