TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches

By integrating Chain-of-Thought (CoT) reasoning, Vision-Language-Action (VLA) models have demonstrated strong capabilities in robotic manipulation, particularly by improving generalization and interpretability. However, the security of CoT-based reasoning mechanisms remains largely unexplored. In this paper, we show that CoT reasoning introduces a novel attack vector for targeted behavior hijacking—for example, causing a robot to mistakenly deliver a knife to a person instead of an apple—without modifying the user’s instruction. We first provide empirical evidence that CoT strongly governs action generation, even when it is semantically misaligned with the input instructions. Building on this observation, we propose TRAP, the first targeted behavior-hijacking adversarial attack against CoT-reasoning VLA models. By targeting the reasoning-to-action pathway, TRAP uses an adversarial patch (e.g., a tablecloth placed on the table) to steer intermediate CoT reasoning and downstream actions toward adversary-defined behaviors. Extensive evaluations on three representative reasoning VLAs, spanning distinct CoT reasoning mechanisms, demonstrate the effectiveness of TRAP. Notably, we implemented the patch by printing it on paper in a real-world setting. Our findings highlight the urgent need to secure CoT reasoning in VLA systems. The project page is available at https://zhengxian-huang.github.io/TRAP-website/.

Modern robots can use vision, language, and step-by-step reasoning to understand instructions and act in the physical world. This reasoning can improve their abilities, but it may also introduce a new security risk. We show that a printed pattern placed in a robot’s workspace, such as a tablecloth, can mislead the robot’s reasoning. This can cause a targeted wrong action, such as choosing a knife when the user asked for an apple, even though the user’s instruction is unchanged. We introduce TRAP, a method that studies this risk by steering a robot’s reasoning and actions toward an attacker-chosen goal. We evaluate TRAP on three representative robot AI systems that reason in different ways. In both simulations and real-world tests with printed patches, TRAP shows that corrupted reasoning can lead to corrupted actions. The results suggest that making a robot’s reasoning visible does not automatically make it safe. The reasoning process itself must be checked and protected. Our work highlights the need for defenses that keep robot reasoning aligned with human instructions before such systems are deployed in everyday settings.