Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models

Vision–Language–Action (VLA) policies often fail under distribution shift, suggesting that decisions may depend on spurious visual correlations rather than task-relevant causes. We formulate visual–action attribution as an interventional estimation problem. Accordingly, we introduce the Interventional Significance Score (ISS), an interventional masking procedure for estimating the causal influence of visual regions on action predictions, and the Nuisance Mass Ratio (NMR), a scalar measure of attribution to task-irrelevant features. We analyze the statistical properties of ISS and show that it admits unbiased estimation, and we characterize conditions under which action prediction error provides a valid proxy for causal influence. Experiments across diverse manipulation tasks indicate that NMR predicts generalization behavior and that ISS yields more faithful explanations than existing interpretability methods. These results suggest that interventional attribution provides a simple diagnostic approach for identifying causal misalignment in embodied policies.

Robots are increasingly being trained to follow language instructions by looking at their surroundings and deciding how to move. For example, a robot asked to “pick up the cup” must connect the words to the right object in the camera view and then choose the correct action. A major problem is that these robot AI systems can fail when the room, lighting, background, or object appearance changes, even if the task itself is the same. We study whether this happens because the robot is paying attention to distracting visual details, such as shadows, textures, or backgrounds, instead of the objects that matter. To test this, we developed a way to check which parts of an image actually influence the robot’s predicted actions. We then measure how much of that influence comes from irrelevant parts of the scene. Across our experiments, robots that relied more on irrelevant visual details were less successful in new environments. This gives researchers a practical way to diagnose when a robot policy is likely to fail, helping build more reliable robotic AI before deploying it in the real world.