Nouns, Not Modifiers: OpenVLA Parses Objects but Fails at Spatial Composition
Abstract
Vision-Language-Action models are trained predominantly on absolute instructions that name specific objects, but real-world deployment demands the capacity to understand compositional instructions that combine object identifiers with spatial modifiers. We evaluate OpenVLA-7B on relational instructions using a matched-pair design in SimplerEnv by testing six instruction types on 20 scenes: two absolute, two distance based, and two spatial. OpenVLA demonstrates 90.0\% accuracy on absolute instructions but fails at chance on distance (52.5\%) and spatial (47.5\%) instructions, with 15\% accuracy on 'left' and 80\% accuracy on 'right' that reveals a systematic rightward default. Failure mode analysis reveals that 93\% of failures involve the end-effector reaching for the wrong can, rather than the third distractor object, which shows that the model correctly parses the object category but fails to compose it with the relational modifier. This paper shows that compositional spatial reasoning does not emerge from demonstration-based learning alone, and identifies key failure patterns that must be addressed prior to VLA deployment in the real world.