The Hitchhiker’s Guide to Mechanistic Interpretability of Vision-Language-Action Models

The mechanistic interpretability toolkit for large language models (LLMs) was built around properties that autoregressive text transformers happen to have: one residual stream, one unembedding matrix, homogeneous layers, a single forward pass per output, a verifiable target token. These are not laws of computation but conveniences of one architectural choice. As the field moves from LLMs to Vision-Language-Action (VLA) models, between two and six of these conveniences quietly disappear depending on the VLA's design, and the tools that depend on them keep producing numerically plausible outputs that no longer mean what they did on an LLM. We name seven such conveniences as cautions A1--A7, classify VLAs into seven architecture types, and identify which caution each type breaks. The result is a compact map of where the existing LLM-style toolkit is safe to apply, where it requires a small adaptation, and where it cannot be made to work without re-engineering.