LAST: Bridging Vision-Language and Action Manifolds via Gromov-Wasserstein Alignment

We formulate the learning of generalist Vision-Language-Action (VLA) models as a Gromov-Wasserstein alignment problem, aiming to map semantically similar VL embeddings to physically similar motion primitives. However, solving this is challenging due to the mathematical heterogeneity between the domains: the semantic space of vision-language is topologically linear and isotropic, while the physical manifold of robotic action is non-Euclidean and anisotropic. As a result, direct regression approaches fail due to the disjoint metric structures of these domains, making standard distance minimization ill-posed. To resolve this incompatibility, we introduce LAST (Lie-algebraic Action Space Tokenizer). LAST reconstructs the action space to establish a more consistent metric alignment between the VL and Action modalities. Specifically, LAST bridges the heterogeneity via two stages: (1) Global Topological Linearization, which linearizes the action manifold through Lie-algebraic mapping, converting trajectories into a fixed-length, physically additive representation; and (2) Local Metric Discretization, where the representation is discretized hierarchically into schemas and whitened residuals, establishing a mathematical isomorphism with the isotropic Euclidean metric. By addressing the structural mismatch globally and locally, LAST enables VLA models with enhanced convergence and generalizability.

We increasingly want robots to follow natural-language instructions, for example, "place the cup in the basket" or "insert the test tube into the rack." Modern AI systems try to bridge language and action by treating each motion as a sequence of discrete "words" that a language model can predict. But the geometry of physical motion is fundamentally different from the geometry of language: a robot's rotations and translations don't simply add up like ordinary numbers, and motion patterns vary far more lopsidedly than the balanced vectors used by vision–language models. Forcing one shape into the other leads to imprecise control and poor generalization to new situations. This paper introduces LAST, a new way to represent robot motion so that its underlying structure becomes compatible with how vision–language models naturally organize information. LAST first remaps trajectories into a flat space where physical actions can be combined coherently, then statistically rebalances local motion patterns so that small variations look uniform in every direction. The result is a compact "vocabulary" of motion patterns that robot policies can predict and refine with high precision. We test LAST across simulated and real-world manipulation tasks. Robots equipped with LAST learn faster, generalize better to unfamiliar settings, and achieve higher success rates than systems built on prior motion representations. By aligning the geometry of motion with the geometry of language, LAST is a step toward generalist robots that can reliably understand and act on everyday instructions.