Specialist VLA: Planner-Routed LoRA Specialization for Long-Horizon Robotic Manipulation
Abstract
Long-horizon robotic manipulation requires chaining semantically distinct primitives, reaching, grasping, moving, and placing, where per-primitive failures compound into end-to-end task breakdown. Existing hierarchical approaches use VLM planners to decompose instructions but route all primitives through a single generalist low-level controller, creating a planner-executor mismatch that limits reliability. We propose Specialist VLA, which retains a shared frozen TinyVLA-1.3B backbone while activating primitive-specific LoRA adapters selected by a Gemini-based planner, with dynamic re-querying every 30 control steps for mid-execution recovery. On a pick-and-place benchmark in Robosuite, Specialist VLA achieves 90% full-task success with re-querying versus 62% for a generalist baseline, demonstrating that primitive-level specialization and adaptive planning are complementary bottlenecks for long-horizon manipulation.