Turning Adaptation into Assets: Cross-Domain Bridging for Online Vision-Language Navigation

Navigating under non-stationary environment shifts poses a critical challenge for a Vision-and-Language Navigation (VLN) agent deployed in the wild. Yet, existing Test-Time Adaptation (TTA) methods for VLN largely treat online adaptation as transient, isolated updates, leading to catastrophic forgetting and negative transfer. To overcome these issues, we propose Inter-Domain BridgE with Historical Assets (IDEA), a novel TTA framework that transforms adaptation into the accumulation and composition of assets. Specifically, IDEA introduces soft prompts optimized via a Fisher-guided weighting scheme to capture the transferable knowledge. These optimized prompts are then augmented with domain coordinates to form a dynamic asset library. Leveraging this library, IDEA constructs a cross-domain bridge by projecting the target domain onto the convex hull of historical knowledge. These designs form a complementary loop: the evolving library underpins bridge construction, while the bridge provides superior initialization to accelerate asset optimization. Extensive experiments across REVERIE, R2R, and R2R-CE benchmarks demonstrate the consistent superiority of IDEA over existing methods, showcasing its ability to enable training-free adaptation via asset sharing.

Robots and AI assistants that follow language instructions need to work in many different places, such as homes, offices, or other indoor spaces. However, a system trained in one environment may perform poorly when it is used in a new place with different layouts, lighting, objects, or visual appearances. Existing methods try to adjust the system while it is being used, but they often forget what they learned before or apply past experience in the wrong situation. This paper proposes IDEA, a new way for navigation systems to turn past adaptation experience into reusable “assets.” Instead of treating each new environment as a separate challenge, IDEA stores useful knowledge from previous environments and reuses it when facing related new ones. It can also combine several past experiences to form a better starting point for the current environment, reducing the need for repeated trial-and-error updates. Experiments on several navigation benchmarks show that IDEA helps different navigation models perform more reliably in both familiar and unfamiliar environments. The results suggest that reusing accumulated experience can make instruction-following navigation agents more efficient, stable, and practical for real-world deployment.