HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control

Current Vision-Language-Action (VLA) models excel at robotic manipulation but often struggle with non-Markovian tasks requiring long-term memory and reasoning due to their reliance on immediate observations. Existing solutions face a frequency-competence paradox, where high-performance models are too slow for real-time control, while faster models lack sufficient reasoning capabilities. To resolve this architectural misalignment, we propose HiMe, a Hierarchical Embodied Memory framework that decouples embodied intelligence into a high-frequency Executor for execution, a Sentry for working memory, and a Planner for long-term strategy. We also introduce a dynamic knowledge system based on cross-modal semantic schemas and active management mechanisms, allowing robots to maintain memory plasticity through "Add, Update, and Delete" operations. This hierarchical design effectively balances the conflict between real-time execution and slow thinking planning, significantly improving success rates in long-horizon tasks. Experiments demonstrate that this approach not only outperforms flat memory baselines but also exhibits the novel ability to self-correct its internal knowledge based on human preferences.

Vision-Language-Action (VLA) models have made robots increasingly capable of following language instructions and performing manipulation tasks. However, they still struggle with long-horizon tasks that require remembering past events, user preferences, or object states that are no longer visible. For example, a robot may need to recall where an object was placed earlier, or remember that a specific user prefers one item over another. Existing robot systems often face a trade-off: stronger reasoning models can handle complex memory and planning, but they are too slow for real-time control, while faster models can act quickly but usually lack sufficient reasoning ability. We propose HiMe, a Hierarchical Embodied Memory framework for long-horizon robotic control. HiMe separates robot intelligence into three parts: a fast Executor that handles real-time actions, a Sentry that monitors task progress and decides when higher-level planning is needed, and a Planner that maintains long-term memory and generates strategies. In addition, HiMe introduces a dynamic memory system that combines visual and textual information and allows the robot to add new knowledge, update outdated knowledge, and delete irrelevant or conflicting knowledge during execution. This hierarchical design allows robots to act efficiently while still using memory and reasoning when necessary. Our experiments show that HiMe improves performance on long-horizon tasks compared with flat memory baselines. It also enables robots to self-correct their internal knowledge when human preferences or environmental states change, making them more reliable in dynamic human-robot interaction scenarios.