AdaMEM: Test-Time Adaptive Memory for Language Agents

A central challenge for language agents is utilizing past experience to adapt to dynamic test-time conditions. While recent work demonstrates the promise of agentic memory mechanisms, most systems restrict retrieval to episode initiation. Consequently, agents are forced to rely on static guidance that becomes increasingly misaligned as long-horizon tasks unfold. To address this rigidity, we propose the Adaptive Memory Agent (AdaMEM), a novel framework for agent test-time adaptation. Without updating model parameters online, AdaMEM adapts agent behavior via a hybrid memory architecture: it maintains a long-term trajectory memory of raw experiences collected offline while generating dynamic short-term strategy memory on-the-fly to guide decision-making. This mechanism enables the trade-off between token efficiency and adaptability across varying inference-time compute levels. Empirically, AdaMEM significantly outperforms static memory baselines, achieving relative gains of up to 13% on ALFWorld and 11% on WebShop, with consistent leading performance extending to agentic search on HotpotQA. To further enhance this adaptation, we develop Step-MFT, a Step-wise Memory Fine-Tuning technique that trains the policy to synthesize high-quality strategies from retrieved experiences, yielding additional performance gains. Our work establishes a new scaling dimension for agentic memory, supporting continuous reasoning and self-evolution post-deployment in real-world environments. Our code is available at https://github.com/yunx-z/AdaMEM.

As AI agents tackle complex tasks like web navigation, they must adapt to unexpected changes. Currently, agents use a memory of past experiences for guidance. However, they typically consult this memory only once at the beginning of a task. This rigid approach leaves them stuck when initial plans fail. We introduce the Adaptive Memory Agent (AdaMEM) to solve this rigidity. Instead of relying on a static initial plan, AdaMEM continuously adapts during the task. It pulls relevant successful experiences from a long-term storage bank and actively writes itself a fresh, state-specific strategy on the fly. Our evaluation shows AdaMEM significantly outperforms older static methods in simulated environments like online shopping and household chores. By empowering AI to update its strategy dynamically without expensive retraining, this framework enables the creation of smarter, more reliable, and highly adaptable real-world AI assistants.