Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution

Multimodal Large Language Model (MLLM) agents facilitate Graphical User Interface (GUI) automation but struggle with long-horizon, cross-application tasks due to limited context windows. While memory systems provide a viable solution, existing paradigms struggle to adapt to dynamic GUI environments, suffering from a granularity mismatch between high-level intent and low-level execution, and context pollution where the static accumulation of outdated experiences drives agents into hallucination. To address these bottlenecks, we propose the Darwinian Memory System (DMS), a self-evolving architecture that constructs memory as a dynamic ecosystem governed by the law of "survival of the fittest." DMS decomposes complex trajectories into independent, reusable units for compositional flexibility, and implements Utility-driven Natural Selection to track survival value, actively pruning suboptimal paths and inhibiting high-risk plans. This evolutionary pressure compels the agent to derive superior strategies. Extensive experiments on real-world multi-app benchmarks validate that DMS boosts general-purpose MLLMs without training costs or architectural overhead, achieving average gains of 18.0% in success rate and 33.9% in execution stability, while reducing task latency, establishing it as an effective self-evolving memory system for GUI tasks. The implementation is available at https://anonymous.4open.science/r/DMS-C48C.

Modern AI digital assistants can help automate tasks on smartphones and computers, but they often struggle with long, complex projects that require jumping between multiple applications. Over time, these assistants either run out of memory or get confused by a cluttered buildup of outdated and irrelevant past experiences, causing them to make mistakes. To solve this, we introduced the Darwinian Memory System (DMS), a new way for AI assistants to manage their memories using the natural law of "survival of the fittest". Instead of saving long, rigid sequences of actions that break easily when an app's layout changes, DMS breaks tasks down into small, flexible, and reusable chunks. Acting like a living ecosystem, DMS constantly grades its own memories based on how often they are used, how recently they were active, and whether they are genuinely reliable. It automatically deletes broken or obsolete strategies, updates its memory bank when it discovers a faster way to do things, and actively suppresses risky actions to avoid repeating past mistakes. Our experiments show that DMS significantly boosts the success rate and consistency of various general-purpose AI models while noticeably reducing task delays. Best of all, it achieves these performance leaps instantly without requiring any expensive retraining or modifications to the core AI system.