ACON: Optimizing Context Compression for Long-horizon LLM Agents

Large language models (LLMs) are increasingly deployed as agents in dynamic real-world environments, where success depends on maintaining precise records of actions and observations. However, the resulting unbounded context growth in long-horizon agentic tasks makes two critical bottlenecks: prohibitive inference memory costs and reasoning degradation due to irrelevant information. Existing compression methods fail to fully address this, often relying on brittle heuristics or requiring parameter updates impractical for proprietary or large-scale LLMs. We introduce Agent Context Optimization (ACON), a unified framework that optimally compresses both observations and history into concise, informative representations. Distinct from prior works, ACON employs an optimization in natural language space: it iteratively refines compression guidelines based on failure analysis of the agent, ensuring critical state information is preserved without model fine-tuning. To further minimize computational overhead, we distill the optimized compressor into smaller models. Experiments on AppWorld, OfficeBench, and Multi-objective QA demonstrate that ACON reduces peak token usage by 26–54\% while improving task success over existing compression baselines. Notably, it enables smaller LMs to function effectively as long-horizon agents, achieving up to 46% performance improvement by mitigating context distraction. Our code is available at https://github.com/microsoft/acon.

Artificial Intelligence (AI) assistants are increasingly used to solve complex, multi-step tasks. However, as they work, they must remember everything they have done and seen, causing their "memory" to grow continuously. This unbounded growth demands massive amounts of computing power and often distracts the AI with irrelevant details, leading to reasoning mistakes. To solve this, we developed a method to smartly compress this memory. Instead of modifying the AI's complex internal parameters, we use plain natural language instructions to teach a "compressor" AI how to summarize the information. By analyzing the AI's past failures, we refine these instructions so the compressor learns to keep only the most crucial facts and discard the clutter. Our approach significantly reduces the memory required, cutting information overload by up to 54%, while maintaining or even improving the AI's success rate. Furthermore, by providing a concise and focused memory, our method enables smaller, more affordable AI models to successfully tackle complex, long-term tasks without getting overwhelmed.