MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning

Long-horizon agentic reasoning necessitates effectively compressing growing interaction histories into a limited context window. Most existing memory systems serialize history as text, where token-level cost is uniform and scales linearly with length, often spending scarce budget on low-value details. To this end, we introduce MemOCR, a multimodal memory agent that improves long-horizon reasoning under tight context budgets by allocating memory space with adaptive information density through visual layout. Concretely, MemOCR maintains a structured rich-text memory (e.g., headings, highlights) and renders it into an image that the agent consults for memory access, visually prioritizing crucial evidence while aggressively compressing auxiliary details. To ensure robustness across varying memory budgets, we train MemOCR with reinforcement learning under budget-aware objectives that expose the agent to diverse compression levels. Across long-context multi-hop and single-hop question-answering benchmarks, MemOCR outperforms strong text-based baselines and achieves more effective context utilization under extreme budgets.

AI agents that perform long, multi-step tasks must compress their growing history into a small working memory. Existing approaches store this memory as text, where every piece of information — whether critical or trivial — occupies space proportional to its length. This wastes precious memory budget on unimportant details. We introduce MemOCR, which renders memory as a formatted image rather than a text stream. By controlling visual layout — using large headings for key evidence and small text for secondary details — MemOCR packs more useful information into fewer tokens. We train the system with reinforcement learning under varying memory budgets, teaching it to visually prioritize what matters most. On question-answering benchmarks requiring reasoning over long contexts, MemOCR achieves roughly 8× better memory utilization than text-based methods and degrades far less gracefully when memory is scarce.