ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management

LLM-based multi-agent simulations are increasingly adopted across application domains, but remain difficult to scale due to GPU memory pressure. Each agent maintains private GPU-resident states, including models, prefix caches, and adapters, which quickly exhaust device memory as the agent count grows. We identify two key properties of these workloads: sparse agent activation and an estimable agent invocation order. Based on an analysis of representative workload classes, we introduce invocation distance, a unified abstraction that estimates the relative order in which agents will issue future LLM requests. Leveraging this abstraction, we present ScaleSim, a memory-efficient LLM serving system for large-scale multi-agent simulations. ScaleSim enables proactive prefetching and priority-based eviction, supports diverse agent-specific memory through a modular interface, and achieves up to 1.74$\times$ speedup over SGLang on simulation benchmarks. ScaleSim's source code is available at https://github.com/PanZaifeng/KVFlow.

Large language model-based multi-agent simulations are increasingly used to study systems with many interacting entities, such as societies, robots, and economic environments. However, scaling these simulations is difficult because each agent keeps its own memory on GPUs, and GPU memory quickly becomes a bottleneck. Existing serving systems do not know which agents will need the language model next, so they often make inefficient memory decisions and trigger unnecessary data transfers. We observe that in many simulations, only a small portion of agents actively use the language model at any moment, and the future activation order of agents can often be estimated. Based on this observation, we introduce invocation distance, a relative measure that compares which agents are expected to invoke the language model earlier. ScaleSim uses this information to preload memory proactively and evict memory that is less likely to be reused soon. Experiments across multiple simulation workloads show that ScaleSim reduces memory transfer overhead and improves execution efficiency. Compared with existing systems, it achieves up to 1.74$\times$ higher performance and makes large-scale multi-agent simulation more practical on limited hardware.