AgentKV: Phase-Aware KV Eviction for Agentic LLMs

Long agentic reasoning creates multi-turn contexts that interleave instructions, reasoning, tool calls, and tool observations, making KV cache memory a major bottleneck for inference. Existing eviction methods score cached keys using anchor queries drawn from the recent context, which works when future attention resembles recent attention. We show that agentic generation violates this assumption: its future query distribution is a phase mixture over \texttt{sys}, \texttt{think}, \texttt{act}, and \texttt{tool}, whose components occupy distinct directions in a shared subspace, so recency anchors overrepresent the current phase and under-score keys needed by other phases. We propose \textsc{AgentKV}, a phase-aware eviction method that maintains a small set of queries per phase and scores cached keys against their union. \textsc{AgentKV} matches recency-based methods at half of their KV budget on $\tau^2$-bench \texttt{retail} tasks and recovers about $90\%$ of the baseline score at $\sim 4{\times}$ compression and on all evaluated tasks.