Mitigating Conversational Inertia in Multi-Turn Agents

Large language models excel as few-shot learners when provided with appropriate demonstrations, yet this strength becomes problematic in multi-turn agent scenarios, where LLMs erroneously mimic their own previous responses as few-shot examples. Through attention analysis, we identify \textbf{conversational inertia}, a phenomenon where models exhibit strong diagonal attention to previous responses, which is associated with imitation bias that constrains exploration. This reveals a tension when transforming few-shot LLMs into agents: longer context enriches environmental feedback for exploitation, yet also amplifies conversational inertia that undermines exploration. Our key insight is that for identical states, actions generated with longer contexts exhibit stronger inertia than those with shorter contexts, enabling construction of preference pairs without environment rewards. Based on this, we propose Context Preference Learning to calibrate model preferences to favor low-inertia responses over high-inertia ones. We further provide context management strategies at inference time to balance exploration and exploitation. Experimental results across eight agentic environments and one deep research scenario validate that our framework reduces conversational inertia and achieves performance improvements.

Large language models are increasingly used as agents that solve tasks through repeated observation and action. In these settings, longer interaction histories can provide useful feedback, but they can also make the model over-rely on its own previous responses. We identify this failure mode as conversational inertia, where the agent tends to imitate earlier response patterns instead of exploring actions better suited to the current state. To address this, we propose two simple ways to reduce this self-imitation. First, we train the model to prefer responses produced from shorter, less biased histories. Second, we introduce a context management strategy that periodically clears old interaction history while keeping recent useful information. Across several agent tasks and a deep research setting, these methods reduce conversational inertia and improve performance. This work helps make long-running AI agents more reliable by allowing them to use past experience without becoming trapped by it.