Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length. Naive truncation or summarization degrades fidelity, while existing context compressors lack cross-turn memory sharing or revision, causing information loss and compounding errors in long dialogues. We revisit the context compression under conversational dynamics and empirically present its fragility. To improve both efficiency and robustness, we introduce Context-Driven Incremental Compression (C-DIC), which treats a conversation as interleaved contextual threads and stores revisable per-thread compression states in a single, compact dialogue memory. At each turn, a lightweight retrieve → revise → write-back loop shares information across turns and updates stale memories, stabilizing long-horizon behavior. In addition, we adapt truncated backpropagation-through-time (TBPTT) to our multi-turn setting, learning cross-turn dependencies without full-history backpropagation. Extensive experiments on long-form dialogue benchmarks demonstrate superior performance and efficiency of C-DIC; notably, C-DIC maintains near-constant inference time and stable perplexity even over hundreds of dialogue turns, supporting a scalable path to high-quality dialogue modeling.

Many AI assistants are useful in short chats but struggle when conversations become long. As a conversation grows, a model often has to reread more and more past messages, which makes responses slower and more expensive, while shortcuts such as cutting off old messages or using fixed summaries can cause the assistant to forget important details. We introduce Context-Driven Incremental Compression, a method that helps an assistant keep a compact, revisable memory of a conversation. Instead of storing one long history, our method organizes the dialogue into topic-based threads, retrieves the memories that are relevant to the current message, updates them when the topic continues, and creates a new memory when the topic changes. This lets the assistant use earlier information without repeatedly processing the entire conversation. In experiments on long multi-session dialogue datasets, our method produced more coherent responses than truncation, summarization, retrieval, and prior compression methods, while keeping response time stable over hundreds of turns. This work is a step toward conversational AI that can remember long discussions more reliably and efficiently.