EAKV: An Entropy-Driven Adaptive KV Compression Framework for Long Video Understanding

Although Multimodal Large Language Models (MLLMs) have made remarkable progress, they still struggle with long-video understanding due to the massive memory footprint of KV Caches. Existing methods often resort to disjoint retrieval or attention-based reduction with a uniform layer-wise budget to achieve compression. However, these methods disrupt temporal continuity and ignore the varying information density across network layers. In this work, we reveal that memory allocation should mirror layer-wise semantic density, rather than adhering to a uniform budget. To this end, we introduce EAKV, a training-free entropy-driven adaptive KV compression framework that leverages attention entropy to adaptively allocate compression budgets, selectively preserving critical tokens while distilling redundant contexts into compact contextual anchors, thereby achieving granular memory allocation proportional to semantic density. Extensive experiments on various benchmarks demonstrate that EAKV surpasses existing methods across diverse model architectures and varying parameter scales, yielding improvements ranging from 0.6% to 6.5%.

Advanced AI models can now watch and analyze videos, but they struggle with long ones because storing all that continuous information uses up too much computer memory. Current methods try to save memory by blindly cutting data across the board, which often deletes important details and disrupts the AI's understanding of how the video flows. To fix this, we developed a new, smart memory-saving method called EAKV. Instead of using a "one-size-fits-all" approach to cut data, EAKV acts like a smart highlighter. It automatically assigns more memory to the complex, information-heavy parts of a video, while shrinking repetitive or less important details into tiny summaries. This ensures the AI remembers the crucial moments without getting overwhelmed by useless data. Our tests show that EAKV significantly outperforms existing memory-saving methods across a wide variety of AI models, making long-video analysis much more efficient and accurate.