MoVie: Multimodal Video Compression with Text Guidance

Most deep video codecs emphasize low-level motion modeling and remain largely semantics-agnostic, which can degrade perceptual quality in complex scenes. We propose **MoVie**, a **M**ultim**o**dal **Vi**d**e**o compression framework built on a Text-guided Video Transformer–CNN Mixed block (*Text-VideoTCM*). MoVie adopts a video-centric architecture that jointly models local spatial structures and temporal dynamics via window-based processing, delivering a favorable computation--perception trade-off. To incorporate semantics, we introduce dual-stage text fusion with *Extractor* and *Injector* modules. We further present history-conditioned coding that leverages both previous and aggregated historical frames, and a spatial--channel factorized entropy model that estimates probabilities over spatial neighborhoods and channel groups for adaptive bit allocation. Together, these designs reduce redundancy and improve rate control and temporal coherence, yielding reconstructions at low bitrates. On UVG and MCL-JCV, MoVie achieves **$-$50.23\%** BD-rate for FID and **$-$14.64\%** for LPIPS (VGGNet) relative to HM, while requiring only **55.76\%** of DCVC-FM's per-pixel kMACs. A human perceptual study further confirms consistent subjective preference over strong baselines.

Videos take up a large amount of storage and bandwidth, especially when people watch or share them online. Video compression aims to reduce file size while keeping the video visually clear. However, many existing methods mainly focus on pixel-level changes and motion, and may lose important visual details when the bitrate is very low. This paper introduces MoVie, a new video compression method that uses text descriptions to help preserve the most meaningful parts of a video. For example, if a video contains people, animals, or objects, a short description can guide the model to better understand what content should be kept visually clear. MoVie combines this text guidance with efficient video modeling, so it can maintain better visual quality without greatly increasing computation. Our experiments show that MoVie produces videos that look better at low bitrates than several strong existing compression methods. It also uses less computation than many learned video codecs. In addition to automatic evaluation, a human study shows that viewers more often prefer videos compressed by MoVie over those produced by competing methods. These results suggest that using language as guidance can make video compression more perceptually effective and efficient.