CLEAR: Context-Aware Learning with End-to-End Mask-Free Inference for Adaptive Video Subtitle Removal

Video subtitle removal is essential for content localization and media re-editing, yet existing mask-guided diffusion methods face critical limitations: training inefficiency requiring extensive annotations and full model fine-tuning, inference complexity demanding explicit mask sequences, and static prior utilization unable to adapt to quality variations. We present CLEAR (Context-aware Learning for End-to-end Adaptive subtitle Removal), a lightweight adapter-based framework addressing these challenges through three technical innovations. First, self-supervised prior learning (Stage I) extracts occlusion guidance from video pairs using pixel differences as weak supervision, eliminating annotation dependency while learning generalizable subtitle features across languages. Second, LoRA-based adaptive refinement (Stage II) enables parameter-efficient training that preserves pre-trained visual priors while achieving true mask-free end-to-end inference without external detection modules. Third, adaptive focal weighting dynamically adjusts prior influence based on local quality assessment, effectively handling diverse subtitle styles and noisy guidance signals. Extensive experiments demonstrate CLEAR's superior performance in multilingual subtitle removal while requiring only 0.77% trainable parameters, establishing a new paradigm for efficient video text removal without inference-time mask dependencies.

Videos often have burned‑in subtitles that are hard to remove cleanly while keeping the original background. Existing tools need manual text masks or extra detection steps, which are inconvenient and often leave blurs or text traces. We built CLEAR, a fully automatic video subtitle removal system that works without any masks or extra tools. It uses a two‑stage method: first learning to spot text areas using self‑supervised learning, then fine‑tuning a small part of a video diffusion model to erase text while preserving details. Remarkably, CLEAR works on many languages including English, Korean, Japanese, and French without extra training. It produces sharper, more consistent videos with no flickering or leftover text. This tool makes video editing simpler for creators, platforms, and researchers.