VlogReward: Learning Multi-Dimensional Evaluation for Vlog Editing

The rapid rise of vlogs as a personalized storytelling medium has created a demand for automated systems to evaluate and refine vlog editing plans. However, vlog assessment is highly subjective and remains challenging due to a lack of standardized criteria, dataset and benchmark, and effective reward models. To address these challenges, we define a comprehensive vlog evaluation framework guided by professional vlog creators and product managers, establishing a taxonomy of six key dimensions, i.e., Creativity, Consistency, Concept Design, Cinematography, Narration, and Pacing. Subsequently, we curate a large-scale dataset of 100k vlog edits and a dedicated benchmark, VRMBench, to evaluate the vlog rewarding capabilities of Multimodal Large Language Models (MLLMs). Finally, we present VlogReward, a robust vlog reward model that can provide both fine-grained multi-dimensional scores and actionable feedback for iterative refinement. Technically, we enhance the Group Relative Policy Optimization (GRPO) framework by introducing an adjustable inter-group comparison reward, which mitigates the "direction blindness" issue of standard GRPO and enables the model to better distinguish varied-quality edits. VlogReward achieves state-of-the-art results that significantly outperform existing MLLMs, including GPT-5 and Gemini-3-Pro. We hope that our study can help vlog creators and foster automated vlog evaluation and refinement systems.

Video blogs (vlogs) are a hugely popular way to tell stories, but it is incredibly difficult for computers to automatically judge if a vlog is edited well because "good" editing is highly subjective and multi-dimensional. To solve this, we worked with professional creators to define exactly what makes a good vlog, breaking it down into six key areas like creativity, pacing, and camera work. We collected a massive dataset of 100,000 vlog edits and built a testing ground to see how well AI models can evaluate them. We then developed our own AI tool, VlogReward, which scores vlogs across these six areas and gives actionable advice on how to improve them. To achieve this, we improved the underlying way the AI learns, helping it better compare different video edits and avoid getting confused about which version is actually better. Our tool performs exceptionally well, even beating advanced models like GPT-5 and Gemini-3-Pro. We hope this work helps creators improve their videos and leads to smarter, automated video editing software.