PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards

Text-to-video (T2V) generation aims to synthesize videos with high visual quality and temporal consistency that are semantically aligned with input text. Reward-based post-training has emerged as a promising direction to improve the quality and semantic alignment of generated videos. However, recent methods either rely on large-scale human preference annotations or operate on misaligned embeddings from pre-trained vision-language models, leading to limited scalability or suboptimal supervision. We present PISCES, an annotation-free post-training algorithm that addresses these limitations via a novel Dual Optimal Transport (OT)-aligned Rewards module. To align reward signals with human judgment, PISCES uses OT to bridge text and video embeddings at both distributional and discrete token levels, enabling reward supervision to fulfill two objectives: (i) a Distributional OT-aligned Quality Reward that captures overall visual quality and temporal coherence; and (ii) a Discrete Token-level OT-aligned Semantic Reward that enforces semantic, spatio-temporal correspondence between text and video tokens. To our knowledge, PISCES is the first to improve annotation-free reward supervision in generative post-training through the lens of OT. Experiments on both short- and long-video generation show that PISCES outperforms both annotation-based and annotation-free methods on VBench across Quality and Semantic scores, with human preference studies further validating its effectiveness. We show that the Dual OT-aligned Rewards module is compatible with multiple optimization paradigms, including direct backpropagation and reinforcement learning fine-tuning.

Text-to-video systems can now create short videos from written prompts, but they often still miss important details: the wrong number of objects may appear, actions may not match the text, or the video may look unstable across time. Improving these systems usually requires large amounts of human feedback, where people compare generated videos and label which ones are better. This process is expensive, slow, and difficult to scale. In this paper, we introduce PISCES, a method for improving text-to-video models without relying on costly human annotations. Instead of asking people to label many videos, PISCES uses signals from existing video-and-language models to judge whether a generated video looks realistic, moves coherently, and matches the text prompt. A key challenge is that these existing models often represent text and video in slightly mismatched ways, making their feedback imperfect. PISCES addresses this by better aligning the text and video representations before using them as training feedback. Our experiments show that PISCES improves both the visual quality of generated videos and how faithfully they follow text prompts, for both short and longer video generation. Human preference studies also show that people tend to prefer videos improved by PISCES over strong existing baselines. This suggests a scalable path toward better text-to-video generation, especially without expensive human annotations.