TAGRPO: Boosting GRPO on Image-to-Video Generation with Direct Trajectory Alignment

Recent studies have demonstrated the efficacy of integrating Group Relative Policy Optimization (GRPO) into flow matching models, particularly for text-to-image and text-to-video generation. However, we find that directly applying these techniques to image-to-video (I2V) models often fails to yield consistent reward improvements. To address this limitation, we present TAGRPO, a robust post-training framework for I2V models inspired by contrastive learning. Our approach is grounded in the observation that rollout videos generated from identical initial noise provide superior guidance for optimization. Leveraging this insight, we propose a novel GRPO loss applied to intermediate latents, encouraging direct alignment with high-reward trajectories while maximizing distance from low-reward counterparts. Furthermore, we introduce a memory bank for rollout videos to enhance diversity and reduce computational overhead. Despite its simplicity, TAGRPO achieves significant improvements over DanceGRPO in I2V generation.

AI tools can now bring static images to life by turning them into videos, but teaching these models exactly what humans find visually pleasing is remarkably difficult. Current training methods struggle because they grade each generated video in isolation, missing the bigger picture of how different attempts compare to one another. To address this, we developed a new training approach called TAGRPO. Instead of evaluating videos one by one, we have the AI compare a batch of videos generated from the exact same starting point. We teach the AI to directly copy the internal "creation path" of the best video in the batch, while actively steering away from the worst one. We also gave the AI a "memory bank" of past attempts to learn from, which saves massive amounts of computing power. Our approach significantly boosts the visual quality, motion smoothness, and overall realism of AI-generated videos. By making the training process smarter and more efficient, this work makes high-quality video generation more reliable and accessible for creators, animators, and the visual effects industry.