Temporal-aware Flow Matching for Video Generation with Temporally Coherent Motion

Despite rapid advances in text-to-video generation, state-of-the-art generative models still suffer from producing temporally incoherent and unrealistic motion for videos. The key weakness of existing works is that they commonly treat videos as frame sequences and directly adopt Flow Matching (FM) objectives, which are originally designed for images. This practice fails to explicitly model motion priors or temporal dependencies, resulting in suboptimal dynamics that may appear incoherent and unrealistic. To solve this problem, we propose Temporal-aware Flow Matching (TFM), a novel training paradigm that embeds inter-frame constraints into the flow objective, leading to temporally coherent motion modeling in video generation. More specifically, the proposed TFM enforces temporal correlations across frames while retaining the desirable properties of FM, and further introduces a residual-type loss that aligns naturally with this new flow. We theoretically prove that models trained with TFM are able to exhibit remarkably enhanced temporal perception ability. Notably, TFM imposes no additional cost during inference and is applicable to any model using FM. Extensive experiments demonstrate that our TFM can significantly improve motion realism across diverse motion types. Generated videos are presented at https://pzrain.github.io/tfm.

Today’s AI systems can generate short videos from text prompts, but the motion in these videos is often unrealistic or inconsistent over time. For example, objects may suddenly change direction, human movements may look unnatural, or consecutive frames may not match smoothly. We found that a major reason for this problem is that many existing video generation methods were originally designed for images and treat videos simply as collections of separate frames, without explicitly learning how motion evolves across time. To address this issue, we developed a new training method called Temporal-aware Flow Matching (TFM). Instead of learning each frame independently, TFM teaches the model to understand temporal relationships between neighboring frames and encourages motions to evolve consistently throughout the video. We also designed a new learning objective that naturally fits this temporal modeling process. Our experiments show that TFM produces videos with smoother, more realistic motion across many different scenarios, including complex human and object movements. Importantly, TFM improves video quality without increasing generation cost, making it easy to integrate into existing video generation systems.