Trajectory-Stabilized Inference for Diffusion-Based Video Inpainting
Abstract
Video inpainting aims to restore missing regions while preserving spatial and temporal coherence. Diffusion-based methods achieve strong per-frame reconstruction, but their sampling implicitly generates temporally coupled latent trajectories whose long-horizon stability is not explicitly modeled, leading to a trade-off between temporal consistency and structural detail. We revisit video inpainting from the perspective of temporal trajectory stability, viewing temporal inconsistency as instability along time-indexed denoising trajectories rather than an output-level error. Based on this view, we propose an inference-time trajectory stabilization framework that monitors motion-aligned deviation and triggers risk-aware correction only when instability accumulates. It combines sparsely sampled trajectory anchors as stability references with neighborhood-consistent propagation to regulate trajectory evolution while preserving local generative freedom. Implemented as a lightweight control layer in the sampling loop, it selectively contracts unstable trajectories toward motion-consistent manifolds instead of enforcing uniform temporal constraints. Experiments show consistent improvements in temporal coherence and structural fidelity.
Lay Summary
Videos are often edited to remove unwanted objects or repair missing regions, but the filled-in areas must remain believable not only in each frame, but also across time. Recent generative models can create realistic details for single frames, yet their results may still flicker, drift, or gradually change in long videos. This happens because the model does not explicitly check whether the restored content follows the motion of the video in a stable way. We propose a lightweight method that improves video inpainting while the video is being generated. Our method watches how the restored regions move and change over time, and only applies corrections when the content starts to become unstable. This avoids forcing all frames to look overly similar and helps preserve fine visual details. The result is video inpainting that is more consistent across frames while still looking natural. Since our method can be added to existing modern video generation systems without retraining them, it may be useful for video restoration, object removal, and creative editing workflows.