Unpaired Visual Editing with Self-Consistent Flow Matching
Abstract
Modern generative models possess a deep understanding of visual content, yet training them for image editing typically requires massive datasets of paired examples. This limits scalability, especially for video editing where collecting paired data is prohibitively expensive. We propose a general framework for unpaired training of flow matching editing models. It leverages the base model's knowledge without any external signal. Our approach pairs instruction-following cues extracted from the frozen model with cycle-consistency for structure preservation. To make this tractable, we propose to route gradients from downstream losses over clean predictions to noisy training states. We demonstrate state-of-the-art results on challenging data-scarce image and video editing scenarios. Extensive evaluations and user studies show that our method effectively generalizes to unseen domains and outperforms supervised baselines trained on millions of samples. Analysis reveals that our gradient routing bridges the train-inference gap, and extracting semantic cues from a base model provides a robust training signal that obviates the need for external reward models.
Lay Summary
Generative AI models that create images and videos have learned an enormous amount about how the visual world looks. But teaching one of these models to edit existing images, for instance changing the season in a photo or repainting a landscape in a Minecraft-like style, currently requires huge collections of before-and-after examples. For video, collecting such paired examples at scale is prohibitively expensive and often impossible. We ask whether those paired examples are really necessary. Our method teaches an editing model using only individual images or videos and short text instructions, with no matched before-and-after pairs. It combines two ideas. First, the base model's existing knowledge guides the edit: it already knows what a forest or a cartoon scene looks like, and that knowledge becomes the teacher. Second, every edit must be reversible. If we turn a photo into a cartoon, applying the reverse instruction should recover the original, which forces the editing model to preserve everything the instruction does not explicitly change. Despite never seeing a paired example, our method matches or surpasses leading systems trained on millions of pairs. In a study comparing video edits, people preferred our results 75% of the time, and the method generalizes to unusual visual styles such as Minecraft, voxel art, and Lego, none of which it was specifically trained on. By removing the need for paired data, the approach opens up image and video editing in domains where collecting such data is impractical.