Timezone: »

 
Latent Space Editing in Transformer-Based Flow Matching
Tao Hu · David Zhang · Meng Tang · Pascal Mettes · Deli Zhao · Cees Snoek

@
in New Frontiers in Learning, Control, and Dynamical Systems »
Event URL: https://openreview.net/forum?id=Bi6E5rPtBa »
This paper strives for image editing via generative models. Flow Matching is an emerging generative modeling technique that offers the advantage of simple and efficient training. Simultaneously, a new transformer-based U-ViT has recently been proposed to replace the commonly used UNet for better scalability and performance in generative modeling. Hence, Flow Matching with a transformer backbone offers the potential for scalable and high-quality generative modeling, but their latent structure and editing ability are as of yet unknown. Hence, we adopt this setting and explore how to edit images through latent space manipulation. We introduce an editing space, we call $u$-space, that can be manipulated in a controllable, accumulative, and composable manner. Additionally, we propose a tailored sampling solution to enable sampling with the more efficient adaptive step-size ODE solvers. Lastly, we put forth a straightforward yet powerful method for achieving fine-grained and nuanced editing using text prompts. Our framework is simple and efficient, all while being highly effective at editing images while preserving the essence of the original content.We will provide our source code and include it in the appendix.

Author Information

Tao Hu (University of Amsterdam)
David Zhang (University of Amsterdam)
Meng Tang (University of California, Merced)
Pascal Mettes (University of Amsterdam)
Deli Zhao (Alibaba Group)
Cees Snoek (University of Amsterdam)

More from the Same Authors