Timezone: »
Despite the rapid advancement of semantic discovery in the latent space of Generative Adversarial Networks (GANs), existing approaches either are limited to finding global attributes or rely on a number of segmentation masks to identify local attributes. In this work, we present a highly efficient algorithm to factorize the latent semantics learned by GANs concerning an arbitrary image region. Concretely, we revisit the task of local manipulation with pre-trained GANs and formulate region-based semantic discovery as a dual optimization problem. Through an appropriately defined generalized Rayleigh quotient, we manage to solve such a problem without any annotations or training. Experimental results on various state-of-the-art GAN models demonstrate the effectiveness of our approach, as well as its superiority over prior arts regarding precise control, region robustness, speed of implementation, and simplicity of use.
Author Information
Jiapeng Zhu (The Hong Kong University of Science and Technology)
Yujun Shen (Ant Group)
Yinghao Xu (Chinese University of Hong Kong)
Deli Zhao (Alibaba Group)
Qifeng Chen (HKUST)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: Region-Based Semantic Factorization in GANs »
Tue. Jul 19th through Wed the 20th Room Hall E #329
More from the Same Authors
-
2023 : Latent Space Editing in Transformer-Based Flow Matching »
Tao Hu · David Zhang · Meng Tang · Pascal Mettes · Deli Zhao · Cees Snoek -
2023 Poster: Cones: Concept Neurons in Diffusion Models for Customized Generation »
Zhiheng Liu · Ruili Feng · Kai Zhu · Yifei Zhang · Kecheng Zheng · Yu Liu · Deli Zhao · Jingren Zhou · Yang Cao -
2023 Poster: RLEG: Vision-Language Representation Learning with Diffusion-based Embedding Generation »
Liming Zhao · Kecheng Zheng · Yun Zheng · Deli Zhao · Jingren Zhou -
2023 Poster: Composer: Creative and Controllable Image Synthesis with Composable Conditions »
Lianghua Huang · Di Chen · Yu Liu · Yujun Shen · Deli Zhao · Jingren Zhou -
2023 Oral: Cones: Concept Neurons in Diffusion Models for Customized Generation »
Zhiheng Liu · Ruili Feng · Kai Zhu · Yifei Zhang · Kecheng Zheng · Yu Liu · Deli Zhao · Jingren Zhou · Yang Cao -
2022 Poster: Principled Knowledge Extrapolation with GANs »
Ruili Feng · Jie Xiao · Kecheng Zheng · Deli Zhao · Jingren Zhou · Qibin Sun · Zheng-Jun Zha -
2022 Spotlight: Principled Knowledge Extrapolation with GANs »
Ruili Feng · Jie Xiao · Kecheng Zheng · Deli Zhao · Jingren Zhou · Qibin Sun · Zheng-Jun Zha -
2021 Poster: Understanding Noise Injection in GANs »
Ruili Feng · Deli Zhao · Zheng-Jun Zha -
2021 Spotlight: Understanding Noise Injection in GANs »
Ruili Feng · Deli Zhao · Zheng-Jun Zha -
2021 Poster: Uncertainty Principles of Encoding GANs »
Ruili Feng · Zhouchen Lin · Jiapeng Zhu · Deli Zhao · Jingren Zhou · Zheng-Jun Zha -
2021 Spotlight: Uncertainty Principles of Encoding GANs »
Ruili Feng · Zhouchen Lin · Jiapeng Zhu · Deli Zhao · Jingren Zhou · Zheng-Jun Zha