StyleDistillation: A New Insight of Image Style Enables Personalized Aesthetic Manipulation

Text-guided stylized image generation has yielded promising advances by leveraging the powerful capabilities of text-to-image diffusion models. However, the inherent coupling of style and content information within the reference image presents a significant challenge. To address this, we propose StyleDistillation, a novel approach grounded in two key observations about the CLIP embedding space from a style perspective. By leveraging a lightweight StyleDistiller module, combined with carefully designed optimization objectives based on geometric and semantic priors, we can extract fine-grained style representation from the reference image. Additionally, we introduce a Prompt Alignment Enhancement mechanism during inference, which significantly improves the control that text prompts exert over the generated images. Extensive experiments demonstrate that our method achieves outstanding performance in both style reproduction and prompt alignment. Furthermore, StyleDistillation supports various personalized operations, including style editing and style fusion, highlighting its substantial potential for diverse applications.

Text-to-image models let ordinary users create images from text and are widely used in digital creation; users often need to control both image content and aesthetic style. However, complex artistic styles are hard to describe in words, especially for non-experts; instead of explaining colors, brushstrokes, and atmosphere, showing a reference image is often more natural. Yet a reference image contains both style and concrete content, such as people, objects, or backgrounds; if a model cannot separate them, it may “learn” these contents with the style, making the result drift from what the user wants. To solve this problem, we propose StyleDistillation, which extracts cleaner style information from a single reference image. Our key observation is that the same style leaves relatively consistent visual traces across images, while text descriptions, though incomplete, help the model judge which information is more like content. Based on this, StyleDistillation separates reference visual information into style-oriented and content-oriented parts, using constraints to preserve color, brushstrokes, lighting, and texture while reducing content interference. Experiments show that it generates images that keep the reference aesthetics and better match the text, while supporting style editing and fusion, giving users more flexible control over personalized image creation.