AnyCanvas: Potential Field Guidance for Training-Free Spatial Control in Text-to-Image Diffusion
Abstract
Diffusion-based text-to-image (T2I) models have demonstrated remarkable advancements in generating high-quality images. However, while real-world applications like product packaging and logo design necessitate synthesis within irregular geometries, existing methods struggle to handle such constraints. Therefore, generating complete pictures that conform to arbitrary-shaped canvas constraints while maintaining semantic integrity remains a significant challenge. To address this, we introduce AnyCanvas, a training-free framework that leverages a Mask-to-Potential Field paradigm to convert binary masks into a differentiable potential field, which guides content to naturally converge within target regions. Extensive experiments demonstrate that AnyCanvas achieves 4.23\% higher spatial adherence to user-specified constraints while maintaining 99.45\% of the semantic fidelity measured by CLIP score, leading to a superior harmonic mean of spatial and semantic metrics. AnyCanvas also exhibits robust generalizability across different model backbones and versatile spatial control objectives.
Lay Summary
Recent artificial intelligence models are capable of creating high-quality images from text descriptions. However, they often struggle to generate images that fit within specific, irregular boundaries, which is a common requirement in real-world tasks like product packaging or logo design. To solve this, we introduce AnyCanvas, a method that helps existing AI models understand and respect these complex shapes. Without requiring costly model retraining, AnyCanvas acts as an invisible spatial guide, naturally directing the AI to draw the requested content inside the target region. Our evaluations show that this approach improves the AI's ability to stay within user-defined layouts while keeping the visual quality and the original meaning of the image intact. Furthermore, AnyCanvas is adaptable and can be integrated into various existing image generation systems.