Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generation

The layout-to-image (L2I) task enables fine-grained control over image generation via object categories and spatial layouts. However, existing L2I methods yield fragmented and distorted generations under few-shot atypical settings. We term this failure as representation fragmentation, arising from a granularity mismatch that entangles semantic identity with visual details. To address this issue, we propose a representation-driven framework that disentangles semantics from primitives for robust few-shot adaptation. Specifically, Semantic Anchoring aggregates categorical semantics into anchors for stable identity, while Primitive Imbuing models recomposable primitives for robust local detail modeling. Conceptual Steering further regulates optimization with a saliency-aware objective to preserve foreground semantic consistency. Extensive experiments demonstrate consistent improvements in the 5-shot regime over state-of-the-art L2I methods in both visual fidelity and alignment across diverse atypical domains. The source code is publicly available at https://github.com/iCVTEAM/DSP.

AI image generators can create complex scenes when guided by user instructions about what objects to generate and where to place them. However, when applied to unusual environments with very little training data, such as murky underwater scenes or high altitude aerial views, these AI models struggle. Instead of drawing coherent objects, they produce distorted and fragmented images. This happens because the AI confuses the core, unchanging identity of an object with its highly variable local visual details. To solve this, we developed a new method that explicitly separates "what an object is" from "what it specifically looks like." Instead of mixing these signals, our system captures the broad meaning and category of an object, while separately processing its local shapes and physical textures. By handling the general concept and the fine details independently, we prevent the AI from getting confused by the limited training examples. By cleanly disentangling these elements, our approach allows the AI to generate highly realistic images for specialized environments using as few as five example images. This makes advanced, controllable image generation accessible and reliable for specialized fields where collecting massive datasets is impossible.