StructMAR: Structure-Aware Masked Autoregression for Explicit Layout Alignment in Text-to-Image Generation

Although text-to-image generation has achieved significant progress, strict instance-level layout alignment remains a challenge for many applications. Masked Autoregressive (MAR) models on continuous latents are both efficient and high-fidelity, yet the standard practice of flattening 2D latents into 1D sequences weakens spatial topology, limiting precise controllability. To address this, we propose StructMAR, a structure-aware masked autoregressive framework that transforms layout alignment from a soft correlation into an explicit structural alignment. By integrating 2D Rotary Positional Embeddings with a Layout-Guided Attention Bias, StructMAR explicitly biases latent tokens toward their corresponding layout instances during attention computation. We further use Group Relative Policy Optimization (GRPO) as a final-stage policy refinement to reduce the mismatch between the MAR training objective and detector-based layout evaluation metrics. Evaluated on the COCO-Position and COCO-MIG benchmarks, StructMAR achieves state-of-the-art performance, reaching 57.2 AP and 79.4 mIoU on the former, and 61.7 ISR and 56.9 mIoU on the latter. These results, coupled with a 4.05$\times$ inference speedup, underscore the efficacy of explicit structural inductive biases in controllable autoregressive generation.

Text-to-image systems can now create realistic images from written descriptions, but they often struggle when users need precise control over where objects should appear. For example, a user may want a cat in the upper-left corner and a dog on the right, but the generated image may place objects incorrectly, merge them, or ignore smaller objects. This limitation matters for applications such as design, visual content creation, and synthetic data generation, where spatial placement is important. This paper introduces StructMAR, a layout-controlled image generation method built on masked autoregressive generation. This framework is attractive because it can generate images efficiently, but existing masked autoregressive models are not naturally designed to preserve precise spatial relationships after an image is represented as a sequence of visual tokens. StructMAR addresses this limitation by adding explicit layout awareness to the generation process, helping the model maintain the connection between each image region and the object box specified by the user. The key idea is to treat layout information as a strong spatial guide rather than a loose suggestion. StructMAR encourages each image region to stay connected to the object box it is supposed to follow, while still using the text prompt to preserve the correct object meanings. The model is also refined using feedback from object-detection results, helping it better match standard layout evaluation criteria. Experiments show that StructMAR places objects more accurately than strong existing methods on layout-controlled image generation benchmarks, especially in crowded scenes with multiple objects. It also generates images much faster than a strong diffusion-based baseline. Overall, this work shows that masked autoregressive generation can be an efficient and effective foundation for controllable text-to-image generation when precise spatial alignment is required.