Benchmarking and Improving Fine-Grained Text-to-Image Alignment via Paired Reinforcement Learning

While recent autoregressive models have achieved text-to-image generation performance comparable to diffusion models, they significantly struggle with fine-grained semantic alignment. To rigorously evaluate this limitation, we introduce DeltaBench, a benchmark featuring paired prompts with subtle fine-grained differences, which reveals that existing models fail to achieve precise control over visual tokens. To bridge this gap, we propose FocusDiff, a comprehensive framework that enhances alignment by learning from subtle differences in similar text-image pairs. Specifically, we construct FocusDiff-Data, a large-scale dataset of paired samples derived from image editing tasks to capture localized semantic shifts. Furthermore, we introduce Pair-GRPO, an improved reinforcement learning algorithm that extends GRPO to paired samples. Extensive experiments demonstrate that our approach outperforms most prior prominent methods on both DeltaBench and existing benchmarks.

In this paper, we developed FocusDiff, a training method that teaches the AI by comparing pairs of very similar pictures and descriptions side by side, so it learns exactly which words control which visual details. We also collected a large set of such paired examples and designed a learning rule that rewards the model when it correctly reacts to small changes in the prompt. As a result, our model follows fine-grained instructions noticeably better than previous approaches, making AI image generation more controllable and trustworthy for real-world use.