PromptRL: Prompt Matters in RL for Flow-Based Image Generation

Flow matching models (FMs) have revolutionized text-to-image (T2I) generation, with reinforcement learning (RL) serving as a critical post-training strategy. We show that current RL pipelines for FMs suffer from two underappreciated limitations: sample inefficiency from collapsed generation diversity, and prompt overfitting, where models memorize training phrasings and collapse on semantically equivalent paraphrases. We present PromptRL (Prompt Matters in RL for Flow-Based Image Generation), a framework that incorporates language models (LMs) as trainable prompt refinement agents directly within the flow-based RL loop, yielding both a learned prompt rewriter and a synergistic training regime that reshapes optimization dynamics. PromptRL attains state-of-the-art scores of 0.97 on GenEval, 0.98 on OCR accuracy, and 24.05 on PickScore, and on instructional editing it improves FLUX.1-Kontext from 1.19 to 1.43 EditReward with only 0.06M rollouts, surpassing Gemini 2.5 Flash Image (Nano Banana, 1.37) and matching ReasonEdit-Think (1.44), which relied on fine-grained data annotations and a multi-stage pipeline. Our experiments show that PromptRL achieves higher performance ceilings with approximately 50% fewer rollouts than naive flow-only RL. Our code is available at https://github.com/G-U-N/UniRL.

Modern diffusion and flow-based image generation models can produce impressive images from text, but improving them with reinforcement learning (RL) is still inefficient and sometimes brittle. One problem is that strong image models often generate very similar outputs for the same prompt, leaving RL with too little diversity to learn from. Another problem is that RL-trained models can become overly sensitive to the exact wording of the prompts used during training, rather than learning the underlying visual intent. This paper introduces PromptRL, a framework that uses a large language model (LLM) as a trainable prompt rewriter inside the RL process. The LLM learns to rewrite prompts in different but semantically consistent ways, giving the image model more diverse and useful training signals while preserving the user’s intent. By jointly training the prompt rewriter and the image generator, PromptRL makes RL more sample-efficient and improves robustness to different phrasings. Experiments show that this approach improves both text-to-image generation and instruction-based image editing, while requiring fewer training samples than standard RL pipelines.