Breaking Dual Bottlenecks: Evolving Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners
Abstract
Recent unified models integrate multimodal understanding and generation within a single framework. However, an ``understanding-generation gap'' persists, where models can capture user intent but often fail to translate this semantic knowledge into precise pixel-level manipulation. This gap results in two bottlenecks in anything-to-image task~(X2I): the \textbf{attention entanglement bottleneck}, where blind planning struggles with complex prompts, and the \textbf{visual refinement bottleneck}, where unstructured feedback fails to correct imperfections efficiently. In this paper, we propose a novel framework that empowers unified models to autonomously switch between generation strategies based on instruction complexity and model capability. To achieve this, we construct a hierarchical data pipeline that constructs execution paths across three adaptive modes: direct generation for simple cases, self-reflection for quality refinement, and multi-step planning for decomposing complex scenarios. Building on this pipeline, we contribute a high-quality dataset with over 50,000 samples and implement a two-stage training strategy comprising SFT and RL. Specifically, we design step-wise reasoning rewards to ensure logical consistency and intra-group complexity penalty to prevent redundant computational overhead. Extensive experiments demonstrate that our method outperforms existing baselines on X2I, achieving superior generation fidelity among simple-to-complex instructions.
Lay Summary
Modern MLLMs can often understand complex user instructions, but they may still fail when turning that understanding into accurate images or edits. For example, they may miss a logical rule, confuse objects, or introduce visual mistakes while generating an image. This paper proposes a method that lets the model choose different reasoning strategies during generation: it can answer simple requests directly, reflect on and fix visual errors, or break complex instructions into multiple steps. We also build a large training dataset and use post-training methods to improve the model’s ability to follow instructions, maintain consistency, and avoid unnecessary reasoning. Our results show that this approach makes image generation and editing more reliable for complex X2I tasks.