Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation

Unified Multimodal Models (UMMs) integrate both visual understanding and generation within a single framework. Their ultimate aspiration is to create a cycle where understanding and generation mutually reinforce each other. While recent post-training methods have successfully leveraged understanding to enhance generation, the reverse direction of utilizing generation to improve understanding remains largely unexplored. In this work, we propose UniMRG (Unified Multi-Representation Generation), a simple yet effective architecture-agnostic post-training method. UniMRG enhances the understanding capabilities of UMMs by incorporating auxiliary generation tasks. Specifically, we train UMMs to generate multiple intrinsic representations of input images, namely pixel (reconstruction), depth (geometry), and segmentation (structure), alongside standard visual understanding objectives. By synthesizing these diverse representations, UMMs capture rich complementary information regarding appearance, spatial relations, and structural layout. Consequently, UMMs develop a deeper and more comprehensive understanding of visual inputs. Extensive experiments across diverse UMM architectures demonstrate that our method notably enhances fine-grained perception, reduces hallucinations, and improves spatial understanding, while simultaneously boosting generation capabilities.

Modern AI systems are increasingly expected to both understand images and create them, but these two abilities are often improved separately. In unified multimodal models, previous work has mostly used image understanding to make generation better. Our work asks whether generation can also help models understand images better. We introduce UniMRG, a training method that asks a model not only to answer questions about an image, but also to create several useful versions of it: the original image, a depth map showing which parts are nearer or farther away, and a segmentation map showing object regions and boundaries. These extra generation tasks push the model to learn more than colors and textures. They encourage it to notice where objects are, how they are separated, and how they sit in 3D space. We tested UniMRG on several types of unified multimodal models. The trained models became better at spotting small visual details, reasoning about spatial relationships, and avoiding mistakes such as saying an object exists when it does not. They also produced better images. Our results suggest that teaching AI models to create meaningful visual representations can help them understand images more reliably, without changing the model design or making use slower.