UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis

Medical workflows routinely combine reading images with producing visual and textual outputs, making both image understanding and generation central to medical AI. Most existing systems, however, address these abilities in isolated models, losing the shared knowledge that a unified architecture could exploit. To bridge this gap, we present UniMedVL, the first unified medical model that seamlessly integrates multimodal understanding and generation capabilities within a single model without switching weights. We achieve this via a tailored progressive training pipeline where understanding and generation mutually reinforce each other. To effectively train UniMedVL, we curate UniMedVL-5M, the first large-scale medical dataset comprising over 5.6M instances across 8 medical imaging modalities, tailored for multimodal input-output tasks in unified medical understanding and generation. Experimental results demonstrate that UniMedVL achieves competitive performance on five medical understanding benchmarks. Crucially, UniMedVL natively supports diverse interleaved generation tasks, e.g., virtual staining, super-resolution, cross-modal synthesis, essential for complex medical workflows. Our code and dataset are publicly available.

In medical diagnosis, doctors need to simultaneously understand medical images such as X-rays and CT scans and generate various outputs including written reports, image annotations, and comparison images. However, existing artificial intelligence systems typically require multiple specialized models to handle these tasks separately, which is similar to needing different experts to examine the same patient. To address this challenge, we developed UniMedVL, a unified artificial intelligence model capable of both understanding and generating medical images. Like an all-in-one doctor, it uses a single brain to perform multiple tasks without switching between different models. To train this model, we created UniMedVL-5M, the first large-scale medical dataset containing over 5.6 million image samples across 8 different medical imaging modalities. Experimental results show that UniMedVL achieves top-level performance in both understanding and generating medical images. More importantly, the model can naturally handle diverse complex medical tasks such as virtual staining, image enhancement, and cross-modal synthesis, all of which are essential capabilities in modern medical workflows. Our code and dataset are publicly available, with the hope of advancing medical artificial intelligence and making future medical diagnosis more efficient and accurate.