From Blind Spots to Gains: Diagnostic-Driven Iterative Training for Large Multimodal Models

As Large Multimodal Models (LMMs) scale up and reinforcement learning (RL) methods mature, LMMs have made notable progress in complex reasoning and decision making. Yet training still relies on static data and fixed recipes, making it difficult to diagnose capability blind spots or provide dynamic, targeted reinforcement. Motivated by findings that test driven error exposure and feedback based correction outperform repetitive practice, we propose Diagnostic-driven Progressive Evolution (DPE), a spiral loop where diagnosis steers data generation and reinforcement, and each iteration re-diagnoses the updated model to drive the next round of targeted improvement. DPE has two key components. First, multiple agents annotate and quality control massive unlabeled multimodal data, using tools such as web search and image editing to produce diverse, realistic samples. Second, DPE attributes failures to specific weaknesses, dynamically adjusts the data mixture, and guides agents to generate weakness focused data for targeted reinforcement. Experiments on Qwen3-VL-8B-Instruct and Qwen2.5-VL-7B-Instruct show stable, continual gains across eleven benchmarks, indicating DPE as a scalable paradigm for continual LMM training under open task distributions. Our code, models, and data are publicly available at https://github.com/hongruijia/DPE.

Large Multimodal Models that can see and reason — processing images alongside text — are increasingly powerful, but training them remains costly and data-hungry. A promising direction is self-evolving training, where a model generates its own practice problems to keep improving. However, current self-evolving methods suffer from two fundamental problems. First, they lack meaningful self-awareness: rather than identifying why the model fails, they rely on rough signals like prediction confidence, so training resources get wasted on tasks the model has already mastered while genuine weak spots go unaddressed. Second, and equally important, they are visually stuck: because they reuse the same fixed set of images iteration after iteration, the variety of visual situations the model encounters never truly grows. Text questions may change, but the underlying pictures stay the same — meaning rare or complex visual concepts, like unusual chart layouts or specialized diagrams, are perpetually underrepresented. Performance on these long-tail cases plateaus or even degrades over time. DPE addresses both problems together. Before each training round, it systematically tests the model to pinpoint specific capability gaps, then deploys a team of AI agents that actively retrieve new images from the web, edit and compose them to target the identified weaknesses, and generate verified question-answer pairs from this fresh visual material. This diagnosis-generation-training loop repeats, so both the topics and the images keep evolving in response to what the model still struggles with. The result is broad, stable improvement across diverse reasoning tasks — using only a small fraction of the data that conventional approaches require.