From Diagrams to Code: Multilingual Programming with Visual Design

In modern software development, particularly in emerging ``vibe coding'' paradigms, project implementation increasingly begins with visual interactions between users and AI coding assistants, where system architectures are communicated through visual designs before coding. This visual-first approach necessitates AI systems capable of interpreting diagrams across multiple programming languages. However, the development of such systems is severely hindered by the lack of large-scale multimodal training data and evaluation benchmarks. To address these limitations, we present M²C-INSTRUCT, a comprehensive multilingual multimodal instruction-tuning dataset containing over 13.1M samples across 50+ programming languages, designed for visual understanding and diagram interpretation in code generation tasks. We validate our dataset by training M²-CODER, a multilingual multimodal software developer that successfully integrates visual design inputs with textual instructions. We also introduce M²EVAL, a novel multilingual evaluation benchmark for multimodal code generation performance. Experiments show our 7B M²-CODER, performs on par with much larger 70B+ models, confirming the quality and effectiveness of our M²C-INSTRUCT. Together, M²C-INSTRUCT, M²-CODER, and M²EVAL provide essential infrastructure for visual-assisted programming in vibe-coding and visual-interactive development workflows.

Modern software development often begins not with code, but with visual designs such as diagrams, workflows, and architecture sketches. However, most AI coding assistants are still mainly trained to understand text prompts, which makes it difficult for them to turn visual software designs into correct programs. This work studies how to build AI systems that can read diagrams and generate code in many programming languages. We introduce M²C-INSTRUCT, a large multilingual multimodal training dataset containing over 13 million examples across more than 50 programming languages. Using this dataset, we train M²-CODER, a model that learns to combine visual design information with written instructions when producing code. We also create M²EVAL, a benchmark for testing whether models can solve diagram-based programming tasks in 10 programming languages. Experiments show that our 7B model performs competitively with much larger models, demonstrating the value of targeted multimodal code training. This work helps move AI coding assistants closer to real development workflows, where people often communicate software ideas through both language and visual design.