LiveFigure: Generating Editable Scientific Illustration with VLM Agents

Scientific illustrations are essential for depicting conceptual designs, methodologies, and experimental workflows in research, playing a pivotal role in communicating complex academic insights. However, creating high-quality scientific illustrations remains a labor-intensive task for human scientists. While recent generative image models have advanced prompt-based editing, the synthesis of fully editable figures remains a fundamental challenge. Valid editability involves structured transformations of graphical elements, scales, attributes, and text, rather than simple pixel-level changes. Existing models generate raster outputs that do not support manual correction or layout adjustment, limiting their utility in scientific publishing, where editable vector figures are typically required for submission. To address this challenge, we introduce LiveFigure, an agentic framework driven by VLM agents that imitates the multi-step drawing workflow of human researchers. It first plans figure blueprints by drawing inspiration from high-quality references in previous works, then generates executable scripts that produce figures via the PowerPoint interface based on skills and experience, and finally refines the outputs with targeted visual diagnostics, producing fully vectorized, editable figures that meet publication standards. Extensive experiments demonstrate that LiveFigure generates inherently editable figures, achieving 80% publication-readiness in only 17 manual edits, far surpassing the 24% rate of the strongest baseline, NanoBanana. Human preference studies further validate this advantage, with LiveFigure securing a 60% win rate against NanoBanana. Our code is available at https://github.com/tsinghua-fib-lab/LiveFigure.git.

Scientific papers often rely on clear illustrations to explain ideas, methods, and workflows. However, making these figures usually takes a lot of time and requires experience with design tools. Recent image generation systems can create attractive pictures, but their outputs are usually fixed images. This makes them hard to correct, resize, or adjust for publication, especially when researchers need to edit individual text labels, arrows, boxes, or other parts of a figure. We introduce LiveFigure, a system that automatically creates scientific illustrations in an editable format. Instead of directly producing a flat image, LiveFigure imitates how researchers design figures: it first looks at high-quality examples for inspiration, then plans the layout, builds the figure using editable objects, and finally checks and improves the result. The output is a PowerPoint file, so researchers can easily modify the figure using familiar tools. Our experiments show that LiveFigure can generate figures that are clearer, more faithful to the intended scientific content, and much easier to edit than figures produced by standard image generation models. In human evaluations, most LiveFigure outputs required only a small number of manual edits before they were considered ready for use in a paper. This work aims to reduce the burden of figure creation and make high-quality scientific communication more accessible to researchers.