Op-CAD: Benchmarking and Investigating Operation-oriented CAD Generation

Recent research has made growing efforts to leverage large language models (LLMs) for computer-aided design (CAD), a domain that demands advanced geometric and spatial reasoning across long operation sequence. However, existing studies remain limited in addressing complex modeling tasks that necessitate step-by-step reasoning, primarily due to the scarcity of high-quality CAD datasets and the absence of fine-grained evaluation frameworks. In response to these challenges, we introduce Op-CAD, the first large-scale, multi-modal dataset for operation-oriented CAD generation, encompassing four operation types and five modalities. Furthermore, we introduce a novel CAD parsing module together with a geometry-guided hierarchical annotation pipeline, which decomposes modeling sequences into discrete operations and substantially improves the annotation accuracy of Vision-Language Models (VLMs). Based on our dataset, we redefine the CAD modeling task by decoupling geometric and spatial perspectives and introduce a novel metric, Chamfer/Fillet Intersection over Union (CF-IoU), to fill the void in assessing chamfer and fillet operations. By comprehensively evaluating eight LLMs on Op-CAD, we establish a benchmark for current models on operation-oriented tasks. Finally, we investigate performance enhancement strategies through fine-tuning on Op-CAD and propose Chain-of-Operation (COOP), a novel prompting strategy that emulates human-engineer reasoning.

This paper introduces Op-CAD, a new benchmark for evaluating how well AI systems can help create CAD models through step-by-step instructions. Instead of asking an AI model to generate a complete design all at once, Op-CAD focuses on whether the model can understand and perform each individual modeling operation, such as adding, cutting, or modifying parts of a 3D object. This better reflects how engineers and designers usually work with CAD software in practice. We also study the strengths and weaknesses of current language and vision-language models on this task. Our results show that existing models still struggle with long modeling processes and precise geometric operations, highlighting the need for more reliable and controllable AI tools for computer-aided design.