CADFit: Precise Mesh-to-CAD Program Generation with Hybrid Optimization

Despite recent progress, recovering parametric CAD construction sequences from geometric input, such as meshes or point clouds, is a key challenge for design and manufacturing, as existing CAD reconstruction and generation methods are largely restricted to difficult-to-edit formats like meshes or Breps or editable simple sketch-and-extrude pipelines and low-complexity datasets. We introduce CADFit, a hybrid optimization-based CAD reconstruction framework that recovers complex, editable CAD construction sequences from meshes by incrementally fitting and validating parametric operations using geometric feedback. Our approach is distinguished by formulating reconstruction as an IoU-driven optimization over structured CAD programs and supporting a rich set of operations, including extrusions, revolutions, fillets, and chamfers. Experiments on multiple CAD benchmarks show that CADFit outperforms state-of-the-art mesh-to-CAD methods in volumetric Intersection-over-Union and Chamfer Distance, while substantially reducing the Invalid Ratio of reconstructed CAD programs, particularly for complex designs. We further present a multimodal pipeline that enables end-to-end reconstruction of CAD construction sequences from images by combining image-based geometry reconstruction with CADFit. By enabling accurate reconstruction of higher-complexity CAD models, CADFit provides a practical foundation for generating richer datasets and advancing future learning-based approaches to CAD reverse engineering. The code is available at: https://github.com/ghadinehme/CADFit.

When engineers design a mechanical part, their CAD software records every step: draw a shape, stretch it into a solid, cut a hole, round an edge. This history is what makes a design easy to tweak later. But what if you only have a 3D scan or photo of an existing part, can you recover those steps? Existing AI approaches try to guess the full sequence of steps in one shot, but they only handle simple operations and frequently produce programs that crash when you try to run them. CADFit takes a different approach. It starts by slicing through the 3D shape to extract flat cross-sections, the kinds of 2D profiles an engineer would sketch as the starting point for each operation. This produces hundreds of candidates, most of them useless, so a lightweight AI model scores each one by how likely it is to appear in the final design, keeping only the promising ones. CADFit then tests each surviving candidate against the target shape using real engineering software, keeps only what improves the fit, and repeats on whatever geometry is still missing. This trial-and-error strategy lets it handle a much richer set of design operations while guaranteeing every output is a valid, runnable program. Across benchmarks of varying complexity, CADFit substantially outperforms prior methods and often recovers designs more concise than the original human-authored ones.