GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback

Generating executable CAD programs from images requires alignment between visual geometry and symbolic program representations, a capability that current methods fail to learn reliably as design complexity increases. Existing fine-tuning approaches rely on either limited supervised datasets or expensive post-training pipelines, resulting in brittle systems that restrict progress in generative CAD design. We argue that the primary bottleneck lies not in model or algorithmic capacity, but in the scarcity of diverse training examples that align visual geometry with program syntax. This limitation is especially acute because the collection of diverse and verified engineering datasets is both expensive and difficult to scale, constraining the development of robust generative CAD models. We introduce Geometric Inference Feedback Tuning (GIFT), a data augmentation framework that leverages geometric feedback to turn test-time compute into a bootstrapped set of high-quality training samples. GIFT combines two mechanisms: Soft-Rejection Sampling (GIFT-REJECT), which retains diverse high-fidelity programs beyond exact ground-truth matches, and Failure-Driven Augmentation (GIFT-FAIL), which converts near-miss predictions into synthetic training examples that improve robustness on challenging geometries. By amortizing inference-time search into the model parameters, GIFT captures the benefits of test-time scaling while reducing inference compute by 80%. It improves mean IoU by 12% over a strong supervised baseline and remains competitive with more complex multimodal systems, without requiring additional human annotation or specialized architectures.

This paper introduces GIFT, a method that helps AI generate editable 3D design code from a single image more accurately and efficiently. Current systems often struggle when designs become more complex, largely because there are not enough high-quality training examples that connect what an object looks like to the code needed to build it. GIFT addresses this problem by allowing the model to learn from its own attempts: when it produces a different piece of code that still creates the right shape, that example is reused for training, and when it produces a close but imperfect result, that mistake is turned into a new learning opportunity. By learning from both successful alternatives and near misses, the model becomes more reliable, handles difficult shapes better, and reduces the amount of extra computation needed to get strong results.