Bridging Tokens and Geometry: Token-wise 3D Supervision for CAD Generation

Computer-Aided Design (CAD) generation is typically formulated as a sequence modeling task over parametric tokens. Recent studies introduce visual information through additional visual inputs or rendering of the final generated programs. However, these methods provide no intermediate visual feedback, hindering the association of individual tokens with their geometric effects. In this work, we propose an Argument-induced 3D Point Loss (A3PL) that maps argument tokens to corresponding 3D points, enabling dense token-wise geometric supervision. To reduce learning complexity and invalid sequences, we further introduce a Grammar-constrained Operator (GCO) that leverages the structured nature of CAD programs to regulate sequence generation. We evaluate our approach on five CAD generation tasks with diverse input modalities, including text, Scalable Vector Graphics (SVG) sketches, point clouds, and CAD sequences. Our approach improves generation accuracy and program validity across different input modalities. Code is available at https://github.com/JumpJumpTigger-GYJ/A3PL.

Imagine describing a product in words and having an AI system automatically generate an editable 3D CAD design. Such technology could help engineers prototype ideas faster and make design tools accessible to more people. However, current AI systems still struggle with understanding how individual design instructions influence the final 3D shape. Many existing methods generate CAD models as sequences of symbols or tokens, similar to generating text. But unlike ordinary text, a small mistake in a CAD sequence can significantly alter the geometry of the final object. Existing approaches often judge the quality of the final generated object but provide little guidance during the intermediate design process. Our work introduces a new approach that teaches AI to understand the geometric consequences of its decisions while generating a CAD model. Instead of waiting until the end to evaluate the design, our method provides fine-grained geometric feedback throughout the generation process. We also introduce grammar-based constraints that help the system follow the rules required for valid CAD construction. Across multiple tasks involving text descriptions, sketches, point clouds, our method generates more accurate and reliable CAD designs. By strengthening the connection between symbolic instructions and geometric understanding, this work takes a step toward AI systems that can become more capable design assistants for engineering and manufacturing applications.