Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning

Monocular 3D shape recovery is fundamental to geometric understanding, yet achieving robust generalization across arbitrary viewpoints and unseen object categories remains a significant challenge. In this paper, we present a generalizable deformation learning framework that reconstructs 3D objects by explicitly deforming a category-level shape template to match the target observation. To address complex shape variations between the template and the target, we introduce a geometry-guided feature modeling mechanism. This process first enriches foundation features with template topology to yield a geometry-aware representation, which is then explicitly correlated with the target observation to guide precise deformation. Furthermore, to bridge the disparity between the fixed template and arbitrary target views, we propose a view-adaptive feature aggregation module. This module leverages multi-view template features and their corresponding camera poses to enrich the canonical template representation, ensuring robust feature alignment regardless of the target's perspective. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods in handling large shape variations and diverse viewpoints, exhibiting strong generalization to novel categories and effectively supporting downstream real-world dexterous robotic manipulation tasks. Project homepage: https://GODeform.github.io/

Creating a 3D model from a single flat image is a major challenge for computers, especially when viewing objects from unusual angles or encountering entirely new types of items. To solve this, we developed an AI system that acts like a digital sculptor. Instead of building a 3D shape from scratch, our program starts with a basic 3D template and learns how to carefully mold and deform it to match the specific object seen in the 2D image. To ensure this works regardless of the camera angle, our system examines the template from multiple viewpoints to better understand how to align it with the target picture. Our approach easily handles large shape variations and outperforms existing methods, even on objects it has never seen before. Ultimately, this enhanced 3D understanding can power real-world applications, such as helping robots accurately see and grasp everyday objects.