GHOST: Geometry-Guided Hallucination of Opaque Surface Textures

Transparent objects pose a fundamental challenge for depth estimation and 3D reconstruction due to their violation of Lambertian assumptions, leading to severe geometry degradation in downstream tasks. To address this, we propose a novel geometry-guided preprocessing framework GHOST that leverages visual foundation models to transform transparent regions into opaque, structurally consistent representations without requiring downstream model retraining. Specifically, our pipeline utilizes (1) TransDINO and (2) TransDecomp to disentangle masks and transparency physical properties, while (3) DAF-Net recovers surface normal priors to encode geometric curvature. Subsequently, (4) GeoSemTransNet integrates these multi-modal cues to synthesize a texture-rich opaque RGB image that preserves the transparent object's 3D structure. Extensive experiments demonstrate that our method significantly enhances the accuracy of state-of-the-art depth estimation and reconstruction models on transparent objects by restoring essential photometric cues.

Creating 3D models or measuring the depth of see-through objects (like glass or transparent plastic) is challenging. These objects reflect light differently than solid ones, making it hard for computer systems to get accurate 3D information, which hurts related tasks like virtual modeling. To solve this, we developed GHOST, a new tool that turns see-through object parts into non-see-through ones, while keeping the object’s true 3D shape—no need to retrain existing systems. It identifies see-through areas, analyzes how light passes through them, and adds realistic details to make these areas easier for computers to process. Our tests show this tool significantly improves the accuracy of 3D modeling and depth measurement for see-through objects.