Object-level Semantic and Spatial Distillation for Open Vocabulary Detection

Recent Open-vocabulary Object Detection (OVD) approaches adapt CLIP through region-level distillation to improve semantic alignment for novel categories. However, the distilled regional features are often used for both classification and localization, enhancing semantic consistency at the expense of spatial fidelity. To resolve this, we propose Object-level Semantic and Spatial Distillation (OSSD), a two-stage framework that explicitly decouples semantic and spatial feature learning. OSSD first distills object-level semantics from CLIP’s global [CLS] embeddings to enhance region discrimination, and then injects fine-grained spatial and structural priors via spatial distillation from a detector trained only on COCO base categories. Furthermore, we propose a Location Quality Estimation Head (LQEH) that predicts class-agnostic localization quality, complementing objectness confidence to improve the novel-object perception. Extensive experiments show that our method achieves 49.2 AP50 on the OV-COCO benchmark. exceeding the best previous result by 3.6\%, On the OV-LVIS benchmark, our method reaches 40.5 mAP on novel categories, outperforming previous state-of-the-art methods.

Computer vision systems are widely used to identify and locate objects in images. Traditionally, these systems can only recognize a fixed set of items they were explicitly trained on. To break this limitation, recent research uses "Open-vocabulary Object Detection" (OVD) to match images with open-ended text, allowing computers to recognize unseen, novel objects (such as rare animals or unique tools) by learning from vast internet data. However, existing methods often struggle to balance "what an object is" (semantics) with "where it is" (localization). When models try to learn rich language descriptions from AI foundational models like CLIP, they often lose precision in drawing the bounding boxes around objects. To solve this conflict, this paper introduces a new training framework named OSSD (Object-level Semantic and Spatial Distillation). Our key idea is "decoupling": we split the learning process into two specialized stages. First, the model focuses purely on understanding the meaning of objects using global AI knowledge. Second, it learns high-precision positioning skills from a standard detector that excels at pinpointing shapes. Additionally, we designed a smart quality checker that estimates how accurately a box covers an object, significantly reducing false detections. Our method achieves state-of-the-art performance on standard industry benchmarks, outperforming previous methods by a large margin. This research paves the way for more adaptable and precise vision systems in real-world scenarios, such as robotics and autonomous driving, where encountering unfamiliar objects is inevitable.