OOVDet: Low-Density Prior Learning for Zero-Shot Out-of-Vocabulary Object Detection

Zero-shot out-of-vocabulary detection (ZS-OOVD) aims to accurately recognize objects of in-vocabulary (IV) categories provided at zero-shot inference, while simultaneously rejecting undefined ones (out-of-vocabulary, OOV) that lack corresponding category prompts. However, previous methods tend to overfit IV classes, resulting in undefined OOV objects being confidently misclassified as semantically similar IV categories. To mitigate this issue, this paper proposes a zero-shot OOV detector (OOVDet), a novel framework that effectively detects predefined classes while reliably rejecting undefined ones in zero-shot scenes. Specifically, due to the model’s lack of prior knowledge about the distribution of OOV data, we synthesize region-level OOV prompts by sampling from the low-likelihood regions of the class-conditional Gaussian distributions in the hidden space, motivated by the assumption that unknown semantics are more likely to emerge in low-density areas of the latent space. For OOV images, we further propose a Dirichlet-based gradient attribution mechanism to mine pseudo-OOV image samples, where the attribution gradients are interpreted as Dirichlet evidence to estimate prediction uncertainty, and samples with high uncertainty are selected as pseudo-OOV images. Building on these synthesized OOV prompts and pseudo-OOV images, we construct the OOV decision boundary through a low-density prior constraint, which regularizes the optimization of OOV classes using Gaussian kernel density estimation in accordance with the above assumption. Experimental results show that our method significantly improves the OOV detection performance in zero-shot scenes. The code is available at \url{https://github.com/binyisu/OOVDet}.

Many modern AI systems can recognize only the object categories they were trained or instructed to detect. In real-world environments, however, images often contain unfamiliar objects that the system has never seen before. These unknown objects may still be mistakenly identified as known categories with high confidence, which reduces the reliability of AI systems in practical applications such as autonomous driving, robotics, and intelligent surveillance. This paper introduces OOVDet, a new framework designed to help AI systems better recognize when an object does not belong to any predefined category. Instead of assuming that every object must match a known label, the method teaches the model to identify regions in its internal feature space where unfamiliar objects are more likely to appear. The framework also estimates prediction uncertainty to automatically discover potential unknown objects during training, even without manually labeled examples.