Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model

Open-world object detection (OWOD) requires incrementally detecting known categories while reliably identifying unknown objects. Existing methods primarily focus on improving unknown recall, yet overlook interpretability, often leading to known–unknown confusion and reduced prediction reliability. This paper aims to make the entire OWOD framework interpretable, enabling the detector to truly “knowing the unknown.” To this end, we propose a concept-driven InterPretable OWOD framework(IPOW) by introducing a Concept Decomposition Model (CDM) for OWOD, which explicitly decomposes the coupled RoI features in Faster R-CNN into discriminative, shared, and background concepts. Discriminative concepts identify the most discriminative features to enlarge the distances between known categories, while shared and background concepts, due to their strong generalization ability, can be readily transferred to detect unknown categories. Leveraging the interpretable framework, we identify that known–unknown confusion arises when unknown objects fall into the discriminative space of known classes. To address this, we propose Concept-Guided Rectification (CGR) to further resolve such confusion. Extensive experiments show that IPOW significantly improves unknown recall while mitigating confusion, and provides concept-level interpretability for both known and unknown predictions.

Open-world object detection allows AI systems to recognize familiar objects while also discovering objects they have never seen before. However, current methods often mistake unknown objects for known categories because the model cannot clearly explain what features it relies on when making decisions. This makes the predictions less reliable in real-world applications such as autonomous driving and robotics. To address this problem, we design an interpretable detection framework that separates an object’s visual information into different concepts: features that distinguish categories, features shared across objects, and background-related features. This helps the model better understand what makes an object known or unknown, instead of relying on mixed and unclear representations. We further introduce a correction mechanism that reduces cases where unseen objects are incorrectly classified as familiar ones. By making the detection process more transparent and understandable, our work helps AI systems more reliably “know the unknown” when encountering new objects in open-world environments.