ConEx: Human-Interpretable Saliency Maps via Concept-Aware Attribution

Many visual explanation methods in computer vision highlight pixel importance but struggle to link these low-level cues to semantically meaningful concepts, limiting their interpretability and trustworthiness. We introduce Concept-based Explanations (ConEx), a novel framework that bridges saliency visualization with concept-based reasoning to provide both faithfulness and interpretability. ConEx automatically discovers class-specific concepts and represents them through concept activation vectors (CAVs), learned without manual supervision using an architecture-specific masking mechanism that reduces noise introduced by the segmentation masks to enhance concept purity. ConEx generates faithful saliency maps that reveal where each concept appears in the image and how it contributes to the prediction. To evaluate the reliability of these learned concepts, we propose two complementary metrics, Vector-Concept Match (VCM) and Concept-Class Match (CCM), that quantify concept alignment and enable direct comparison with existing methods. Extensive experiments across diverse settings demonstrate that ConEx achieves state-of-the-art performance on faithfulness, segmentation, and concept-quality benchmarks. Overall, ConEx advances the field toward truly interpretable and concept-grounded explanations in vision models.

When artificial intelligence (AI) systems analyze images, it is often difficult to understand exactly why they make a specific decision. While existing tools can highlight which pixels the AI focused on, they fail to explain what those pixels actually represent to the model - for example, whether the AI specifically recognized a bird's "yellow beak" or just a random color pattern. To solve this, we created Concept-based Explanations (ConEx), a new method that makes AI decisions much more transparent. ConEx acts as an automatic translator: it identifies human-understandable visual concepts (like "floppy ears" or "striped fur") and shows exactly where the AI sees them in an image, as well as how much those specific features influenced the final choice. Importantly, our tool discovers these concepts completely automatically, without requiring humans to manually label examples beforehand. By successfully combining where the AI is looking with what the AI is seeing, ConEx makes complex computer vision models more trustworthy and easier to understand.