Hidden in Plain Sight -- Class Competition Focuses Attribution Maps

Attribution methods reveal which input features a neural network uses for a prediction, adding transparency to their decisions. A common problem is that these attributions seem unspecific, highlighting both important and irrelevant features. We revisit the common attribution pipeline and observe that using logits as attribution target is a main cause of this phenomenon. We show that the solution is in plain sight: considering distributions of attributions over multiple classes using existing attribution methods yields specific and fine-grained attributions. On common benchmarks, including the grid-pointing game and randomization-based sanity checks, this improves the ability of 18 attribution methods across 7 architectures up to $2\times$, agnostic to model architecture.

Attribution methods help us understand which parts of an input image influence a neural network’s prediction. However, these methods often highlight not only the truly relevant parts of the image, but also broad, non-specific regions that are not clearly tied to the predicted class. We find that a key reason for this is that standard attribution pipelines often fail to account for how the neural network actually makes a decision for one class over another. To address this, we propose a lightweight solution. Instead of computing attributions for only a single class, we compute attributions for multiple classes and compare their patterns. This comparison allows us to identify which image regions are specifically important for predicting a given class, rather than merely important to the model in general. Used in this way, existing attribution methods produce clearer, more fine-grained explanations of what the model relies on when making its prediction. Across several benchmarks and model types, this approach consistently improves the performance of 18 attribution methods across 7 neural network architectures, with gains of up to 2x.