Improving Adversarial Robustness of Attribution via Implicit Regularization

The adversarial robustness of attributions is a fundamental requirement for reliable explainability in deep learning, yet existing approaches typically rely on computationally expensive explicit regularization. In this work, we show that attribution robustness can arise implicitly from the learning dynamics of standard stochastic gradient descent. We theoretically motivate this effect through connections between parameter-space and input-space curvature, and validate it across architectures, datasets, and attribution methods, with negligible computational overhead. In contrast, we prove that such robustness gains often does not transfer to attention-based attribution under softmax normalization, due to inherent entropy constraints, and we validate this limitation experimentally. Finally, we show that replacing softmax attention with kernel-based attention restores the robustness gains in transformer models. Our results highlight learning dynamics as a principled and practical mechanism for robust explainability, and reveal fundamental limitations of attention-based attribution under normalization.

Modern AI systems can make highly accurate predictions, but it is often unclear why they make those decisions. To address this, researchers use explanation methods that highlight which parts of the input influenced a model’s output. However, these explanations can be unstable: even small changes to the input may lead to very different explanations, reducing trust in the system. In this work, we show that stable explanations can emerge naturally during standard training, without requiring additional computational techniques. We provide both theoretical and empirical evidence that the way models are trained already encourages this robustness. At the same time, we identify an important limitation: widely used attention-based explanations in transformer models do not benefit from this effect due to how they normalize information. We show that replacing this mechanism with an alternative design restores robustness. Overall, our results suggest a simpler and more efficient pathway to reliable AI explanations, while highlighting important limitations in commonly used methods.