Timezone: »

 
Poster
Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation
Sahil Singla · Eric Wallace · Shi Feng · Soheil Feizi

Thu Jun 13 06:30 PM -- 09:00 PM (PDT) @ Pacific Ballroom #69
in Posters Thu »

Current saliency map interpretations for neural networks generally rely on two key assumptions. First, they use first-order approximations of the loss function, neglecting higher-order terms such as the loss curvature. Second, they evaluate each feature’s importance in isolation, ignoring feature interdependencies. This work studies the effect of relaxing these two assumptions. First, we characterize a closed-form formula for the input Hessian matrix of a deep ReLU network. Using this formula, we show that, for classification problems with many classes, if a prediction has high probability then including the Hessian term has a small impact on the interpretation. We prove this result by demonstrating that these conditions cause the Hessian matrix to be approximately rank one and its leading eigenvector to be almost parallel to the gradient of the loss. We empirically validate this theory by interpreting ImageNet classifiers. Second, we incorporate feature interdependencies by calculating the importance of group-features using a sparsity regularization term. We use an L0 − L1 relaxation technique along with proximal gradient descent to efficiently compute group-feature importance values. Our empirical results show that our method significantly improves deep learning interpretations.

Keywords: Interpretability · Optimization

Author Information

Sahil Singla (University of Maryland)
Eric Wallace (Allen Institute for Artificial Intelligence)
Shi Feng (University of Maryland)
Soheil Feizi (University of Maryland)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors