Timezone: »

 
Poster
XAI for Transformers: Better Explanations through Conservative Propagation
Ameen Ali · Thomas Schnake · Oliver Eberle · Grégoire Montavon · Klaus-robert Mueller · Lior Wolf

Tue Jul 19 03:30 PM -- 05:30 PM (PDT) @ Hall E #911
in Poster Session 1 »

Transformers have become an important workhorse of machine learning, with numerous applications. This necessitates the development of reliable methods for increasing their transparency. Multiple interpretability methods, often based on gradient information, have been proposed. We show that the gradient in a Transformer reflects the function only locally, and thus fails to reliably identify the contribution of input features to the prediction. We identify Attention Heads and LayerNorm as main reasons for such unreliable explanations and propose a more stable way for propagation through these layers. Our proposal, which can be seen as a proper extension of the well-established LRP method to Transformers, is shown both theoretically and empirically to overcome the deficiency of a simple gradient-based approach, and achieves state-of-the-art explanation performance on a broad range of Transformer models and datasets.

Keywords: SA: Accountability, Transparency and Interpretability

Author Information

Ameen Ali (Tel Aviv University, Israel)
Thomas Schnake (TU Berlin)
Oliver Eberle (TU Berlin)
Grégoire Montavon (Technische Universität Berlin)
Klaus-robert Mueller (Technische Universität Berlin)
Lior Wolf (Facebook AI Research and Tel Aviv University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors