Can I read Attention vs non-attention for a Shapley-based explanation method on EtoBox?

Attention vs non-attention for a Shapley-based explanation method by Kersten, Tom; Wong, Hugh Mee; Jumelet, Jaap; Hupkes, Dieuwke is a scholarly article available to read on EtoBox.

What is Attention vs non-attention for a Shapley-based explanation method about?

The field of explainable AI has recently seen an explosion in the number of explanation methods for highly non-linear deep neural networks. The extent to which such methods -- that are often proposed and tested in the domain of computer vision -- are appropriate to address the explainability challenges in NLP is yet relatively unexplored. In this work, we consider Contextual Decomposition (CD) -- a Shapley-based input feature attribution method that has been shown to work well for recurrent NLP models -- and we test the extent to which it is useful for models that contain attention operations. To this end, we extend CD to cover the operations necessary for attention-based models. We then compare how long distance subject-verb relationships are processed by models with and without attention, considering a number of different syntactic structures in two different languages: English and Dutch. Our experiments confirm that CD can successfully be applied for attention-based models as well, providing an alternative Shapley-based attribution method for modern neural networks. In particular, using CD, we show that the English and Dutch models demonstrate similar processing behaviour, but t

Author

Kersten, Tom; Wong, Hugh Mee; Jumelet, Jaap; Hupkes, Dieuwke

Published

2021

Language

EN