Can I read Attention Patterns in Multilingual BERT on EtoBox?

Attention Patterns in Multilingual BERT by 임예온 is a document available to read on EtoBox.

What is Attention Patterns in Multilingual BERT about?

This study investigates the capability of multilingual BERT (mBERT) to reflect syntactic structures through its attention mechanisms across 18 languages. It explores whether dependency trees can be decoded from attention patterns, the consistency of attention heads across languages, and the effects of fine-tuning on these patterns. The findings suggest that while attention can capture some linguistic structures, the interpretability of these patterns remains complex and varies across languages.

Author

임예온

Language

EN