Can I read Do Word Embeddings Really Understand Loughran-McDonald's Polarities? on EtoBox?

Do Word Embeddings Really Understand Loughran-McDonald's Polarities? by Li, Mengda; Lehalle, Charles-Albert is a scholarly article available to read on EtoBox.

What is Do Word Embeddings Really Understand Loughran-McDonald's Polarities? about?

In this paper we perform a rigorous mathematical analysis of the word2vec model, especially when it is equipped with the Skip-gram learning scheme. Our goal is to explain how embeddings, that are now widely used in NLP (Natural Language Processing), are influenced by the distribution of terms in the documents of the considered corpus. We use a mathematical formulation to shed light on how the decision to use such a model makes implicit assumptions on the structure of the language. We show how Markovian assumptions, that we discuss, lead to a very clear theoretical understanding of the formation of embeddings, and in particular the way it captures what we call frequentist synonyms. These assumptions allow to produce generative models and to conduct an explicit analysis of the loss function commonly used by these NLP techniques. Moreover, we produce synthetic corpora with different levels of structure and show empirically how the word2vec algorithm succeed, or not, to learn them. It leads us to empirically assess the capability of such models to capture structures on a corpus of around 42 millions of financial News covering 12 years. That for, we rely on the Loughran-McDonald Sentime

Author

Li, Mengda; Lehalle, Charles-Albert

Published

2021

Language

EN