Can I read Correlation and Navigation in the Vocabulary Key Representation Space of Language Models on EtoBox?

Correlation and Navigation in the Vocabulary Key Representation Space of Language Models by Peng, Letian; An, Chenyang; Shang, Jingbo is a scholarly article available to read on EtoBox.

What is Correlation and Navigation in the Vocabulary Key Representation Space of Language Models about?

Language model (LM) decoding is based on the next-token prediction (NTP) probability distribution. For neural LMs (e.g., Transformer-based), NTP distribution is essentially a softmax-regularized dot product between an encoded input context (query) and fixed vocabulary representations (keys). In this paper, we study the effect of the key distribution on the NTP distribution, with a focus on whether the similarity between keys will trigger spurious correlations in NTP. Through knowledge-probing tasks, we show that in the NTP distribution, the few top-ranked tokens are typically accurate. However, the middle-ranked prediction is highly biased towards the tokens that are distributionally (not necessarily semantically) similar to these top ones. For instance, if "P" is predicted as the top-1 token, "A"-"Z" will all be ranked high in NTP, no matter whether they can lead to correct decoding results. This hurts the sampling diversity and makes the sampling of correct, long-tail results hopeless and noisy. We attempt to alleviate this issue via a novel in-context method that iteratively pushes the query representation away from explored regions. Specifically, we include the explored decodin

Author

Peng, Letian; An, Chenyang; Shang, Jingbo

Published

2024

Language

EN