Can I read Lec 14 on EtoBox?
Lec 14 by howe108 is a document available to read on EtoBox.
What is Lec 14 about?
The lecture covers the architecture of transformer models, focusing on components such as self-attention, position embeddings, layer normalization, and feedforward layers. It discusses the process of language modeling with decoder-only transformers, emphasizing the training objective of predicting the next token based on preceding tokens and the use of cross-entropy loss for modeling loss. The session also includes a recap of attention mechanisms and the importance of maintaining word order information in t
- Author
- howe108
- Language
- EN