Can I read Day 9 - Transformer Revision on EtoBox?
Day 9 - Transformer Revision by Shabi abbas is a document available to read on EtoBox.
What is Day 9 - Transformer Revision about?
The document discusses tokenization, which involves breaking text into smaller units called tokens, with examples including word, subword, character, and sentence tokenization. It also outlines various AI model tokenization techniques such as GPT, BERT, and RoBERTa, and explains the use of dot products in the self-attention mechanism of Transformer models to measure word similarity. Additionally, it mentions the importance of masking in Transformers to prevent models from accessing future tokens during text
- Author
- Shabi abbas
- Language
- EN