About this document
CS336 Assignment 1: Transformer LM Basics by rainnamratasg is a document available to read on EtoBox.
The CS336 Assignment 1 requires students to build components for training a Transformer language model from scratch. Key tasks include implementing a Byte-pair encoding tokenizer, a Transformer LM, a cross-entropy loss function, and a training loop. Students will train the tokenizer and the LM using the TinyStories dataset and generate samples from the trained model.
- Author
- rainnamratasg
- Language
- EN