Can I read Formal Algorithms for Transformers on EtoBox?

Formal Algorithms for Transformers by Phuong, Mary; Hutter, Marcus is a scholarly article available to read on EtoBox.

What is Formal Algorithms for Transformers about?

This document aims to be a self-contained, mathematically precise overview of transformer architectures and algorithms (*not* results). It covers what transformers are, how they are trained, what they are used for, their key architectural components, and a preview of the most prominent models. The reader is assumed to be familiar with basic ML terminology and simpler neural network architectures such as MLPs.

Author

Phuong, Mary; Hutter, Marcus

Published

2022

Language

EN