Can I read GLM: Autoregressive Blank Infill Model on EtoBox?

GLM: Autoregressive Blank Infill Model by ianchangism is a document available to read on EtoBox.

What is GLM: Autoregressive Blank Infill Model about?

The document proposes a new pretraining framework called General Language Model (GLM) based on autoregressive blank infilling. GLM randomly blanks out spans of text from the input and trains the model to sequentially reconstruct the spans. This approach combines ideas from autoencoding and autoregressive pretraining. The document argues that GLM outperforms prior pretraining methods like BERT, T5, and GPT across tasks of natural language understanding, conditional generation, and unconditional generation.

Author

ianchangism

Language

EN