Can I read Stem on EtoBox?

Stem by guanfelixcaptain is a document available to read on EtoBox.

What is Stem about?

The document introduces STEM (Scaling Transformers with Embedding Modules), a novel approach that enhances the efficiency and interpretability of transformer models by replacing the up-projection in feed-forward networks with a token-indexed embedding lookup. STEM achieves improved training stability, reduces per-token FLOPs and parameter accesses, and enhances long-context performance, leading to accuracy improvements of 3-4% on various benchmarks. This method allows for better knowledge storage and editin

Author

guanfelixcaptain

Language

EN