Can I read Beyond Components Singul on EtoBox?
Beyond Components Singul by kurvadimal is a document available to read on EtoBox.
What is Beyond Components Singul about?
This document presents a novel approach to interpretability in transformer models by decomposing attention heads and MLP layers into orthogonal singular directions, revealing independent computations within these components. The authors validate their method on standard tasks, demonstrating that transformer computations are more distributed and structured than previously assumed, with overlapping subfunctions encoded within low-rank subspaces. This perspective enhances mechanistic interpretability and sugge
- Author
- kurvadimal
- Language
- EN