Can I read Interpretability at Scale: Identifying Causal Mechanisms in Alpaca on EtoBox?

Interpretability at Scale: Identifying Causal Mechanisms in Alpaca by Wu, Zhengxuan; Geiger, Atticus; Icard, Thomas; Potts, Christopher; Goodman, Noah D. is a scholarly article available to read on EtoBox.

What is Interpretability at Scale: Identifying Causal Mechanisms in Alpaca about?

Obtaining human-interpretable explanations of large, general-purpose language models is an urgent goal for AI safety. However, it is just as important that our interpretability methods are faithful to the causal dynamics underlying model behavior and able to robustly generalize to unseen inputs. Distributed Alignment Search (DAS) is a powerful gradient descent method grounded in a theory of causal abstraction that has uncovered perfect alignments between interpretable symbolic algorithms and small deep learning models fine-tuned for specific tasks. In the present paper, we scale DAS significantly by replacing the remaining brute-force search steps with learned parameters -- an approach we call Boundless DAS. This enables us to efficiently search for interpretable causal structure in large language models while they follow instructions. We apply Boundless DAS to the Alpaca model (7B parameters), which, off the shelf, solves a simple numerical reasoning problem. With Boundless DAS, we discover that Alpaca does this by implementing a causal model with two interpretable boolean variables. Furthermore, we find that the alignment of neural representations with these variables is robust t

Author

Wu, Zhengxuan; Geiger, Atticus; Icard, Thomas; Potts, Christopher; Goodman, Noah D.

Published

2023

Language

EN