Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Paper: Transformers learn in-context by gradient descent, published by Lawrence Chan on December 16, 2022 on The AI Alignment Forum.The paper argues that auto-regressive transformers implement in-context learning via gradient-based optimization on in-context data.The authors start by pointing out that with a single linear self-attention (LSA) layer (that is, no softmax), a Transformer can implement one step of gradient descent on the l2 regression loss (a fancy way of saying w -= LR (w x-y)x^T), and confirm this result empirically. They extend this result by showing that an N-layer LSA-only transformer is similar to N-steps of gradient descent for small linear regression tasks, both in and out of distribution. They also find that the results pretty much hold with softmax self-attention (which isn’t super surprising given you can make a softmax pretty linear).Next, they show empirically that the forward pass of a small transformer with MLPs behaves similarly to an meta-learned MLP + one steps of gradient descent on a toy non-linear regression task, again in terms of both in-distribution and OOD performance.They then show how you can interpret an induction head as a single step of gradient descent, and provide circumstantial evidence that this explains some of the in-context learning observed in Olsson et al 2022. Specially, they show that 1) a two layer attention-only transformers converge to loss consistent with one step of GD on this task, and 2) the first layer of the network learns to copy tokens one sequence position over in the first layer, prior to the emergence of in-context learning.(EDIT:) davidad says below:this is strong empirical evidence that mesa-optimizers are real in practicePersonally, while I think you could place this in the same category as papers like RL^2 or In-context RL with Algorithmic Distillation, which also show mesa optimization, I think the more interesting results are the mechanistic ones -- i.e., that some forms of mesa optimization in the model seem to be implemented via something like gradient descent.(Nitpick: I do want to push back a bit on their claim that they've "mechanistically understand the inner workings of optimized Transformers that learn in-context", since they've only really looked at the mechanism of how single attention-only transformers perform in-context learning. )Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.