Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some open-source dictionaries and dictionary learning infrastructure, published by Sam Marks on December 5, 2023 on The AI Alignment Forum.As more people begin work on interpretability projects which incorporate dictionary learning, it will be valuable to have high-quality dictionaries publicly available.[1] To get the ball rolling on this, my collaborator (Aaron Mueller) and I are:open-sourcing a number of sparse autoencoder dictionaries trained on Pythia-70m MLPsreleasing our repository for training these dictionaries[2].Let's discuss the dictionaries first, and then the repo.The dictionariesThe dictionaries can be downloaded from here. See the sections "Downloading our open-source dictionaries" and "Using trained dictionaries" here for information about how to download and use them. If you use these dictionaries in a published paper, we ask that you mention us in the acknowledgements.We're releasing two sets of dictionaries for EleutherAI's 6-layer pythia-70m-deduped model. The dictionaries in both sets were trained on 512-dimensional MLP output activations (not the MLP hidden layer like Anthropic used), using ~800M tokens from The Pile.The first set, called0_8192, consists of dictionaries of size 8192=16512. These were trained with an L1 penalty of1e-3.The second set, called1_32768, consists of dictionaries of size 32768=64512. These were trained with an l1 penalty of3e-3.Here are some statistics. (See our repo's readme for more info on what these statistics mean.)For dictionaries in the0_8192 set:LayerMSE LossL1 lossL0% Alive% Loss Recovered00.0566.1329.9510.9980.98410.0896.67744.7390.8870.92420.10811.4462.1560.5870.86730.13523.773175.3030.5880.90240.14827.084174.070.8060.92750.17947.126235.050.6720.972For dictionaries in the1_32768 set:LayerMSE LossL1 lossL0% Alive% Loss Recovered00.094.322.8730.1740.94610.132.79811.2560.1590.76820.1526.15116.3810.1180.72430.21111.57139.8630.2260.76540.22213.66529.2350.190.81650.26526.443.8460.130.931And here are some histograms of feature frequencies.Overall, I'd guess that these dictionaries are decent, but not amazing.We trained these dictionaries because we wanted to work on a downstream application of dictionary learning, but lacked the dictionaries. These dictionaries are more than good enough to get us off the ground on our mainline project, but I expect that in not too long we'll come back to train some better dictionaries (which we'll also open source). I think the same is true for other folks: these dictionaries should be sufficient to get started on projects that require dictionaries; and when better dictionaries are available later, you can swap them in for optimal results.Some miscellaneous notes about these dictionaries (you can find more in the repo).The L1 penalty for1_32768 seems to have been too large; only 10-20% of the neurons are alive, and the loss recovered is much worse. That said, we'll remark that after examining features from both sets of dictionaries, the dictionaries from the1_32768 set seem to have more interpretable features than those from the0_8192 set (though it's hard to tell).In particular, we suspect that for0_8192, the many high-frequency features in the later layers are uninterpretable but help significantly with reconstructing activations, resulting in deceptively good-looking statistics. (See the bullet point below regarding neuron resampling and bimodality.)As we progress through the layers, the dictionaries tend to get worse along most metrics (except for % loss recovered). This may have to do with the growing scale of the activations themselves as one moves through the layers of pythia models (h/t to Arthur Conmy for raising this hypothesis).We note that our dictionary fea...