Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Auditing games for high-level interpretability, published by Paul Colognese on November 1, 2022 on The AI Alignment Forum. This post was written under Evan Hubinger’s direct guidance and mentorship, as a part of the Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program. Thanks to Evan Hubinger, Arun Jose, and Dane Sherburn for discussions/feedback. Introduction This post aims to extend the auditing games framework to high-level interpretability; a framework for adversarially testing future interpretability methods developed for understanding and working with a model’s internal properties at a high-level (e.g. understanding whether or not a model is performing optimization, etc.) We begin by discussing what we mean by high-level interpretability and why we should care about it. We then introduce an auditing games framework in this context along with some simple examples. We then argue that this framework provides good benchmarks for evaluating our high-level interpretability methods. Finally, we discuss a fundamental difficulty that we encounter when trying to create good tests within the auditing games framework described in this post. What is high-level interpretability and why should we care? Understanding what our models are doing internally seems necessary for us to have confidence that they will perform well during deployment. One key problem in prosaic alignment is that of distributional shifts: the set of models that perform well during and at the end of training may include models that perform poorly, potentially catastrophically, during deployment. Deceptive models fit this description, i.e., a model that performs well during the training process as an instrumental goal so that it can be deployed and pursue its unaligned terminal goal without fear of modification from the training process. Unfortunately, observing the external behavior of models during training can’t help us differentiate between an aligned model and a competent deceptive model. One way we can gain insight into how a model will behave in deployment is by understanding how the model operates internally/what algorithm(s) it’s running. For example, if we could use interpretability tools to look inside a model and understand whether it’s an optimizer, and if so what its terminal goals are, then we would better understand how it’s going to behave in deployment, in particular, whether it’s going to pursue aligned goals or not. Many alignment proposals require the capacity to interpret and work with a model’s internals at a high level, at least to some degree, for us to have confidence that the proposals will work. Some examples of the types of insight/leverage we might want to gain from using interpretability tools include: Knowing whether a model is deceptive. Knowing whether a model is doing optimization/search. If the model is doing optimization, what is it optimizing for? If we have a model that’s supposed to make predictions, how is it making its predictions? If the model is an optimizer with objectives, can we modify its objectives? If the model has a world model, can we modify the knowledge stored in its world model? Developing good benchmarks for interpretability methods that aim to tackle the above questions is important to ensure we’re progressing in the right direction. In the next section, we’ll explore a candidate for such a benchmark. What are auditing games? Auditing games are a framework for adversarially testing our interpretability methods. In this section, we begin by explaining what auditing games are in the context of high-level interpretability. Suppose an interpretability team (the Blue Team) has invented an interpretability method that they claim can do X, where X could be something like “detecting whether a model based on a transformer architectu...