Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Precursor checking for deceptive alignment, published by Evan Hubinger on August 3, 2022 on The AI Alignment Forum. This post is primarily an excerpt from “Acceptability Verification: a Research Agenda” that I think is useful enough on its own such that I’ve spun it off into its own post. The central idea of this section in the original agenda document is to understand the necessary desiderata for doing precursor checking for deceptive alignment. The basic idea of precursor checking here is that, if you want to prevent deceptive alignment from ever arising in the first place—e.g. because you think it’ll be too difficult to detect after the fact—you need to find some condition to look for instead that rules out the possibility of deceptive alignment. In the language of this post, I’ll refer to the precursor we’re looking for as an acceptability predicate, with the idea being that it’s some predicate that determines whether a model is “acceptable” in the sense that it excludes the possibility of deceptive alignment. Thus, the goal here is to understand what desiderata such an acceptability predicate would have to satisfy. Acceptability desiderata What follows is a near-verbatim excerpt from “Acceptability Verification: a Research Agenda.” Thus, the below writing is primarily from around 2020. Given that significant scaling of transparency and interpretability is possible, if we want to do acceptability verification we still need to figure out the very important question of what we need those interpretability tools to be able to understand about our models—that is, what acceptability predicate should we be checking? To answer this question, the first thing we need to do is define what makes an acceptability predicate good—that is, what are the desiderata that we want our acceptability predicate to satisfy? We’ll start by making some definitions. Let M be the full model space and we’ll define the following three predicates on that model space. Let S:MB represent whether the model actually avoids whatever problematic thing we’re trying to avoid (e.g. deception). The S here stands for “safe,” though note that it’s a very weak form of safety. The idea here is that we should get actual safety—that is, the model is not just non-catastrophic but also trying to do what we want—by adding in good (behavioral) performance on the original loss function. In other words, acceptability verification is just trying to solve the problem of worst-case guarantees—but you still need standard outer alignment techniques for getting average-case guarantees. We’ll use the general term “non-catastrophic” to describe models that satisfy S, but in practice S could refer to any number of possible good properties that we’re trying to achieve. The most important thing about S is that we will purposefully be trying not to assume anything about models that don’t satisfy S—thus, we can also think about ¬S as referring to those models such that, once we get them, we don’t know how to verify any properties of them (e.g. because they might be trying to deceive us). Let A:MB be the acceptability predicate under consideration. That is, we want A to be some condition such that we think we’ll be able to get S by having an overseer attempt to verify whether or not A holds. A simple example of such an acceptability predicate might be something like corrigibility, though the canonical example that I usually imagine here is myopia. Note that A only refers to whether the model actually satisfies the condition—it says nothing about whether we’re able to detect that fact. Let OA:MB be the evaluation of whether the acceptability predicate A holds according to whatever sort of acceptability verification system we’re using, perhaps an amplified overseer or perhaps just engineers looking at transparency tools. The O here...