Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Eliciting Latent Knowledge Via Hypothetical Sensors, published by John Maxwell on December 30, 2021 on The AI Alignment Forum. This is a response to ARC's first technical report: Eliciting Latent Knowledge. But it should be fairly understandable even if you didn't read ARC's report, since I summarize relevant parts of their report as necessary. Here I propose some approaches to the problem ARC outlines which are very different from the approaches they explore. Idea #1: Detecting failure using a hold-out sensor The core challenge in ARC's report is to obtain accurate knowledge about the actual presence / absence of a diamond in a protective "SmartVault", as opposed to accurate knowledge about whether a human observer would think the diamond is present if the human observer had access to data from a sensor I'll call S1. This is difficult because it's hard to get labeled data which differentiates between the two. If the AI protecting the diamond in the vault is optimizing for whether a human observer would think the diamond is present, it will find that corrupting readings from S1 while allowing a thief to take the diamond is a perfectly legitimate strategy. When the AI simulates this diamond-protection strategy, it will receive the maximum score, since score is calculated based on what a human with access to S1 readings would think. Suppose we install an additional sensor S2 in the vault, on a closed-circuit system that isn't present in the AI's model of the vault. It seems reasonable to guess that the AI won't be able to boost the simulated score of its diamond-protection strategy by corrupting readings from S2 in addition to S1. So if ARC has some method they think will get the AI to actually guard the diamond, secretly observing S2 gives them a chance to check if the method is working. This strategy could fail. For example, if (a) S2 ends up being present in the AI's model of the vault despite our best intentions, and (b) the AI's learned scoring function happens to be based on what a human with access to all known sensors would think. (This latter possibility is a bigger risk if humans have been peeking at S2 periodically during the course of iteratively developing the AI. The analogy to train/dev/test splits should be pretty obvious here.) So it's not a perfect detection method. Idea #2: Predicting the readings of hypothetical sensors An implicit assumption of the Eliciting Latent Knowledge report is that the SmartVault AI can make predictions about readings from sensor S1, given the actions it takes to manage the vault and other data as predictor variables. I'm imagining the AI has been trained to be able to do this using supervised learning. We have some features X consisting of actions taken and other data, and a target Y consisting of S1 readings. The AI searches for a model which maps X to Y using e.g. gradient descent. We can modify the training procedure to predict the readings from any hypothetical sensor as follows. In addition to the original features X, we add six additional features corresponding to the latitude, longitude, height, yaw, pitch, and roll of the sensor, resulting in an augmented feature matrix X'. To generate training data, we repeatedly: Use a random number generator to generate random values for those six additional features Place the sensor according to those random values Use whatever data generation method we were using in the original SmartVault scenario to populate Y with readings from the randomly placed sensor If the AI correctly learns a mapping from the augmented feature matrix X' to Y, we've now got a model that can make predictions about the readings we would get from any hypothetically placed sensor. This idea mainly serves as a building block for later ideas, but the basic intuition is that sensor corruption is tougher if t...