Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How complex are myopic imitators?, published by Vivek Hebbar on February 8, 2022 on The AI Alignment Forum. This post was written under Evan Hubinger’s direct guidance and mentorship, as a part of the Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program. TL;DR: Time-limited myopic agents might be desirable for their safety properties; one such case is that of "myopic imitators", which imitate a non-myopic process like a human, but do so without considering consequences. Since DL systems are biased towards simple functions, I consider the question of whether myopic imitators are simpler or more complex than their deceptively aligned competitors. For this analysis, we impose a strong assumption which is similar to minimal program complexity, as well as a clean division between a world model, an "internal interpreter", and a main algorithm which is executed. The analysis is structured as follows: Operationalizations for "myopia" and "simplicity prior" Methodology for assessing the simplicity of a candidate algorithm Analysis of candidate algorithms under various sets of further assumptions Under these very strong assumptions, and out of the algorithms analyzed, myopic imitators are usually favored. One of the two leading myopic imitators is a straightforward human-imitator which seems to have nice properties, but is sometimes disadvantaged by an inability to do a certain Goodhart-style exploit. The other one, though myopic, is a "training set predictor" which generalizes terribly in some cases. That being said, the specific results are to be taken with a grain of salt, since the assumptions are unlikely to hold in reality. The main value going forward is the general approach and methodology. Assumptions, worldview, and research vision The analysis in this post applies to a world where a very specific set of assumptions applies. To be clear, I am quite confident that this set of assumptions does not apply to current deep learning systems. Current deep learning systems seem to be characterized more by circuits and interpolation than by the types of programs I consider here. However, the analysis here is interesting as a limiting case, and the methodology might be transferable to more realistic assumption sets. The main value of this post is to provide a particular frame and methodology for thinking about the question: "How is a system with a strong simplicity prior likely to turn out by default?" By default, when I say "simple" or "complex", I am referring to program length, not runtime. However, note that program length is not the only possible measure of complexity; in fact, the current inductive bias literature uses a different measure, as will be discussed later. 1 - Program approximation and power: The trained model closely approximates a program in some language, which is allowed to contain such abstractions as function reuse, recursion, interpretation, etc. This may be realized in reality through things like recurrence. 2 - Internal interpreter: The model will contain some sort of "interpreter" internally which can take an algorithm represented in terms of internally defined tokens and execute it. This might follow from minimal program length; a large program in an inefficient language can be refactored as {interpreter for a more suitable language} + {much shorter program}. One effect of this assumption is that the complexity of other parts can depend on the lengths of tokens in the language of this internal interpreter. 3 - Simplicity prior: Training a model to zero loss corresponds to taking the set of all programs below a certain length and runtime which perfectly fit the training set, then choosing one at random, weighted by some simplicity prior. In other words, the probability ratio between two perfect algorithms depends on only th...