Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Meta-execution, published by Paul Christiano on the AI Alignment Forum. This post describes meta-execution, my current proposal for capability amplification and security amplification. (Meta-execution is annotated functional programming + strong HCH + a level of indirection. It is implemented in the amplify module of my ALBA repository.) Goal We are given an efficient agent A which competently pursues some values. We’d like to use a bunch of copies of A in order to implement a more powerful and robust agent Meta(A) with the same values. Outline Our basic plan is to build a machine out of copies of the agent; instead of asking the agent to make a decision directly, we ask it to implement the decision-making process by answering a sequence of questions of the form “what should happen next?” The basic object in meta-execution is a message, which consists of text along with pointers to other messages or to agents. Each box is a message. A is an agent who can respond to queries like “which of X and Y is larger?” We can represent arbitrarily large objects as giant trees of messages and agents. Meta-execution first forms a tree representing the question “what should be done?” It then asks the agent A to perform a sequence of operations on the tree that eventually lead to an answer. Then it executes that answer. The initial tree might look something like this: If you can answer this question, you can implement an agent. At any given time, an agent who is operating on this tree can only “see” a few messages: it can read the text of those messages, and see pointers like [red] and [blue]. Initially the agent can see only the root of the tree. If you are an agent tasked with processing a message, there are a few basic operations you can perform. You specify the “targets” of the action by specifying pointers you want to follow: Look at another part of the tree which is not currently visible. Spawn a new agent, and see a pointer to that agent. Send a message to an agent, and see its reply. You can compose a message by writing it out with sub-messages in parentheses (); for example, “What is the smallest element in (the list with first element [green] and remaining elements [purple]) according to [blue]” would produce the message in the first image above, if [green], [purple], [blue] had appropriate values. Terminate the current computation by composing a reply. This reply gets sent to the “parent” who initiated the current computation. In the case of the very first agent, who was created in order to answer the original question “what should an agent in state [red] do after receiving input [blue]?”, the reply specifies what the overall system should do. An example execution is illustrated here. I made a quick demo of the execution process, you can find it here. And that’s basically it. We spawn a new agent, and hand it the “what should we do?” message. It can then take any of the basic actions listed above and see the result. We repeat that process until the agent returns a message indicating what should be done. We parse the message as an action and new state (see the section on parsing below), we execute the action, and we update the system’s state. The details Hopefully for most purposes that outline tells you everything you need to know. If not, the easiest way to learn exactly how this works is probably just to look at the code. Meta-execution is implemented as lambda A : Meta(HCH(A, n)) in the package amplify.__init__, where n is the computational budget and A is the meta-executor. You can experience being the meta-executor by calling examples.meta.act("test") . The available commands are described in the README. Everything is immutable I assume that we have a digital implementation of A, and so we can snapshot and copy it freely. Each time an agent makes an observation and t...