Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Where I currently disagree with Ryan Greenblatt’s version of the ELK approach, published by Nate Soares on September 29, 2022 on The AI Alignment Forum. Context: This post is my attempt to make sense of Ryan Greenblatt's research agenda, as of April 2022. I understand Ryan to be heavily inspired by Paul Christiano, and Paul left some comments on early versions of these notes. Two separate things I was hoping to do, that I would have liked to factor into two separate writings, were (1) translating the parts of the agenda that I understand into a format that is comprehensible to me, and (2) distilling out conditional statements we might all agree on (some of us by rejecting the assumptions, others by accepting the conclusions). However, I never got around to that, and this has languished in my drafts folder too long, so I'm lowering my standards and putting it out there. The process that generated this document is that Ryan and I bickered for a while, then I wrote up what I understood and shared it with Ryan, and we repeated this process a few times. I've omitted various intermediate drafts, on the grounds that sharing a bunch of intermediate positions that nobody endorses is confusing (moreso than seeing more of the process is enlightening), and on the grounds that if I try to do something better then what happens instead is that the post languishes in the drafts folder for half a year. (Thanks to Ryan, Paul, and a variety of others for the conversations.) Nate's model towards the end of the conversation Ryan’s plan, as Nate currently understands it: Assume AGI is going to be paradigmatic, in the sense of being found by something roughly like gradient descent tuning the parameters in some fixed architecture. (This is not intended to be an argument for paradigmaticity; attempting to align things in the current paradigm is a good general approach regardless (or so Nate understands Ryan to claim).) Assume further that Earth's first AGIs will be trained according to a process of our choosing. (In particular, it needs to be the case that AGI developers can train for more-or-less any objective they want, without thereby sacrificing competitiveness. Note that this might require significant feats of reward-shaping.) Assume further that most capability gains will be driven by something roughly like gradient descent. (Ryan has some hope that this plan would generalize to various other drivers of capability gains, but Nate hasn't understood any of the vague gestures towards those ideas, and as of April 2022, Ryan's plans were mostly stated in terms of SGD, so I'll set that aside for now.) With those three assumptions in hand, part one of the plan is to pretend like we have the One True Objective Function, and that we've contracted the services of the Devil Himself, which we utilize to train the AGI to pursue the given objective. The special power of the devil is that he can generate the inputs that cause the nascent AI to do as poorly as possible according to the one true objective function, and we can use this to find the activations of the worst misbehaviors, and expose them to gradient updates. This is intended to result in a system that Nate might call "inner aligned" with the objective we use in training. Part two of the plan is to solve the problem of eliciting a paradigmatic AI system's latent knowledge, and then use it to substitute for the One True Objective Function and the Devil Himself. The basic idea here is that you don't need to actually have a perfect objective function or a perfect adversary for part one to work; it should in general suffice to nip any misbehavior in the bud the moment that the AI itself starts to have any knowledge of that misbehavior. The scope of this plan is to produce a system that would work well in-distribution, and that would at least no...