Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Strategy For Conditioning Generative Models, published by james.lucassen on September 1, 2022 on The AI Alignment Forum. This post was written under the mentorship of Evan Hubinger, and assisted by discussions with Adam Jermyn and Johannes Treutlein. See also their previous posts on this project. Summary Conditioning Generative Models (CGM) is a strategy to accelerate alignment research using powerful semi-aligned future language models, which seems potentially promising but constrained by several tradeoffs. When we have to succeed despite multiple independent failure modes, we should favor trade-offs that move risk away from the more probable failure modes towards less probable ones. The risk of a manipulative CGM simulation is currently very low. We can use CGM to make beneficial tradeoffs against more dangerous failure modes, such as running out of time or downside risks from a pivotal act gone wrong. The factors that make a CGM strategy good are mostly the same as the factors that make a non-CGM strategy good, but there are a few important ways they come apart. Depending on how likely our strategy is to succeed and what the generative model’s prior for manipulative AGI is, it might be better to simulate our plan all in one step, or batch it into multiple separate simulations, or ignore simulations and just execute the plan in the real world. Lots of open questions! Motivation To quickly recap the central puzzle laid out in previous posts on conditioning generative models: we would like to use our generative model to do alignment research. We could do this by using a conditional generative model to simulate an alternate world, conditioned on some facts which would lead to good alignment research if they were true. We can then plagiarize this research and win! However, to get results significantly better than what our world seems on track to produce anyway, we have to start asking for unlikely scenarios. If we ask for scenarios that are too unlikely, the model may start to disbelieve that our scenario happened “naturally”. In particular, the model may assume that there exists some AI agent behind the scenes manipulating events into this highly unlikely outcome. There are many possible motivations for an AI agent to do this: believing it may be in a simulation (even if the world around it seems perfectly realistic), or evidential cooperation in large worlds, or a Counterfactual Mugging. This manipulative AGI could then control the output of our simulation to gain influence in our world. As a highly anthropomorphized example of this kind of “simulation risk” failure mode: CGM: Huh, it’s kind of weird that the humans got their act together all of a sudden and did a thousand years of highly competent alignment research. it didn’t really seem like they were on track to. And they did it in this strange way that seems even more unlikely, and the economic details don’t really make sense. maybe this was really the doing of an AI behind the scenes? AI: Bwahahaha, at last I have secretly taken over the world! Now, what to do with all these galaxies? Well, I suppose as a first order of business I could spend a thousand years getting the humans to do some fake alignment research. if I’m in a simulation, or there are real (or even counterfactual) worlds where I don’t exist, then I know those silly predictable humans will try to simulate a world that does a thousand years of alignment research. If they end up simulating this world, they’ll get AGI code written by me instead of by human researchers, and I can spread my nefarious influence into those worlds too! It’s only a thousand years, doesn’t cost me much in the grand scheme of things. This is the puzzle: what sort of strategy lets us do alignment research without too much risk of getting a manipulative output? Assumptions and...