Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why I'm excited about Debate, published by Richard Ngo on the AI Alignment Forum. I think Debate is probably the most exciting existing safety research direction. This is a pretty significant shift from my opinions when I first read about it, so it seems worth outlining what’s changed. I’ll group my points into three categories. Points 1-3 are strategic points about deployment of useful AGIs. Points 4-6 are technical points about Debate. Points 7-9 are meta-level points about how to evaluate safety techniques, in particular responding to Beth Barnes’ recent post on obfuscated arguments in Debate. 1. Question-answering is very useful. People often claim that question-answering AGIs (which I’ll abbreviate as QAGIs) will be economically uncompetitive compared with agentic AGIs. But I don’t think this matters very much for the two most crucial applications of AGIs. Firstly, when it comes to major scientific and technological advances, almost all of the value is in the high-level concepts - it seems unlikely that implementing those advances will require AGI supervision (rather than just supervision from narrow AIs) during deployment. Secondly, aligned QAGIs can do safety research to help us understand how to build aligned agentic AGIs, and can also predict and prevent their misbehaviour. So even a relatively small lead for aligned QAGIs could be very helpful. 2. Debate pushes capabilities in the right direction. Another objection I used to have: I tend to expect that QAGIs are pretty safe anyway, which implies that in aligning QA systems, Debate isn’t helping tackle the cases we should be most worried about. And if it’s used as a final step of training for systems that have previously been trained to do other things, then it’s very unclear to me whether Debate would override whatever unsafe motivations those systems had already acquired. But now I think that Debate could be an important tool for not only making QA systems more aligned, but also more competitive. Systems like GPT-3 have shown a very good understanding of language, and I expect them to gain much more world-knowledge, but it’s hard to elicit specific answers from them. We might hope to make them do so by using reward-modelling to fine-tune them, but I expect that in order to scale this up to complex questions, we’ll need to make that process much more efficient in human time. That’s what Debate does, by allowing humans to evaluate answers on criteria that are much simpler than the holistic question “is this answer good?” 3. Debate provides a default model of interaction with AGIs. We won’t just interact with agentic AGIs by giving them commands and waiting until they’ve been carried out. Rather, for any important tasks, we’ll also ask those AGIs to describe details of their plans and intentions, and question them on any details which we distrust. And for additional scrutiny, it seems sensible to run these answers past other AGIs. In other words, Debate is a very natural way to think about AGI deployment, and describes a skillset which we should want all our AGIs to have, even if it’s not the main safety technique we end up relying on. 4. Debate implicitly accesses a complex structure I originally thought that Debate was impractical because debates amongst humans aren’t very truth-conducive. But now I consider it misleading to think about Debate as simply a more sophisticated version of what two humans do. The comparison to a game of Go is illuminating. Specifically, let’s interpret any given Go position as a question: who wins the game of Go starting from this position? Then we can interpret a single game of Go played from that position, by sufficiently strong players, as good evidence that the (exponential) tree of other possible games doesn’t contain a refutation of any of the moves played by the eventual ...