Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Prize for Alignment Research Tasks, published by Andreas Stuhlmüller on April 29, 2022 on The AI Alignment Forum. Can AI systems substantially help with alignment research before transformative AI? People disagree. Ought is collecting a dataset of alignment research tasks so that we can: Make progress on the disagreement Guide AI research towards helping with alignment We’re offering a prize of $200-$2000 for each contribution to this dataset. The debate: Can AI substantially help with alignment research? Wei Dai asked the question in 2019: [This] comparison table makes Research Assistant seem a particularly attractive scenario to aim for, as a stepping stone to a more definitive [AI Safety] success story. Is this conclusion actually justified? Jan Leike thinks so: My currently favored approach to solving the alignment problem: automating alignment research using sufficiently aligned AI systems. It doesn’t require humans to solve all alignment problems themselves, and can ultimately help bootstrap better alignment solutions. Paul Christiano agrees: Building weak AI systems that help improve alignment seems extremely important to me and is a significant part of my optimism about AI alignment. [...] Overall I think that "make sure we are able to get good alignment research out of early AI systems" is comparably important to "do alignment ourselves." Realistically I think the best case for "do alignment ourselves" is that if "do alignment" is the most important task to automate, then just working a ton on alignment is a great way to automate it. But that still means you should be investing quite a significant fraction of your time in automating alignment. Eliezer doesn't: "AI systems that do better alignment research" are dangerous in virtue of the lethally powerful work they are doing, not because of some particular narrow way of doing that work. If you can do it by gradient descent then that means gradient descent got to the point of doing lethally dangerous work. Asking for safely weak systems that do world-savingly strong tasks is almost everywhere a case of asking for nonwet water, and asking for AI that does alignment research is an extreme case in point. Everyone would likely agree that AI can help a little, e.g. using next word prediction to write papers slightly faster. The debate is about whether AI can help enough with alignment specifically that it substantially changes the picture. If AI alignment is 70% easy stuff we can automate and 30% hard stuff that we can't hope to help with, the 30% is still a bottleneck in the end. Motivation for the dataset We’re collecting a dataset of concrete research tasks so that we can: Make progress on the disagreement about whether AI can substantially help with alignment before TAI. Is there even a disagreement? Maybe people aren’t talking about the same kinds of tasks and the collective term “alignment research” obscures important distinctions. If there is a disagreement, concrete tasks will let us make progress on figuring out the correct answer. Guide AI research towards helping with alignment. Figure out if current language models can already be helpful now. If they can, help Ought and others build tools that are differentially useful for alignment researchers. If they can’t, guide future language model work towards supporting those tasks. As an important special case of step two, the dataset will guide the plan for Elicit. Limitations Ideally, we’d come up with tasks and automation together, iterating quickly on how to set up the tasks so that they are within reach of language models. If tasks are constructed in isolation, they are likely to be a worse fit for automation. In practice, we expect that language models won’t be applied end-to-end to tasks like this, mapping inputs to outputs, but will be part of compositi...