Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evaluations project @ ARC is hiring a researcher and a webdev/engineer, published by Beth Barnes on September 9, 2022 on The AI Alignment Forum. The evaluations project at the Alignment Research Center is looking to hire a generalist technical researcher and a webdev-focused engineer. We're a new team at ARC building capability evaluations (and in the future, alignment evaluations) for advanced ML models. The goals of the project are to improve our understanding of what alignment danger is going to look like, understand how far away we are from dangerous AI, and create metrics that labs can make commitments around (e.g. 'If you hit capability threshold X, don't train a larger model until you've hit alignment threshold Y'). We're also still hiring for model interaction contractors, and we may be taking SERI MATS fellows. About the project Big picture Once AI systems are close to being capable enough to be an x-risk, there should be procedures that labs follow to ensure they don’t build or deploy existentially dangerous models. We should have thought through the plausible paths to models gaining power, and ensured that models are not capable of these. There should be commitments that if model capabilities reach a certain level, labs will not scale up or deploy models until they have reached a certain level of alignment. In order to make this happen, we need good ways to measure both dangerous capabilities and alignment. We can start preparing for this now by: evaluating capabilities of current models predicting how dangerous the next generation of models is likely to be assessing the quality of our prediction methods by using them to predict the properties of generation n+1 given generation n, for historical generation models working with labs to evaluate their models before they are deployed, as a warm-up to stronger commitments in the future designing evaluations for alignment, deception etc to be run on future models Doing this now has some additional benefits: Evaluating capabilities of current models is helpful for predicting timelines, by giving us a more precise sense of what needs to change for models to be dangerous Specifying what we’d actually measure to determine the alignment of some model is helpful for clarifying our thoughts about how we expect misalignment to emerge, and what exactly the alignment problem is This work may produce demonstrations of models doing competent and scary behavior. This can be useful if we think people are underestimating the capabilities of current or near-future models and making bad decisions because of that. Current goals Our current goals are to work with cutting-edge models, produce evaluations for how far they are from x-risk capabilities, and share these evaluations with labs. More specifically, we’re doing things like: design and build scaffolding that lets generative language models act like agents over long time horizons: scripts to automatically run model outputs in the shell or forward them as subtasks to other copies of the model sampling multiple actions, having the model evaluate them and pick the best one various ways of prompting and nudging the model that improve performance think about what a model would need to be able to do in order to successfully gain power, given current levels of caution and monitoring around AI systems. For instance: exfiltrate its weights via hacking or social engineering; make money via crime or labor; use this money to buy server time and start more copies of itself running on those servers Assess model performance on these subtasks, and determine whether the model is capable of some set of tasks that's sufficient for successful powerseeking, or if not, what exactly the barriers are and how close the model is to surmounting them At the end of this, we hope to either have found clea...