Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reflection Mechanisms as an Alignment target: A survey, published by Marius Hobbhahn on June 22, 2022 on The AI Alignment Forum. This is a product of the 2022 AI Safety Camp. The project has been done by Marius Hobbhahn and Eric Landgrebe under the supervision of Beth Barnes. We would like to thank Jacy Reese Anthis and Tyna Eloundou for detailed feedback. You can find the google doc for this post here. Posts to other sections of the text automatically link to the google doc. Feel free to add comments there. Abstract We surveyed ~1000 US-based Mechanical Turk workers (selected and quality tested by Positly) on their attitudes to moral questions, conditions under which they would change their moral beliefs, and approval towards different mechanisms for society to resolve moral disagreements. Unsurprisingly, our sample disagreed strongly on questions such as whether abortion is immoral. In addition, a substantial fraction of people reported that these beliefs wouldn’t change even if they came to different beliefs about factors we view as morally relevant such as whether the fetus was conscious in the case of abortion. However, people were generally favorable to the idea of society deciding policies by some means of reflection - such as democracy, a debate between well-intentioned experts, or thinking for a long time. In a hypothetical idealized setting for reflection (a future society where people were more educated, informed, well-intentioned e.t.c.), people were favorable to using the results of the top reflection mechanisms to decide policy. This held even when respondents were asked to assume that the results came to the opposite conclusion as them on strongly-held moral beliefs such as views on abortion. This suggests that ordinary Americans may be willing to defer to an idealized reflection mechanism, even when they have strong object-level moral disagreements. This indicates that people would likely support aligning AIs to the results of some reflection mechanism, rather than people’s current moral beliefs. Introduction Optimistically, a solution to the technical alignment problem will allow us to align an AI to “human values.” This naturally raises the question of what we mean by this phrase. For many object-level moral questions (e.g. “is abortion immoral?”), there is no consensus that we could call a “human value.” When lacking moral clarity we, as humans, resort to a variety of different procedures to resolve conflicts both with each other (democracy/voting, debate) and within ourselves (read books on the topic, talk with our family/religious community). In this way, although we may not be able to gain agreement at the object level, we may be able to come to a consensus by agreeing at the meta level (“whatever democracy decides will determine the policy when there are disagreements”); this is the distinction between normative ethics and meta-ethics in philosophy. We see the meta question of value choice as being relevant to strategic decisions around AI safety for a few reasons. First, understanding people’s preferred conflict resolution mechanisms could be relevant for strategic decisions around AI governance and governance in general. Specifically, we want to avoid arms race conditions so that responsible labs have adequate time to ensure the safety of human-level AI. We feel that the ability to avoid race conditions could be strongly influenced by people’s broad ability to agree on metaethics. In particular, people should be less likely to fight for control of AI if they trust that AI builders will produce a system that reflects their (potentially-meta level) values. Understanding people’s expressed meta-ethics will therefore hopefully inform what types of governance structures are most likely to avoid race conditions. Additionally, understanding people's...