Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some ideas for epistles to the AI ethicists, published by Charlie Steiner on September 14, 2022 on The AI Alignment Forum. Some papers, or ideas for papers, that I'd loved to see published in ethics journals like Minds and Machines or Ethics and Information Technology. I'm probably going to submit one of these to a 2023 AI ethics conference myself. Why should we do this? Because we want today's grad students to see that the ethical problems of superhuman AI are a cool topic that they can publish a cool paper about. And we want to (marginally) raise the waterline for thinking about future AI, nudging the AI ethics discourse towards more matured views of the challenges of AI. Secondarily, it would be good to leverage the existing skillsets of some ethicists for AI safety work, particularly those already working on AI governance. And having an academic forum where talking about AI safety is normalized bolsters other efforts to work on AI safety in academia. The Ideas: Explain the basic ideas of AI safety, and why to take them seriously. Iason Gabriel already had a pretty good paper like this. But it's plausible that, right now, what the ethics discourse needs is more basic explanations of why AI safety is a thing at all. This paper might start out by making a case that superhuman AI is going to change the world, likely in the next 10-60 years (definitely unintuitive to many, but there are AI Impacts surveys and recent results to illustrate the point). Then the basic arguments that superhuman AI will not be automatically benevolent (easy rhetorical trick is to call it "superhuman technology," everyone knows technology is bad). Then the basic arguments that to get things to go well, the AI has to know a whole lot about what humans want (and use that knowledge the way we want). One issue with this might be that it presents the problem, but doesn't really point people towards solutions (this may be a problem that can be solved with quick citations). It also doesn't really motivate why this is an ethics problem. It also doesn't explain why we want the solution to the key "ethics-genre" problems to use a technical understanding of the AI, rather than a human- or society-centric view. A more specific defense of the validity of transformative-AI-focused thinking as a valid use of ethicists' time. The core claim is that getting AIs to want want good things and not bad things is an unsolved ethics problem. Ethics, not engineering, because the question isn't "how do we implement some obvious standard," the question is "what is even a good standard in the first place?" But almost as important are secondary claims about what actual progress on this question looks like. The end goal is a standard that is connected to technical picture of how the AI will learn this information about humans, and how it will use it to make decisions. So the overall thrust is "given that AI safety is important, there is a specific sort of ethics-genre reasoning that is going to be useful, and here are some gestures towards what it might look like." You can put more than one of these ideas into a paper if you want. This particular idea feels to me like it could benefit from being paired with another topic before or after it. Dunking on specific mistakes, like talking about "robots" rather than "optimization processes," should probably be done with care and tact. A worked example of "dual use" ethics - a connection between thinking about present-day problems and superhuman AI. I expect most of the examples to be problems that sound relevant to the modern day, but that sneakily contain most of the alignment problem. E.g. Xuan's AI that takes actions in response to laws that we really want to follow the spirit of the law. Although that's a bit too futuristic, actually, because we don't have much present-day ...