Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Intro to brain-like-AGI safety] 15. Conclusion: Open problems, how to help, AMA, published by Steve Byrnes on May 17, 2022 on The AI Alignment Forum. 15.1 Post summary / Table of contents This is the final post of the “Intro to brain-like-AGI safety” post series! Thanks for reading this far! In Section 15.2, I’ll list seven open problems that came up in the previous posts. I’m putting them all here in one place for the convenience of potential researchers and funders. In Section 15.3, I’ll offer some brief remarks on practical aspects of doing AGI safety (a.k.a. AI alignment) research, including funding sources, connecting to the relevant research community, and where to learn more. In Section 15.4, I’ll wrap up with 8 takeaway messages that I hope readers will have gotten out of this series. Since this is the “Conclusion” post, feel free to use the comment section for more general discussion (or to “ask me anything”), even if it’s not related to this particular post. 15.2 Open problems This is not, by any stretch of the imagination, a complete list of open problems whose progress would help with brain-like-AGI safety, let alone with the more general topic of Safe & Beneficial AGI (see Post #1, Section 1.2). Rather, these are just some of the topics that came up in this series, with ratings proportional to how enthusiastic I am about them. I’ll split the various open problems into three categories: “Open problems that look like normal neuroscience”, “Open problems that look like normal computer science”, and “Open problems that require explicitly talking about AGIs”. This division is for readers’ convenience; you might, for example, have a boss, funding source, or tenure committee who thinks that AGI Safety is stupid, and in that case you might want to avoid the third category. (However, don’t give up so soon—see discussion in Section 15.3.1 below.) 15.2.1 Open problems that look like normal neuroscience 15.2.1.1 The “Is Steve full of crap when he talks about neuroscience?” research program — ⭐⭐⭐⭐ If you didn’t notice, Posts #2–#7 are full of grand theorizing and bold claims about how the human brain works. It would be nice to know if those claims are actually true!! If those neuroscience posts are a bunch of baloney, then I think we should throw out not only those posts, but the whole rest of this series too. In the text of those posts, you’ll see various suggestions and pointers as to why I believe the various neuroscience claims that I made. But a careful, well-researched analysis has yet to be written, as far as I’m aware. (Or if it has, send me a link! Nothing would make me happier than learning that I’m reinventing the wheel by saying things that are already well-established and widely-accepted.) I give this research program a priority score of 4 stars out of 5. Why not 5? Two things: It loses half a star because I have utterly-unjustifiable overconfidence that my neuroscience claims are not, in fact, a bunch of baloney, and therefore this research program would look more like nailing down some of the finer details, and less like throwing this whole post series in the garbage. It loses another half star because I think there are some delicate corners of this research program where it gets uncomfortably close to the “unravel the gory details of the brain’s learning-from-scratch algorithms” research program, a research program to which I assign negative 5 stars, because I’d like to make more progress on how and whether we can safely use a brain-like AGI, long before we figure out how to build one. (See Differential Technology Development discussion in Post #1, Section 1.7.) 15.2.1.2 The “Reverse-engineer human social instincts” research program — ⭐⭐⭐⭐⭐ Assuming that Posts #2–#7 are not, in fact, a bunch of baloney, the implication is that there are circuits for ...