https://www.agisafetyfundamentals.com/ai-alignment-curriculum

The first two readings this week offer several different perspectives on how we should think about artificial general intelligence. This is the key concept underpinning the course, so it’s important to deeply explore what we mean by it, and reasons for thinking that the field of AI is heading towards it. The third reading focuses on grounding these high-level arguments by reference to the behavior of existing ML systems. Steinhardt argues that, in machine learning, novel behaviors which are difficult to predict using standard approaches tend to emerge at larger scales.

These first three readings don’t say much about when we should expect AGI. The most comprehensive report on the topic (summarized by Karnofsky (2021b)) estimates the amount of compute required to train neural networks as large as human brains to do highly impactful tasks, and concludes that this will probably be feasible within the next four decades - although the estimate is highly uncertain. Karnofsky (2021a) helps contextualize the magnitude of this claim, arguing that although it seems wild to believe that this century might see radical change, when we zoom out it becomes much more credible that we live at one of the most impactful times in human history.