Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A short introduction to machine learning, published by Richard Ngo on the AI Alignment Forum. Despite the current popularity of machine learning, I haven’t found any short introductions to it which quite match the way I prefer to introduce people to the field. So here’s my own. Compared with other introductions, I’ve focused less on explaining each concept in detail, and more on explaining how they relate to other important concepts in AI, especially in diagram form; I hope that this makes it useful for people who, like me, prefer to develop a top-down understanding of new fields. I'm aware that high-level taxonomies can be controversial, and also that it's easy to fall into the illusion of transparency when trying to introduce a field; so suggestions for improvements are very welcome! The key ideas are contained in this summary diagram: First, some quick clarifications: None of the boxes are meant to be comprehensive; we could add more items to any of them. So you should picture each list ending with “and others”. The distinction between tasks and techniques is not a firm or standard categorisation; it’s just the best way I’ve found so far to lay things out. The summary is explicitly from an AI-centric perspective. For example, statistical modelling and optimisation are fields in their own right; but for our current purposes we can think of them as machine learning techniques. Let’s dig into each part of the diagram now, starting from the top. Paradigms of artificial intelligence The field of artificial intelligence attempts to develop computer programs that possess the capabilities associated with intelligence in humans: language skills, visual perception, motor control, and so on. It got started around the 1950s. Historically, there have been several different approaches to AI. In the first few decades, the dominant paradigm was symbolic AI, which focused on representing problems using high-level mathematical equations, then solving them using search and logic. One highlight was Deep Blue, the chess AI that beat Kasparov in 1997. However, the symbolic representations designed by AI researchers turned out to be far too simple to allow symbolic AIs to handle complex real-world phenomena. Since the 1990s, the dominant paradigm in AI has been machine learning, which allows AIs to improve their performance based on experience and feedback (known as the learning, training or optimisation process). The most basic machine learning techniques are statistical models, such as linear regression - which in its simplest form only learns the values of two parameters to represent the training data. Although most people don’t think of linear regression as a machine learning technique, it’s hard to draw a clear boundary between statistical models and more central examples of machine learning techniques; hence I’ve included statistical modelling in the diagram above. However, the biggest successes of machine learning have come from applying techniques at a much larger scale than standard statistical modelling - in particular by training large neural networks with many layers, using powerful optimisation techniques like backpropagation. This is known as deep learning. Neural networks have been around since the beginning of AI, but they only became the dominant paradigm in the early 2010s, after increases in compute availability allowed us to train much bigger networks. Let’s explore the components of deep learning in more detail now. Deep learning: neural networks and optimisation Neural networks are a type of machine learning model inspired by the brain. They consist of multiple connected layers of artificial neurons, represented by circles in the diagram below. Note that networks with more than one layer between the input and the output layers are known as deep neural networks; the...