Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Intermittent Distillations #1, published by Mark Xu on the AI Alignment Forum. This is my low-budget version of Rohin's Alignment Newsletter. A critique of pure learning and what artificial neural networks can learn from animal brains (Anthony M. Zador) A critique of pure learning and what artificial neural networks can learn from animal brains Summary This paper points out that human learning algorithm must rely on nearly all unsupervised data because humans don't live that long before they become intelligent. The authors also point out that many animals, potentially including humans, are remarkably functional after only a very short time being alive. This result suggests that there are parts of animal behavior that are not learned. This observation aligns with Yann Lecun's metaphor: "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning." Since the genome contains finite information (only about 1.5Gb in humans), there is a tradeoff between having lots of innate and fixed behaviors and having many learned and adaptable behaviors. Animals in relatively rigid environments might benefit from having lots of innate and fixed behaviors, whereas animals in adaptive environments might benefit more from learning. Additionally, animals that must be functional early to survive might favor innate behaviors, whereas animals with time to learn can achieve higher peak performance. The authors also suggest that the genome's finite information might form a "genomic bottleneck" that serves as a regularizer, potentially improving generalization performance. Opinion I agree with the observation that most humans seem to learn based on unsupervised data, although I think there are denser forms of "labels" in reality than the authors claim. For example, babies can get reinforcement from their parent's facial expressions, which potentially gives them a considerable number of labeled examples. I have already read (although I forget the source) that babies preferentially look at things with two dark spots above a single dark spot because these patterns are likely to be faces. If true, this instinct suggests that in addition to "innate behaviors", animals also have "innate learning heuristics" that direct them to places to get high-quality training data. I am unconvinced by the claim that the size of the genome serves as a meaningful bottleneck. There doesn't seem to have been that much evolutionary pressure for the genome to get larger, so it doesn't seem like it has served as a meaningful bottleneck. I am relatively sympathetic to arguments that the speed at which humans learn things suggests that there exists efficient architectures and/or learning algorithms. Further discussion is available here. Disappointingly, the paper concludes by claiming that artificial general intelligence will "constrained to match human capacities so tightly that only a machine structured similarly to a brain can achieve it", which seems pretty false to me. Learning Normativity: A Research Agenda (Abram Demski) Learning Normativity: A Research Agenda Alignment Newsletter Summary Summary What is the "correct usage" of language? There are more "correct" ways of using language, but no human speaks perfectly. However, humans can often learn how to speak language by being taught by humans who use language imperfectly in a way that lets them speak better than their teachers. The fact that this process is possible suggests that humans learn from data that contains mistakes without learning the mistakes. The author calls the rules that govern the use of language "norms" and proposes that learning these norms is an alignment method. If something is learning from feedback, you want to be able to give it f...