Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI takeoff story: a continuation of progress by other means, published by Edouard Harris on the AI Alignment Forum. Thanks to Vladimir Mikulik for suggesting that I write this, and to Rohin Shah and Daniel Kokotajlo for kindly providing feedback. Prologue This is a story about a universe a lot like ours. In this universe, the scaling hypothesis — which very roughly says that you can make an AI smarter just by making it bigger — turns out to be completely right. It’s gradually realized that advances in AI don’t arise from conceptual breakthroughs or sophisticated deep learning architectures. Just the opposite: the simpler the architecture, the better it turns out to perform at scale. Past a certain point, clever model-building was just slowing down progress. Researchers in this universe discover a rough rule of thumb: each neural network architecture has an intrinsic maximum potential intelligence, or “capability”. When you train a network on a problem, how close it gets to reaching its potential capability depends on two limiting factors: 1) the size and diversity of its dataset; and 2) the amount of compute that’s used to train it. Training a network on a quadrillion games of tic-tac-toe won’t make it smart, but training a network on a quadrillion-word corpus of text might just do it. Even data cleaning and quality control don’t matter too much: as long as you have scale, if you train your system long enough, it learns to separate signal from noise automatically. Generally, the more parameters a neural network has, the higher its potential capability. Neural nets with simple architectures also have a higher potential capability than nets with more sophisticated architectures do. This last observation takes the research community longer to absorb than you might expect — it’s a bitter lesson, after all — so the groups that internalize it first have an early edge. Frontier AI projects begin to deemphasize architecture innovations and any but the most basic data preprocessing. They focus instead on simple models, huge datasets, hard problems, and abundant compute. Initial progress is slowed somewhat by a global semiconductor shortage that increases the cost of running large GPU workloads. Within a year or so, though, this bottleneck clears, and the pace of advance accelerates. Our story opens just as the world’s supply chains are getting back to normal. It begins with Chinese content apps. ByteDance launches an experiment to auto-generate some of the articles on their Toutiao news app using a language model. Initially this is ignored in the West partly because of the language barrier, but also because the articles just aren’t very good. But after a few months, their quality improves noticeably. Within a year of their launch, auto-generated articles make up the bulk of Toutiao’s inventory. Shortly afterward, ByteDance subsidiary Douyin launches auto-generated videos. These begin tentatively, with a handful of AI-generated creators people refer to as “synthetics”. Synthetics are wildly popular, and the program is quickly expanded to TikTok, Douyin’s sister app for users outside mainland China. Popularity explodes after TikTok rolls out super-personalization: everyone sees a different video, and each video is personalized just for you based on your past viewing history. In short order, personalized synthetics roll out across all of TikTok’s regions. Since human creators can’t compete, they get downgraded by TikTok’s recommendation algorithm, which heavily optimizes for viewing time. It’s hotly debated whether TikTok’s synthetic videos contain covert political propaganda — studies of the algorithm are hard to reproduce, since each user’s feed is completely personalized — but experts are concerned. Social networks find themselves at a disadvantage, since human-generated post...