Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How would a language model become goal-seeking?, published by David Mears on July 16, 2022 on The Effective Altruism Forum. Or what should I read to understand this? It seems like some people expect descendants of large language models to pose a risk of becoming superintelligent agents. (By ‘descendants’ I mean adding scale and non-radical architectural changes: GPT-N.) I accept that there’s no reason in principle that LLM intelligence (performance on tasks) should be capped at the human level. But I don’t know why to believe that at some point language models would develop agency / goal-seeking behaviour, where they start to try to achieve things in the real world instead of continuing to perform their ‘output predicted text’ behaviour. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.