Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Future Matters: May 2022, published by Pablo on May 28, 2022 on The Effective Altruism Forum. In the littered field of discredited self-congratulatory chauvinisms, there is only one that seems to hold up, one sense in which we are special: Due to our own actions or inactions, and the misuse of our technology, we live at an extraordinary moment, for the Earth at least—the first time that a species has become able to wipe itself out. But this is also, we may note, the first time that a species has become able to journey to the planets and the stars. The two times, brought about by the same technology, coincide—a few centuries in the history of a 4.5-billion-year-old planet. If you were somehow dropped down on the Earth randomly at any moment in the past (or future), the chance of arriving at this critical moment would be less than 1 in 10 million. Our leverage on the future is high just now. Carl Sagan Future Matters is a newsletter about longtermism. Each month we collect and summarize longtermism-relevant research, share news from the longtermism community, and feature a conversation with a prominent longtermist. You can also subscribe on Substack, listen on your favorite podcast platform and follow on Twitter. Research Stefan Schubert's Against cluelessness notes that, while it is generally very hard to predictably affect the long-term future, this fails to constitute a decisive objection to longtermism. Amidst this "sea of cluelessness", we find "pockets of predictability", or opportunities for making long-lasting changes that are positive in expectation. Schubert describes two types of intervention that escape these worries. First, attempts to reduce short-term risks to our long-term potential: such risks are tractable because they are located in the near-term, but still have longtermist significance. Threats of this type include risks of human extinction as well as risks of value lock-in. Second, efforts to build longtermist capacity, such as increasing the size of the longtermism community or its financial resources, or improving the reputation or knowledge of that community. This capacity can be built over a period of decades or centuries, until good enough opportunities to robustly improve the long-term future emerge. Joseph Carlsmith's presentation on existential risk from power-seeking AI (video and transcript) summarizes the author's comprehensive report published in April last year. Carlsmith focuses specifically on AI systems defined by the possession of three key properties: advanced capability, agentic planning, and strategic awareness (APS, for short), and then relies on this construct to develop an explicit argument for the conclusion that AI poses an existential risk to humanity: It will become possible to develop APS systems. There will be strong incentives to deploy APS systems. It will be much harder to build aligned than misaligned APS systems. Misaligned APS systems, if deployed, will fail in high-impact ways. A failure from a misaligned APS system will permanently disempower humanity. A disempowerment of humanity will constitute an existential catastrophe. Summarizing all the considerations Carlsmith adduces for each of the premises in the argument is beyond the scope of this newsletter, but we encourage the reader to check out the author's commendably lucid talk for the relevant details. Lizka Vaintrob's Against "longtermist" as an identity describes two possible harms of self-identifying as a "longtermist". The first is that the label includes factual claims, e.g. that trying to improve the long-term is generally more cost-effective than improving the world in other ways, which makes it harder to update in response to evidence against these claims. The second is that the label causes people to uncritically accept wholesale the cluster of vie...