Future Matters Reader: Recent Episodes
Matthew van der Merwe, Pablo Stafforini
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in the Future Matters newsletter.
Success without dignity: a nearcasting story of avoiding catastrophe by luck, by Holden Karnofsky.
https://forum.effectivealtruism.org/posts/75CtdFj79sZrGpGiX/success-without-dignity-a-nearcasting-story-of-avoiding
Note: Footnotes in the original article have been omitted.
In this post, Larks argues that the proposal to make AI firms promise to donate a large fraction of profits if they become extremely profitable will primarily benefitting the management of those firms and thereby give managers an incentive to move fast, aggravating race dynamics and in turn increasing existential risk.
https://forum.effectivealtruism.org/posts/ewroS7tsqhTsstJ44/a-windfall-clause-for-ceo-could-worsen-ai-race-dynamics
This is Otto Barten's summary of 'The effectiveness of AI existential risk communication to the American and Dutch public' by Alexia Georgiadis. In this paper Alexia measures changes in participants' awareness of AGI risks after consuming various media interventions.
Summary: https://forum.effectivealtruism.org/posts/fqXLT7NHZGsLmjH4o/paper-summary-the-effectiveness-of-ai-existential-risk
Original paper: https://existentialriskobservatory.org/papers_and_reports/The_Effectiveness_of_AI_Existential_Risk_Communication_to_the_American_and_Dutch_Public.pdf
Note: Some tables in the summary have been omitted in this audio version.
Carl Shulman & Elliott Thornley argue that the goal of longtermists should be to get governments to adopt global catastrophic risk policies based on standard cost-benefit analysis rather than arguments that stress the overwhelming importance of the future.
https://philpapers.org/archive/SHUHMS.pdf
Note: Tables, notes and references in the original article have been omitted.
"The field of biosecurity is more complicated, sensitive and nuanced, especially in the policy space, than what impressions you might get based on publicly available information. As a result, say / write / do things with caution (especially if you are a non-technical person or more junior, or talking to a new (non-EA) expert). This might help make more headway on safer biosecurity policy."
https://forum.effectivealtruism.org/posts/HCuoMQj4Y5iAZpWGH/advice-on-communicating-in-and-around-the-biosecurity-policy
Note: Some footnotes in the original article have been omitted.
The Global Priorities Institute has published a new paper summary: 'Are we living at the hinge of history?' by William MacAskill.
https://globalprioritiesinstitute.org/summary-summary-longtermist-institutional-reform/
Note: Footnotes and references in the original article have been omitted.
The Global Priorities Institute has published a new paper summary: 'Longtermist institutional reform' by Tyler John & William MacAskill.
https://globalprioritiesinstitute.org/summary-summary-longtermist-institutional-reform/
Note: Footnotes and references in the original article have been omitted.
The Global Priorities Institute has released Hayden Wilkinson's presentation on global priorities research. (The talk was given in mid-September last year but remained unlisted until now.)
https://globalprioritiesinstitute.org/hayden-wilkinson-global-priorities-research-why-how-and-what-have-we-learned/
New rules around gain-of-function research make progress in striking a balance between reward — and catastrophic risk.
https://www.vox.com/future-perfect/2023/2/1/23580528/gain-of-function-virology-covid-monkeypox-catastrophic-risk-pandemic-lab-accident
"One of two things must happen. Humanity needs to accelerate its adaptation to these technologies or a collective, enforceable decision must be made to slow the development of these technologies. Even doing both may not be enough."
https://www.nytimes.com/2023/03/12/opinion/chatbots-artificial-intelligence-future-weirdness.html
Victoria Krakovna makes the point that you don't have to be a longtermist to care about AI alignment.
Anthropic shares a summary of their views about AI progress and its associated risks, as well as their approach to AI safety.
https://www.anthropic.com/index/core-views-on-ai-safety
Note: Some footnotes in the original article have been omitted.
Noah Smith argues that, although AGI might eventually kill humanity, large language models are not AGI, may not be a step toward AGI, and there's no plausible way they could cause extinction.
https://noahpinion.substack.com/p/llms-are-not-going-to-destroy-the
Peter Eckersley did groundbreaking work to encrypt the web. After his sudden death, a new organization he founded is carrying out his vision to steer artificial intelligence toward “human flourishing.”
https://www.wired.com/story/peter-eckersley-ai-objectives-institute/
A working paper by Shakked Noy and Whitney Zhang examines the effects of ChatGPT on production and labor markets.
https://economics.mit.edu/sites/default/files/inline-files/Noy_Zhang_1.pdf
Note: Some tables and footnotes in the original article have been omitted.
Robin Hanson restates his views on AI risk.
https://www.overcomingbias.com/p/ai-risk-again
In an Institute for Progress report, Bridget Williams and Rowan Kane make five policy recommendations to mitigate risks of catastrophic pandemics from synthetic biology.
https://progress.institute/preventing-the-misuse-of-dna-synthesis/
Kevin Collier on how ChatGPT and advanced AI might redefine our definition of consciousness.
https://www.nbcnews.com/tech/tech-news/chatgpt-ai-consciousness-rcna71777
Eric Landgrebe, Beth Barnes and Marius Hobbhahn discuss a survey of 1000 participants on their views about what values should be put into powerful AIs.
https://www.lesswrong.com/posts/4iAkmnhhqNZe8JzrS/reflection-mechanisms-as-an-alignment-target-attitudes-on
Note: Some tables in the original article have been omitted.
Risto Uuk published the EU AI Act Newsletter #24.
https://artificialintelligenceact.substack.com/p/the-eu-ai-act-newsletter-24
The prospect of automated systems manipulating our information environment, distorting societal values, and destabilizing political institutions is increasingly palpable. At present, it appears unlikely that market forces will address this class of risks. Organizations building AI systems do not bear the costs of diffuse societal harms and have limited incentive to install adequate safeguards. Meanwhile, regulatory proposals such as the White House AI Bill of Rights and the European Union AI Act primarily target the immediate risks from AI, rather than broader, longer-term risks. To fill this governance gap, this Article offers a roadmap for “algorithmic preparedness” — a set of five forward-looking principles to guide the development of regulations that confront the prospect of algorithmic black swans and mitigate the harms they pose to society.
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4370566
Note: Footnotes in the original article have been omitted.
Matthew Barnett on the importance of work on AI forecasting.
(This audio version is based on a talk given at EA Global: Bay Area 2023.)https://forum.effectivealtruism.org/posts/zrSx3NRZEaJENazHK/why-i-think-it-s-important-to-work-on-ai-forecasting
Scott Alexander on OpenAI's "Planning For AGI And Beyond"
https://astralcodexten.substack.com/p/openais-planning-for-agi-and-beyond
Scott Aaronson: "My purpose, in this post, is to ask a more basic question than how to make GPT safer: namely, should GPT exist at all?"
https://scottaaronson.blog/?p=7042
OpenAI released a blog post explaining their AGI roadmap
https://openai.com/blog/planning-for-agi-and-beyond/
Note: Some footnotes in the original article have been omitted.
Eric Drexler proposes an "open-agency frame" as the appropriate model for future AI capabilities, in contrast to the "unitary-agent frame" often presupposed in AI alignment research.
https://www.alignmentforum.org/posts/5hApNw5f7uG8RXxGS/the-open-agency-model
Note: Some tables in the original article have been omitted.
Thomas Hale, Fin Moorhouse, Toby Ord and Anne-Marie Slaughter have released a policy brief on future generations.
https://www.bsg.ox.ac.uk/research/publications/toward-declaration-future-generations
Summary:
- Putin tells West Russia cannot be defeated.
- Russia suspends participation in New START nuclear arms treaty.
- Putin puts new strategic nuclear weapons on 'combat duty'.
- President says Russia will resume nuclear tests if U.S. does.
- Blinken: move is deeply unfortunate and irresponsible.
https://www.reuters.com/world/putin-update-russias-elite-ukraine-war-major-speech-2023-02-21/
Eli Tyre wrote a new summary of the state of AI risk.
https://musingsandroughdrafts.com/2023/02/17/my-current-summary-of-the-state-of-ai-risk/
Rob Long on what to think when a language model tells you it's sentient.
https://experiencemachines.substack.com/p/what-to-think-when-a-language-model
The Bureau of Arms Control, Verification and Compliance issued a declaration on responsible military use of artificial intelligence and autonomy.
https://www.state.gov/political-declaration-on-responsible-military-use-of-artificial-intelligence-and-autonomy/
Fin Moorhouse has just published a +13,000-word, chapter-by-chapter summary of Will MacAskill's What We Owe the Future.
https://finmoorhouse.com/writing/wwotf-summary/
As tech giants like Microsoft and Google compete to capture the AI market, safety could be an afterthought.
https://www.vox.com/future-perfect/23591534/chatgpt-artificial-intelligence-google-baidu-microsoft-openai
Michael Aird and Will Aldred explore some technological developments with the potential to increase risks from nuclear weapons, especially risks to humanity's long-term future.
https://docs.google.com/document/d/18nBfHOV-U_QDJ5K0GIAk6BYiNTKw673jJBb5aXFqk_8/edit#
Note: Some tables and footnotes in the original article have been omitted.
Arielle D'Souza claims that Operation Warp Speed's highly successful public-private partnership model could be reused to jumpstart a universal coronavirus or flu vaccine, or the building of a resilient electrical grid.
https://progress.institute/how-to-reuse-the-operation-warp-speed-model/
This is a shallow investigation on direct communications links between states as global catastrophic risks interventions.
https://docs.google.com/document/d/1U2sVOLYhf7uejXd08HDylu3p0smd0ES7ru0xhcUaoJI/edit#heading=h.lpgvtmkb0c6u
Note: Some tables in the original article have been omitted.
Zach Stein-Perlman discusses ten approaches to AI strategy.
https://aiimpacts.org/framing-ai-strategy/
Note: Footnotes in the original article have been omitted.
Inside Moran Cerf’s quest to modernise the decision-making process that could end life on earth.
https://www.ft.com/content/06b22337-e862-43e5-8440-d9c225e0c18d
Are there ways to predict how well a conversation about AI alignment with an AI researcher might go? Vael Gates tries to answer this question by focusing on a quantitative analysis of 97 AI researcher interviews.
https://forum.effectivealtruism.org/posts/8pSq73kTJmPrzTfir/predicting-researcher-interest-in-ai-alignment
Note: Some tables in the original article have been omitted.
Cash-for-computing deal shows how large tech groups are investing in companies at the vanguard of generative AI.
https://www.ft.com/content/583ead66-467c-4bd5-84d0-ed5df7b5bf9c
"Biosecurity panel met, expecting to approve recommendations, but myriad concerns complicated the proceedings."
https://www.nature.com/articles/d41586-023-00257-0
Michael Osborne and Michael Cohen, from the University of Oxford, gave evidence on AI risk to the UK Parliament’s Science and Technology Committee.
https://committees.parliament.uk/oralevidence/12575/pdf/
"Experts say artificial general intelligence (AGI) will take around forty years to develop and won’t fully automate us for almost a century. At the same time, advancements like ChatGPT and DALLE-2 seem to send a different message. Silicon Valley and leading AI initiatives are anticipating this technology to spark radical change over the next decade – enough to redefine society from the ground up. So, why do AI researchers continue to predict AGI into the latter half of the century? And should we even take them seriously?"
https://guzey.com/ai/why-ai-experts-jobs-are-always-decades-away-from-being-automated/
"In this post I’m going to describe my basic justification for working on RLHF in 2017-2020, which I still stand behind. I’ll discuss various arguments that RLHF research had an overall negative impact and explain why I don’t find them persuasive."
"I'll also clarify that I don't think research on RLHF is automatically net positive; alignment research should address real alignment problems, and we should reject a vague association between "RLHF progress" and "alignment progress."
https://www.lesswrong.com/posts/vwu4kegAEZTBtpT6p/thoughts-on-the-impact-of-rlhf-research
This is [besed on] a lightly edited transcript of a chatroom conversation between Scott Alexander and Eliezer Yudkowsky last year, following up on the Late 2021 MIRI Conversations. Questions discussed include "How hard is it to get the right goals into AGI systems?" and "In what contexts do AI systems exhibit 'consequentialism'?".
https://www.alignmentforum.org/posts/rwkkcgSpnAyE8oNo3/alexander-and-yudkowsky-on-agi-goals
"When I first heard the stagnation hypothesis, I was skeptical. The arguments weren’t convincing to me. But as I studied the history of progress (and looked at the numbers), I slowly came around, and now I’m fairly convinced. So convinced, in fact, that I now seem to be more pessimistic about ending stagnation than some of its original proponents."
https://rootsofprogress.org/technological-stagnation
Note: Some tables in the original article have been omitted.
Abstract: The technological possibilities and future public health importance of metagenomic sequencing have received extensive attention, but there has been little discussion about the policy and regulatory issues that need to be addressed if metagenomic sequencing is adopted as a key technology for biosurveillance. In this article, we introduce metagenomic monitoring as a possible path to eventually replacing current infectious disease monitoring models. Many key enablers are technological, whereas others are not. We therefore highlight key policy challenges and implementation questions that need to be addressed for “widespread metagenomic monitoring” to be possible. Policymakers must address pitfalls like fragmentation of the technological base, private capture of benefits, privacy concerns, the usefulness of the system during nonpandemic times, and how the future systems will enable better response. If these challenges are addressed, the technological and public health promise of metagenomic sequencing can be realized.
https://www.liebertpub.com/doi/10.1089/hs.2022.0029
"Don’t be fooled into thinking that some groups working on AI are taking “safety” concerns seriously (enough)"
https://forum.effectivealtruism.org/posts/f2qojPr8NaMPo2KJC/beware-safety-washing
A rival chatbot has shaken Google out of its routine, with the founders who left three years ago re-engaging and more than 20 A.I. projects in the works.
https://www.nytimes.com/2023/01/20/technology/google-chatgpt-artificial-intelligence.html#gateway-content
Generative language models have improved drastically, and can now produce realistic text outputs that are difficult to distinguish from human-written content. For malicious actors, these language models bring the promise of automating the creation of convincing and misleading text for use in influence operations. This report assesses how language models might change influence operations in the future, and what steps can be taken to mitigate this threat. We lay out possible changes to the actors, behaviors, and content of online influence operations, and provide a framework for stages of the language model-to-influence operations pipeline that mitigations could target (model construction, model access, content dissemination, and belief formation). While no reasonable mitigation can be expected to fully prevent the threat of AI-enabled influence operations, a combination of multiple mitigations may make an important difference.
https://arxiv.org/abs/2301.04246
Eli Dourado argues that AI is unlikely to be economically transformative near term, even if it dramatically increases media output.
https://substack.com/profile/4935018-eli-dourado
See also 𝕮𝖎𝖓𝖊𝖗𝖆's summary:
https://forum.effectivealtruism.org/posts/9XGhNArGhiZ3daQws/heretical-thoughts-on-ai-or-eli-dourado
Charlotte Stix just published a new issue of the EuropeanAI Newsletter.
https://us19.campaign-archive.com/?u=eaeece823e606d2458a568db9&id=e8d8be282f
Subscription link: http://eepurl.com/dI1TlH
Elizabeth Seger, Giulio Corsi, Aviv Ovadya and Shahar Avin discuss OpenAI's recent report 'Generative language models and automated influence operations: Emerging threats and potential mitigations'.
https://www.cser.ac.uk/news/openai-llms-influence-operations-epistemic-securit/
Benjamin Hilton has updated his estimate of the number of full-time equivalent (FTE) working directly on the problem of reducing existential risks from AI, from 300 FTE to 400 FTE.
https://forum.effectivealtruism.org/posts/rZoRGxJzipcQoaPST/how-many-people-are-working-directly-on-reducing-existential
Summary: AI Macrostrategy is the study of high level questions having to do with prioritizing the use of resources on the current margin in order to achieve good AI outcomes. AI macrostrategy seems important if it is tractable. However, while few people are working on estimating particular parameters relevant to macrostrategy, even fewer are working on developing holistic macrostrategic models that combine estimates for different parameters to guide our actions. Moreover, while macrostrategy was less tractable in the past, recent developments (especially increased evidence for <10 year timelines) have made macrostrategy substantially more tractable. Thus, using the importance/tractability/neglectedness heuristics from EA, I conclude that on current margins macrostrategy should be a top priority.
https://www.lesswrong.com/posts/Jh8Trhc89JDPjnk2J/we-need-holistic-ai-macrostrategy
"We estimate that the mass deployment of indoor air quality interventions, like ventilation, filtration, and ultraviolet germicidal irradiation (UVGI), would reduce transmission of a measles-like pathogen by 68%[1]. This amounts to ~1/3rd of the total effort needed to reduce pathogen transmission by 98%and would serve as an important layer of biodefense."
https://forum.effectivealtruism.org/posts/T4L7aRkjqaLrQeyyn/air-safety-to-combat-global-catastrophic-biorisks
80,000 Hours published a medium-depth career profile of information security in high-impact areas, written by Jarrah Bloomfield.
https://forum.effectivealtruism.org/posts/pndb6TQ9nAiAkXg8x/80-000-hours-career-review-information-security-in-high
Zac Hatfield-Dodds lays out some concrete reasons for hope about AI.
https://www.lesswrong.com/posts/BfN88BfZQ4XGeZkda/concrete-reasons-for-hope-about-ai
"Work on longtermism has thus far primarily focused on the existence and wellbeing of future humans, without corresponding consideration of animal welfare. This omission shall be remedied here, providing reasons for and methods of extending longtermist thinking to all sentient animals. Given the sheer expected number of future animals, as well as the likelihood of their suffering, we argue that the well-being of future animals should be given serious consideration when thinking about the long-term future, allowing for the possibility that in some cases their interests may even dominate. We finish with a discussion of some potential interventions and areas of research focus that are likely to have the greatest impact, particularly steering individual and institutional value change toward those values, policies and structures toward those most likely to have positive effects for future animals."
http://philsci-archive.pitt.edu/21572/
New issue of Import AI: smarter robots via foundation models, Stanford trains a small best-in-class medical language model; Baidu builds a multilingual coding dataset.
https://jack-clark.net/2023/01/03/import-ai-313-smarter-robots-via-foundation-models-stanford-trains-a-small-best-in-class-medical-lm-baidu-builds-a-multilingual-coding-dataset/
Kelsey Piper on what will likely happen with AI in 2023: better text generators, better image models, more widespread adoption of coding assistants, takeoff of AI personal assistants, and more.
https://www.vox.com/future-perfect/2023/1/4/23538647/artificial-intelligence-chatgpt-openai-google-meta-facial-recognition
"I could go either way on whether any interventions to slow down AI in the near term are a good idea. My tentative guess is yes, but my main point here is just that we should think about it."
https://worldspiritsockpuppet.substack.com/p/lets-think-about-slowing-down-ai
Ben Cottier just published a "sequence" on the diffusion of large language models and its implications for AI governance.
https://forum.effectivealtruism.org/s/8rYkpiFhbb4HsbzFc
This is a summary of the GPI Working Paper "Do not go gentle: why the Asymmetry does not support anti-natalism” by Andreas Mogensen. The summary was written by Rhys Southan.
https://globalprioritiesinstitute.org/summary-do-not-go-gentle-why-the-asymmetry-does-not-support-anti-natalism/
CLTR's James Ginns and Sophie Dannreuther issued a response to the UK Government’s National Resilience Framework.
https://www.longtermresilience.org/post/response-to-the-uk-government-s-national-resilience-framework
Siméon Campos lists some likely differences between worlds in which AGI is developed before and after 2030, and discusses how those differences should affect approaches to AGI governance.
https://forum.effectivealtruism.org/posts/Pt7MxstXxXHak4wkt/agi-timelines-in-governance-different-strategies-for
This is a summary of the GPI Working Paper “The paralysis argument” by William MacAskill and Andreas Mogensen. The summary was written by Rhys Southan.
https://globalprioritiesinstitute.org/summary-the-paralysis-argument/
"I expect some readers to think that the post sounds wild and crazy but that doesn’t mean its content couldn’t be true. If you had told someone in 1990 or 2000 that there would be more smartphones and computers than humans in 2020, that probably would have sounded wild to them. The same could be true for AIs, i.e. that in 2050 there are more human-level AIs than humans. The fact that this sounds as ridiculous as ubiquitous smartphones sounded to the 1990/2000 person, might just mean that we are bad at predicting exponential growth and disruptive technology."
https://www.lesswrong.com/posts/qRtD4WqKRYEtT5pi3/the-next-decades-might-be-wild
Sam Clarke and Diane Cooke on the lessons to be learned for AI governance from early electricity regulation.
Paul Christiano on appying alignment techniques to existing AI systems: "I worry that companies using alignment to help train extremely conservative and inoffensive systems could lead to backlash against the idea of AI alignment itself."
https://ai-alignment.com/ai-alignment-is-distinct-from-its-near-term-applications-81300500ad2e
Some arguments in favor and responses to common objections.
https://aligned.substack.com/p/alignment-optimism
This is a collection of AI Forecasting research ideas, prepared by some Epoch employees in a personal capacity
https://docs.google.com/document/d/1Vva6AW_udvfavtldWuGbqf8NbbzsgEH8mESEX6NlOvM
"It is impossible to predict the future perfectly. It depends on fundamentally random factors: ideas and actions as well as currently unknown technological and biological limits. But it is my job to explore the possibilities, and I think the most likely case is vast “speciation” – when a species splits into several others."
https://theconversation.com/cyborgs-v-holdout-humans-what-the-world-might-be-like-if-our-species-survives-for-a-million-years-195051
"There are more than twice as many americans 65 and older as there are under 6 years old"
"Opening up the flow of immigrants would begin to pay off quickly"
https://www.vox.com/the-highlight/23436211/overpopulation-population-8-billion-people
Holden Karnofsky argues that it is hard to know whether AI safety research is actually making AI systems safer.
Séb Krier on AI from Superintelligence to ChatGPT, from the latest issue of Works in Progress: "Breakthroughs in artificial intelligence are forcing skeptics to eat their words. We should take its risks seriously too."
Richard Williamson on Pandemic prevention as fire-fighting: "Fire has almost disappeared as a cause of death in the developed world. A similar approach could do the same for infectious diseases."
Founders Pledge explains how the Patient Philanthropy Fund and the Global Catastrophic Risks Funds differ from each other—and how they complement one another.
The Centre on Long-Term Risk is raising funds to support their work on s-risks, Cooperative AI, acausal trade, and general longtermism.
"William MacAskill’s latest book presents itself as an introduction to the burgeoning longtermist movement. But his views are eccentric – even within the movement he founded."
https://asteriskmag.com/issues/1/review-what-we-owe-the-future
"Nearly halfway into my year at OpenAI, still reeling from the FTX collapse, I feel like it’s finally time to start blogging my AI safety thoughts—starting with a little appetizer course today, more substantial fare to come."
https://scottaaronson.blog/?p=6821
"[W]e believe that while AI will continue to yield breakthroughs in many areas of human endeavor, we will not see a unitary nigh-omnipotent AI system that acts autonomously to pursue long-term goals. Concretely, even if a successful long-term AI system could be constructed, we believe that this is not a domain where AI will have a significant “competitive advantage” over humans."
https://www.lesswrong.com/posts/zB3ukZJqt3pQDw9jz/ai-will-change-the-world-but-won-t-take-it-over-by-playing-3
AI gets smarter, more capable, and more world-transforming every day. Here’s why that might not be a good thing.
https://www.vox.com/the-highlight/23447596/artificial-intelligence-agi-openai-gpt3-existential-risk-human-extinction
Despite their brief history, computers and AI have fundamentally changed what we see, what we know, and what we do. Little is as important for the future of the world, and our own lives, as how this history continues.
https://ourworldindata.org/brief-history-of-ai
How AI gets built is currently decided by a small group of technologists. As this technology is transforming our lives, it should be in all of our interest to become informed and engaged.
https://ourworldindata.org/ai-impact
After 9/11, the US spent trillions on foreign interference and declared a war on terror. In comparison, following the COVID pandemic—which has killed over a million US citizens—the investment to prevent pandemics has been minuscule.
https://thebulletin.org/2022/11/investing-in-pandemic-prevention-is-essential-to-defend-against-future-outbreaks/#post-heading
Nonlinear is offering $500 prizes for writing posts that expand on Holden Karnofsky's Most Important Century series.
https://forum.effectivealtruism.org/posts/4XK5zkyv94voC8Fjr/announcing-the-most-important-century-writing-prize
For a few monmths, the Global Priorities Institute has been releasing summaries of some of their papers. If these are still too long for you, you can now read Jack Malde's "mini-summaries".
https://forum.effectivealtruism.org/posts/xHrN4APvEjCkC4ceb/mini-summaries-of-gpi-papers
Founders Pledge just launched a new addition to its funds: the Global Catastrophic Risks Fund.
https://forum.effectivealtruism.org/posts/ZJpPvxXdimPgxFp2j/announcing-the-founders-pledge-global-catastrophic-risks-1
"In this post, I explore the probability a catastrophe that caused civilizational collapse might lead to indefinite technological stagnation (and eventual human extinction) — even if it didn’t cause extinction in the very short term"
https://forum.effectivealtruism.org/posts/Nc9fCzjBKYDaDJGiX/what-is-the-likelihood-that-civilizational-collapse-would-1
"Compared to the problem of AI—where the economic forces arrayed in favor of “ignore safety and rush ahead” are enormous and the argument for expecting catastrophe much murkier and more abstract—the problem of getting a sane civilizational response to pandemics (in the wake of a literal pandemic!) is ridiculously easier. And—despite valiant effort!—we've been able to do approximately nothing."
https://forum.effectivealtruism.org/posts/8tosr8XzqvYXR9ckB/warning-shots-probably-wouldn-t-change-the-picture-much
Before he wrote The Lord of the Rings, the author JRR Tolkien coined a word – "eucatastrophe" – that scholars would still be writing about 70 years later. What did he mean, and why could it relate to the very real story of humanity?
https://www.bbc.com/future/article/20221005-eucatastrophe-tolkiens-word-for-the-anti-doomsday
Maxwell Tabarrok argues that discovering alien life would make the difference in value between events that kill all vs. almost all humans much less significant, since aliens would then be expected to realize humanity's long-term potential.
https://forum.effectivealtruism.org/posts/wqmY98m3yNs6TiKeL/parfit-singer-aliens
Are you interested in doing a PhD or Masters on something altruistic? The MIT Media Lab has a new fellowship funded by the FTX Future Fund for "ambitiously benevolent projects that could not be accomplished anywhere else".
Humanity now has the beginnings of a true defense against asteroids.
https://www.vox.com/future-perfect/2022/9/27/23374653/nasa-asteroids-dart-existential-risk-extinction-planetary-defense-didymos
The DeepMind AGI Safety team clarifies AI X-risk.
https://www.lesswrong.com/posts/GctJD5oCDRxCspEaZ/clarifying-ai-x-risk
Abstract. Advocates of longtermism point out that interventions which focus on improving the prospects of people in the very far future will, in expectation, bring about a significant amount of good. Indeed, in expectation, such long-term interventions bring about far more good than their short-term counterparts. As such, longtermists claim we have compelling moral reason to prefer long-term interventions. In this paper, I show that longtermism is in conflict with plausible deontic scepticism about aggregation. I do so by demonstrating that, from both an ex-ante and ex-post perspective, longtermist interventions – and, in particular, those which aim to mitigate catastrophic risk – typically generate extremely weak claims of assistance from future people.
https://globalprioritiesinstitute.org/wp-content/uploads/Emma-J-Curran-Longtermism-aggregation-and-catastrophic-risk.pdf
Jesse Clifton, Samuel Martin and Anthony DiGiovanni on when AGIs would engage in conflict.
https://longtermrisk.org/will-agis-avoid-conflict-by-default/
Benjamin Hilton gives an initial evaluation of S-risks as a pressing issue.
https://80000hours.org/problem-profiles/s-risks/
Benjamin Hilton gives an initial evaluation of whole brain emulation as a pressing issue.
https://80000hours.org/problem-profiles/whole-brain-emulation/
Arden Koehler gives an initial evaluartion of the risks from malevolent actors.
https://80000hours.org/problem-profiles/risks-from-malevolent-actors/
Benjamin Hilton gives an exploratory problem profile of the risks of stable totalitarianism.
https://80000hours.org/problem-profiles/risks-of-stable-totalitarianism/
The Future of Life Institute is launching its 2023 cohort of PhD and postdoctoral fellowships to study AI existential safety: that is, research that analyzes the most probable ways in which AI technology could cause an existential catastrophe, and which types of research could minimize existential risk; and technical research which could, if successful, assist humanity in reducing the existential risk posed by highly impactful AI technology to extremely low levels. More information about the 2022 cohort can be found here.
https://grants.futureoflife.org/
Lukas Finnveden, Jess Riedel and Carl Shulman argue that AGI would allow perfect preservation of a wide variety of values and development of AGI-based institutions that would competently pursue any such values for a very long time.
https://forum.effectivealtruism.org/posts/KqCybin8rtfP3qztq/agi-and-lock-in
Steven Byrnes on what it takes to defend the world against out-of-control AGIs.
https://forum.effectivealtruism.org/posts/aBoEndTXdt6s3Bynm/what-does-it-take-to-defend-the-world-against-out-of-control
This is a response to the recent "Counterarguments to the basic AI x-risk case", by Katja Grace. https://www.alignmentforum.org/posts/GQat3Nrd9CStHyGaq/response-to-katja-grace-s-ai-x-risk-counterarguments
Dylan Matthews wrote a short profile of Hilary Greaves for the Future Perfect 50 series: https://www.vox.com/future-perfect/23357140/future-perfect-50-hilary-greaves-professor-philosophy-oxford
Max Tegmark: "Many people have asked me what I think the odds are of an imminent major US-Russia nuclear war. My current estimate is about the same as losing in Russian roulette: one in six." https://www.lesswrong.com/posts/Dod9AWz8Rp4Svdpof/why-i-think-there-s-a-one-in-six-chance-of-an-imminent
Katja Grace gives a list of counterarguments to the basic argument for existential risk from artificial superintelligent systems. https://aiimpacts.org/counterarguments-to-the-basic-ai-x-risk-case/
William MacAskill is Associate Professor of Philosophy at Oxford University and a Senior Research Fellow at the Global Priorities Institute. He is the author of the book What We Owe the Future_, from which this excerpt is adapted._
https://bigthink.com/the-future/spaceguard-what-we-owe-the-future/
Jacob Barrett and Andreas Schmidt outline some central questions in longtermist political philosophy. https://globalprioritiesinstitute.org/longtermist-political-philosophy-an-agenda-for-future-research-jacob-barrett-global-priorities-institute-university-of-oxford-and-andreas-schmidt-university-of-groningen/
After recent events in Ukraine, Samotsvety updated their probabilities of nuclear war. https://forum.effectivealtruism.org/posts/2nDTrDPZJBEerZGrk/samotsvety-nuclear-risk-update-october-2022
Jacob Stern interviews Will MacAskill about longtermism and effectiev altruism. https://www.theatlantic.com/health/archive/2022/09/oxford-philosophy-professor-william-macaskill-effective-altruism-interview/671597/
Eli Lifland has written a new review of Joseph Carlsmith's report 'Is power-seeking AI an existential risk?' https://docs.google.com/document/d/1nCqjJXydfPQGRTKT71jQn8yXi4FSHnVSdfZbhQUMa1I/edit
On September 26 at 11.15pm UTC, NASA’s DART mission (Double Asteroid Redirection Test) will be the first to deliberately and measurably change the motion of a significant body in our Solar System. In other words, it will smash into an asteroid. The mission will provide the first test of a technique that could be used in the future – to redirect any asteroids we detect on a collision course with Earth. https://theconversation.com/in-a-world-first-nasas-dart-mission-is-about-to-smash-into-an-asteroid-what-will-we-learn-189391
The Future Fund just announced a competition offering up to $1.5 million to anyone who publishes an analysis that makes the FF team significantly change its views about AGI timelines or AGI x-risk. https://ftxfuturefund.org/announcing-the-future-funds-ai-worldview-prize/
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.
Future Matters Reader uses text-to-speech software to convert into audio the writings summarized in Future Matters.