Should we lobby governments to impose a moratorium on AI research? Since we don’t enforce pauses on most new technologies, I hope the reader will grant that the burden of proof is on those who advocate for such a moratorium. We should only advocate for such heavy-handed government action if it's clear that the benefits of doing so would significantly outweigh the costs.[1] In this essay, I’ll argue an AI pause would increase the risk of catastrophically bad outcomes, in at least three different ways:

1. Reducing the quality of AI alignment research by forcing researchers to exclusively test ideas on models like GPT-4 or weaker.
2. Increasing the chance of a “fast takeoff” in which one or a handful of AIs rapidly and discontinuously become more capable, concentrating immense power in their hands.
3. Pushing capabilities research underground, and to countries with looser regulations and safety requirements.

Along the way, I’ll introduce an argument for optimism [...]

---

Outline:

(01:09) Feedback loops are at the core of alignment

(01:54) Alignment and robustness are often in tension

(03:21) Alignment is doing pretty well

(04:30) Alignment research was pretty bad during the last “pause”

(06:31) Fast takeoff has a really bad feedback loop

(08:43) Slow takeoff is the default (so don’t mess it up with a pause)

(09:25) Alignment optimism: AIs are white boxes

(09:49) Human and animal alignment is black box

(11:41) Status quo AI alignment methods are white box

(13:25) White box alignment in nature

(15:35) Realistic AI pauses would be counterproductive

(15:56) Realistic pauses are not international

(18:19) Realistic pauses don’t include hardware

(19:23) Hardware overhang is likely

(20:58) Likely consequences of a realistic pause

The original text contained 8 footnotes which were omitted from this narration.

---

First published:
September 16th, 2023

Source:
https://forum.effectivealtruism.org/posts/JYEAL8g7ArqGoTaX6/ai-pause-will-likely-backfire

--- Narrated by TYPE III AUDIO.