Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Case study: Reducing catastrophic risk from inside the US bureaucracy, published by Tom Green on June 2, 2022 on The Effective Altruism Forum. This is the first and shorter of two versions of this post; I plan to post the longer version a few weeks after this one. Summary This post outlines how a small group of focused, committed individuals successfully reduced catastrophic risk from inside the US federal bureaucracy, and considers potential lessons from their experiences. Before the mid 70s, US nuclear weapon design didn't fully account for the effects of “abnormal environments” like fires or plane crashes. Meanwhile, there were multiple “near-miss” accidents in exactly those types of environments, some of which came uncomfortably close to catastrophic detonations. Those, in turn, could have plausibly triggered all-out war if misinterpreted as intentional deployments. Engineers at Sandia National Laboratories did intensive design and development work to address the problem between 1968 and 1972. But it took until 1990 before older weapons that didn’t meet the new safety standards were removed from US Quick Reaction Alert. Why the delay? Stonewalling, evasion, and vested interests, both bureaucratic and military. Several of the Sandia engineers, led by Bob Peurifoy, advocated relentlessly over two decades for decision makers to see reason. (Acronym sidebar: at this point, I’m introducing an acronym to refer to Bob Peurifoy and his allies: the SEAs (Sandia Engineer-Advocates). I’m not wedded to it, but I’ve found it a useful shorthand when writing and I suspect it might make for a smoother reading experience compared to the alternatives). Key takeaways With low-ish confidence in this topic (see Framing, just below), here are my main takeaways in terms of potential lessons and points of inspiration: Peurifoy showed considerable personal agency in assuming the mantle of safety advocate as an engineer and manager at Sandia. Relatedly, he and the SEAs (that’s the Sandia Engineer-Advocates) persisted over a long period despite slow progress and significant opposition. The interventions of key individuals, including Senator John Glenn, were important catalysts for the SEAs’ eventual breakthrough. These individuals had credibility and influence, while also being less vulnerable to professional consequences than the SEAs. When the SEAs eventually got the opportunity to advocate directly to those with the power to effect change, their many years of continued effort meant that they had all the arguments and evidence ready to make their case quickly and effectively. Peurifoy was an expert weapon designer himself, combining strong technical and social skills. His excellence enabled him to grow in seniority over time, increasing his influence. The SEAs were a great team whose members used their collective strengths to push for change together. The SEAs worked within the system and pushed at its boundaries—e.g. by using leverage to insist on safety features in new weapons, or by briefing the press. It’s plausible that the SEAs could have achieved earlier or greater success with different approaches to strategy or communications. But they were working in a pretty intractable problem space—it’s likewise plausible that they just had to wait for the Overton window to shift in order to make their breakthrough (and that their advocacy may have contributed to that shift in part). Framing and methodology This is a low-ish confidence post. It’s the result of a few weeks’ research into most of the readily available sources; I’m not a trained journalist or historian, I’ve never worked for a government or a large bureaucracy, and I’m not an expert in nuclear weapons or any other content area covered here. Most of the main players are now dead, and many primary sources are still classified or ...