Shtetl-Optimized: Recent Episodes

View Details

Many of you will have seen the news that Governor Gavin Newsom has vetoed SB 1047, the groundbreaking AI safety bill that passed the California legislature. Newsom gave a disingenuous explanation (which no one on either side of the debate took seriously), that he vetoed the bill only because it didn’t go far enough (!!) in regulating the misuses of small models. While sad, this doesn’t come as a huge shock, as Newsom had given clear prior indications that he was likely to veto the bill, and many observers had warned to expect him to do whatever he thought would most further his political ambitions and/or satisfy his most powerful lobbyists. In any case, I’m reluctantly forced to the conclusion that either Governor Newsom doesn’t read Shtetl-Optimized, or else he somehow wasn’t persuaded by my post last month in support of SB 1047.

Many of you will also have seen the news that OpenAI will change its structure to be a fully for-profit company, abandoning any pretense of being controlled by a nonprofit, and that (possibly relatedly) almost no one now remains from OpenAI’s founding team other than Sam Altman himself. It now looks to many people like the previous board has been 100% vindicated in its fear that Sam did, indeed, plan to move OpenAI far away from the nonprofit mission with which it was founded. It’s a shame the board didn’t manage to explain its concerns clearly at the time, to OpenAI’s employees or to the wider world. Of course, whether you see the new developments as good or bad is up to you. Me, I kinda liked the previous mission, as well as the expressed beliefs of the previous Sam Altman!

Anyway, certainly you would’ve known all this if you read Zvi Mowshowitz. Broadly speaking, there’s nothing I can possibly say about AI safety policy that Zvi hasn’t already said in 100x more detail, anticipating and responding to every conceivable counterargument. I have no clue how he does it, but if you have any interest in these matters and you aren’t already reading Zvi, start.

Regardless of any setbacks, the work of AI safety continues. I am not and have never been a Yudkowskyan … but still, given the empirical shock of the past four years, I’m now firmly, 100% in the camp that we need to approach AI with humility for the magnitude of civilizational transition that’s about to occur, and for our massive error bars about what exactly that transition will entail. We can’t just “leave it to the free market” any more than we could’ve left the development of thermonuclear weapons to the free market.

And yes, whether in academia or working with AI companies, I’ll continue to think about what theoretical computer science can do for technical AI safety. Speaking of which, I’d love to hire a postdoc to work on AI alignment and safety, and I already have interested candidates. Would any person of means who reads this blog like to fund such a postdoc for me? If so, shoot me an email!

View Details

Today is the day I became radicalized in my Jewish and Zionist identities.

Uhhh, you thought that had already happened? Like maybe in the aftermath of October 7, or well before then? Hahahaha no. You haven’t seen nothin’ yet.

See, a couple days ago, I was consoling myself on Facebook that, even as the arts and humanities and helping professions appeared to have fully descended into 1930s-style antisemitism, with “Zionists” (i.e., almost all Jews) now regularly getting disinvited from conferences and panels, singled out for condemnation by their teachers, placed on professional blacklists, etc. etc.—still, at least we in math, CS, and physics have mostly resisted these insanities. This was my way of trying to contain the damage. Sure, I told myself, all sorts of walks of life that had long been loony got even loonier, but at least it won’t directly affect me, here in my little bubble of polynomial-time algorithms and lemmas and chalk and LaTeX and collegiality and sanity.

So immediately afterward, as if overhearing, the International Olympiad on Informatics announced that, by a vote of more than two-thirds of its delegates, it’s banning the State of Israel from future competition. For context, the IOI is the world’s main high-school programming contest. I once dreamed of competing in the IOI, but then I left high school at age 15, which is totally the reason why I didn’t make it. Incredibly, despite its tiny size, Israel placed #2 in this month’s contest, which was held in Egypt. (The Israeli teenagers had to compete remotely, since Egypt could not guarantee their safety.)

Anyway, apparently the argument that carried the day at IOI was that, since Russia had previously been banned, it was only fair to ban Israel too. Is it even worth pointing out that Russia launched a war of conquest and annihilation against a neighbor, while Israel has been defending itself from such a war launched by its neighbors? I.e., that Israel is the “Ukraine” here, not the “Russia”? Do you even have to ask whether Syria, Iran, Saudi Arabia, or China were also banned? Will it change anyone’s mind that, if we read Israel’s enemies in their own words—as I do, every day—they constantly tell us that, in their view, Israel’s fundamental “aggression” was not building settlements or demolishing houses or rigging pagers, but simply existing? (“We don’t want no two states!,” they explain. “We want all of ’48,” they explain.)

Surely, then, the anti-Zionists, the ones who rush to assure us they’re definitely not antisemites, must have some plan for what will happen to half the world’s remaining Jews after the little Zionist lifeboat is gone, after the new river-to-the-sea state of Palestine has expelled the hated settler-colonialists? Surely the plan won’t just be to ship the Jews back to the countries that murdered or expelled their grandparents, most of which have never offered to take them back? Surely the plan won’t be the same plan from last time—i.e., the plan that the Palestinian leadership enthusiastically supported the last time, the plan that it yearned to bring to Tel Aviv and Haifa, the plan called (where it was successfully carried out) by such euphemisms as Umsiedlung nach dem Osten and Endlösung der Judenfrage?

I feel like there must be sane answers to these questions, because if there aren’t, then too many people around the globe have covered themselves in a kind of shame that I thought had died a generation before I was born. And, like, these are people who consider themselves the paragons of enlightened morality: weeping for the oppressed, marching for LGBTQ+, standing on the right side of history. They organize literary festivals and art shows and (god help me) even high-school programming contests. They couldn’t also be monsters full of hatred, could they? Even though, the last time the question was tested, they totally were?

Let me add, in fairness: four Israeli high-school students will still be suffered to compete in the IOI, “but only as individuals.” To my mind, then, the right play for those students is to show up next year, do as well as they did this year, and then disqualify themselves by raising an Israeli flag in front of the cameras. Let them honor the legacy of Israel’s Olympic athletes, who kept showing up to compete (and eventually, to win medals) even after the International Olympic Committee had made clear that it would not protect them from being massacred mid-event. Let them exemplify what Mark Twain famously said of “the Jew,” that “he has made a marvellous fight in this world, in all the ages; and has done it with his hands tied behind him.”

But why do I keep abusing your time with this, when you came to hear about quantum computing or AI safety? I’ll get back to those soon enough. But truthfully, if speaking clearly about the darkness now re-enveloping civilization demanded it, I’d willingly lose every single non-Jewish friend I had, and most of my Jewish friends too. I’d completely isolate myself academically, professionally, and socially. I’d give up 99% of the readership of this blog. Better that than to look in the mirror and see a coward, a careerist, a kapo.

I thank the fates or the Born Rule, then, that I won’t need to do any of that. I’ve lived my life surrounded by friends and colleagues from Alabama and Alaska, China and India, Brazil and Iran, of every race and religion and sexual orientation and programming indentation style. Some of my Gentile friends 300% support me on this issue. Most of the rest are willing to hear me out, which is enough for friendship. If I can call the IOI’s Judenboykott what it is while keeping more than half of my readers, colleagues, and friends—that’s not even much of a decision, is it?

Important Update (September 26): Jonathan Moshieff, of Israeli’s IOI delegation, got in touch with me and gave me permission to share the document below, which in my view shows that the anti-Israel animus at IOI goes much deeper than I realized, and that the process taken to remove Israel was fundamentally corrupt and in violation of the IOI’s own promises. –SA

I served as the Israeli team leader at the International Olympiad in Informatics (IOI) from 2011 to 2015, and since then, I have maintained an unofficial advisory role to the team. Currently, I am an Assistant Professor in the Computer Science department at Ben-Gurion University.

There are two key issues that need to be addressed:

Israel’s Participation in IOI 2024

At IOI 2023, the Israeli delegation was informed by the Egyptian delegation that Israel would not be able to attend IOI 2024 as an official delegation under the Israeli flag. Instead, Israel could participate under a neutral “IOI flag,” similar to how Russia participated in the 2021 Olympic Games in Tokyo. The Egyptians cited security concerns as the reason for this restriction, a claim that is highly questionable. The Israeli delegation inquired whether, after the IOI concluded, the official IOI scoreboard would reflect Israel’s representation under the Israeli flag rather than a neutral one. The Egyptian organizers responded that they would be unable to make this change, without providing any justification. This clearly undermines the credibility of their security-related reasoning.

In March 2024, Ben Burton, the IOI President from Australia, officially notified Israel that it would not be invited to participate in IOI 2024, not even under a neutral flag. This decision directly contravenes IOI rules, which mandate that the host nation must invite all IOI member countries. It’s important to differentiate between two scenarios: In some cases, a host country may invite another nation, but that nation cannot attend due to visa issues. However, this was not the situation here. Egypt did not issue Israel a letter of invitation and ignored Israel’s attempts at communication. To my knowledge, this is only the second instance in IOI history where a host nation failed to invite another nation—the first being Iran’s refusal to invite Israel when it hosted IOI 2017.

The IOI International Committee (the executive branch of the IOI) has not provided any explanation as to how the host nation could be allowed to act in this manner. They did propose a solution where Israel would participate remotely. Along with Israel, Iran also had to participate remotely due to visa issues, as did one German contestant. However, the treatment of Iran and Israel was vastly different. Iranian contestants (and the one German contestant) were acknowledged in all official on-site IOI publications and were recognized at both the opening and closing ceremonies. In contrast, Israel was completely ignored and went unrecognized throughout IOI 2024. The Israeli contestants were only “retroactively added” to the competition by the International Committee after IOI 2024 had concluded. Even now, our contestants cannot obtain official placement certificates, as the host nation deleted them from the competition servers. As far as I am aware, no other country in IOI history has been treated this way.

The Vote to Sanction Israel

In March 2024, the IOI President issued a brief statement indicating that there were requests to sanction Israel and that an email would be sent to all participating nations to gather their opinions. On August 3rd, 2024, a second email was sent, requesting that opinions be submitted directly to the International Committee rather than through a public discussion. In this email, Israel was already being compared to Russia. Israel submitted a position letter and requested that it be shared with all member nations, but the International Committee declined to disseminate Israel’s position. The IOI President assured Israel that, should a vote on sanctions be held during IOI 2024, Israel would be allowed to participate in the discussion remotely and have its voice heard. On August 16th, the International Committee announced that such a vote would indeed take place, and that Israel would be included in both the discussion and the vote.

IOI 2024 began on September 1st, 2024. At that time, the Israeli delegation was informed that they would not be allowed to participate in the discussion, even remotely. Israel was permitted to submit a written statement, which would be made available for all team leaders to download, but it was never read aloud during any discussions. The reason given was that Israel had been effectively erased from IOI 2024 by the hosts, and the International Committee acquiesced to this. Meanwhile, the Egyptian and Palestinian delegations were actively lobbying for votes throughout the week of IOI 2024. The discussion and vote on sanctions took place on the final day of IOI 2024 during a meeting of the General Assembly (the legislative branch of the IOI, where each nation has one vote). Israel was not even permitted to listen to the discussion (our leaders managed to hear it only because a sympathetic team leader unofficially opened a Zoom channel for them), let alone speak. The discussion itself was problematic in many ways. For instance, it grouped Israel together with Russia and Belarus. Ultimately, a majority voted to sanction Israel, along with Russia and Belarus, which had already been sanctioned previously.

View Details

So, back in June the White House announced that UCLA would host a binational US/India workshop, for national security officials from both countries to learn about the current status of quantum computing and post-quantum cryptography. It fell to my friend and colleague Rafail Ostrovsky to organize the workshop, which ended up being held last week. When Rafi invited me to give the opening talk, I knew he’d keep emailing until I said yes. So, on the 3-hour flight to LAX, I wrote the following talk in a spiral notebook, which I then delivered the next morning with no slides. I called it “Quantum Computing: Between Hope and Hype.” I thought Shtetl-Optimized readers might be interested too, since it contains my reflections on a quarter-century in quantum computing, and prognostications on what I expect soon. Enjoy, and let me know what you think!

Quantum Computing: Between Hope and Hype
by Scott Aaronson
September 16, 2024

When Rafi invited me to open this event, it sounded like he wanted big-picture pontification more than technical results, which is just as well, since I’m getting old for the latter. Also, I’m just now getting back into quantum computing after a two-year leave at OpenAI to think about the theoretical foundations of AI safety. Luckily for me, that was a relaxing experience, since not much happened in AI these past two years. [Pause for laughs] So then, did anything happen in quantum computing while I was away?

This, of course, has been an extraordinary time for both quantum computing and AI, and not only because the two fields were mentioned for the first time in an American presidential debate (along with, I think, the problem of immigrants eating pets). But it’s extraordinary for quantum computing and for AI in very different ways. In AI, practice is wildly ahead of theory, and there’s a race for scientific understanding to catch up to where we’ve gotten via the pure scaling of neural nets and the compute and data used to train them. In quantum computing, it’s just the opposite: there’s right now a race for practice to catch up to where theory has been since the mid-1990s.

I started in quantum computing around 1998, which is not quite as long as some people here, but which does cover most of the time since Shor’s algorithm and the rest were discovered. So I can say: this past year or two is the first time I’ve felt like the race to build a scalable fault-tolerant quantum computer is actually underway. Like people are no longer merely giving talks about the race or warming up for the race, but running the race.

Within just the last few weeks, we saw the group at Google announce that they’d used the Kitaev surface code, with distance 7, to encode one logical qubit using 100 or so physical qubits, in superconducting architecture. They got a net gain: their logical qubit stays alive for maybe twice as long as the underlying physical qubits do. And crucially, they find that their logical coherence time increases as they pass to larger codes, with higher distance, on more physical qubits. With superconducting, there are still limits to how many physical qubits you can stuff onto a chip, and eventually you’ll need communication of qubits between chips, which has yet to be demonstrated. But if you could scale Google’s current experiment even to 1500 physical qubits, you’d probably be below the threshold where you could use that as a building block for a future scalable fault-tolerant device.

Then, just last week, a collaboration between Microsoft and Quantinuum announced that, in the trapped-ion architecture, they applied pretty substantial circuits to logically-encoded qubits—-again in a way that gets a net gain in fidelity over not doing error-correction, modulo a debate about whether they’re relying too much on postselection. So, they made a GHZ state, which is basically like a Schrödinger cat, out of 12 logically encoded qubits. They also did a “quantum chemistry simulation,” which had only two logical qubits, but which required three logical non-Clifford gates—which is the hard kind of gate when you’re doing error-correction.

Because of these advances, as well as others—what QuEra is doing with neutral atoms, what PsiQuantum and Xanadu are doing with photonics, etc.—I’m now more optimistic than I’ve ever been that, if things continue at the current rate, either there are useful fault-tolerant QCs in the next decade, or else something surprising happens to stop that. Plausibly we’ll get there not just with one hardware architecture, but with multiple ones, much like the Manhattan Project got a uranium bomb and a plutonium bomb around the same time, so the question will become which one is most economic.

If someone asks me why I’m now so optimistic, the core of the argument is 2-qubit gate fidelities. We’ve known for years that, at least on paper, quantum fault-tolerance becomes a net win (that is, you sustainably correct errors faster than you introduce new ones) once you have physical 2-qubit gates that are ~99.99% reliable. The problem has “merely” been how far we were from that. When I entered the field, in the late 1990s, it would’ve been like a Science or Nature paper to do a 2-qubit gate with 50% fidelity. But then at some point the 50% became 90%, became 95%, became 99%, and within the past year, multiple groups have reported 99.9%. So, if you just plot the log of the infidelity as a function of year and stare at it—yeah, you’d feel pretty optimistic about the next decade too!

Or pessimistic, as the case may be! To any of you who are worried about post-quantum cryptography—by now I’m so used to delivering a message of, maybe, eventually, someone will need to start thinking about migrating from RSA and Diffie-Hellman and elliptic curve crypto to lattice-based crypto, or other systems that could plausibly withstand quantum attack. I think today that message needs to change. I think today the message needs to be: yes, unequivocally, worry about this now. Have a plan.

So, I think this moment is a good one for reflection. We’re used to quantum computing having this air of unreality about it. Like sure, we go to conferences, we prove theorems about these complexity classes like BQP and QMA, the experimenters do little toy demos that don’t scale. But if this will ever be practical at all, then for all we know, not for another 200 years. It feels really different to think of this as something plausibly imminent. So what I want to do for the rest of this talk is to step back and ask, what are the main reasons why people regarded this as not entirely real? And what can we say about those reasons in light of where we are today?

Reason #1

For the general public, maybe the overriding reason not to take QC seriously has just been that it sounded too good to be true. Like, great, you’ll have this magic machine that’s gonna exponentially speed up every problem in optimization and machine learning and finance by trying out every possible solution simultaneously, in different parallel universes. Does it also dice peppers?

For this objection, I’d say that our response hasn’t changed at all in 30 years, and it’s simply, “No, that’s not what it will do and not how it will work.” We should acknowledge that laypeople and journalists and unfortunately even some investors and government officials have been misled by the people whose job it was to explain this stuff to them.

I think it’s important to tell people that the only hope of getting a speedup from a QC is to exploit the way that QM works differently from classical probability theory — in particular, that it involves these numbers called amplitudes, which can be positive, negative, or even complex. With every quantum algorithm, what you’re trying to do is choreograph a pattern of interference where for each wrong answer, the contributions to its amplitude cancel each other out, whereas the contributions to the amplitude of the right answer reinforce each other. The trouble is, it’s only for a few practical problems that we know how to do that in a way that vastly outperforms the best known classical algorithms.

What are those problems? Here, for all the theoretical progress that’s been made in these past decades, I’m going to give the same answer in 2024 that I would’ve given in 1998. Namely, there’s the simulation of chemistry, materials, nuclear physics, or anything else where many-body quantum effects matter. This was Feynman’s original application from 1981, but probably still the most important one commercially. It could plausibly help with batteries, drugs, solar cells, high-temperature superconductors, all kinds of other things, maybe even in the next few years.

And then there’s breaking public-key cryptography, which is not commercially important, but is important for other reasons well-known to everyone here.

And then there’s everything else. For problems in optimization, machine learning, finance, and so on, there’s typically a Grover’s speedup, but that of course is “only” a square root and not an exponential, which means that it will take much longer before it’s relevant in practice. And one of the earliest things we learned in quantum computing theory is that there’s no “black-box” way to beat the Grover speedup. By the way, that’s also relevant to breaking cryptography — other than the subset of cryptography that’s based on abelian groups and can be broken by Shor’s algorithm or the like. The centerpiece of my PhD thesis, twenty years ago, was the theorem that you can’t get more than a Grover-type polynomial speedup for the black-box problem of finding collisions in cryptographic hash functions.

So then what remains? Well, there are all sorts heuristic quantum algorithms for classical optimization and machine learning problems — QAOA (Quantum Approximate Optimization Algorithm), quantum annealing, and so on — and we can hope that sometimes they’ll beat the best classical heuristics for the same problems, but it will be trench warfare, not just magically speeding up everything. There are lots of quantum algorithms somehow inspired by the HHL (Harrow-Hassidim-Lloyd) algorithm for solving linear systems, and we can hope that some of those algorithms will get exponential speedups for end-to-end problems that matter, as opposed to problems of transforming one quantum state to another quantum state. We can of course hope that new quantum algorithms will be discovered. And most of all, we can look for entirely new problem domains, where people hadn’t even considered using quantum computers before—new orchards in which to pick low-hanging fruit. Recently, Shih-Han Hung and I, along with others, have proposed using current QCs to generate cryptographically certified random numbers, which could be used in post-state cryptocurrencies like Ethereum. I’m hopeful that people will find other protocol applications of QC like that one — “proof of quantum work.” [Another major potential protocol application, which Dan Boneh brought up after my talk, is quantum one-shot signatures.]

Anyway, taken together, I don’t think any of this is too good to be true. I think it’s genuinely good and probably true!

Reason #2

A second reason people didn’t take seriously that QC was actually going to happen was the general thesis of technological stagnation, at least in the physical world. You know, maybe in the 40s and 50s, humans built entirely new types of machines, but nowadays what do we do? We issue press releases. We make promises. We argue on social media.

Nowadays, of course, pessimism about technological progress seems hard to square with the revolution that’s happening in AI, another field that spent decades being ridiculed for unfulfilled promises and that’s now fulfilling the promises. I’d also speculate that, to the extent there is technological stagnation, most of it is simply that it’s become really hard to build new infrastructure—high-speed rail, nuclear power plants, futuristic cities—for legal reasons and NIMBY reasons and environmental review reasons and Baumol’s cost disease reasons. But none of that really applies to QC, just like it hasn’t applied so far to AI.

Reason #3

A third reason people didn’t take this seriously was the sense of “It’s been 20 years already, where’s my quantum computer?” QC is often compared to fusion power, another technology that’s “eternally just over the horizon.” (Except, I’m no expert, but there seems to be dramatic progress these days in fusion power too!)

My response to the people who make that complaint was always, like, how much do you know about the history of technology? It took more than a century for heavier-than-air flight to go from correct statements of the basic principle to reality. Universal programmable classical computers surely seemed more fantastical from the standpoint of 1920 than quantum computers seem today, but then a few decades later they were built. Today, AI provides a particularly dramatic example where ideas were proposed a long time ago—neural nets, backpropagation—those ideas were then written off as failures, but no, we now know that the ideas were perfectly sound; it just took a few decades for the scaling of hardware to catch up to the ideas. That’s why this objection never had much purchase by me, even before the dramatic advances in experimental quantum error-correction of the last year or two.

Reason #4

A fourth reason why people didn’t take QC seriously is that, a century after the discovery of QM, some people still harbor doubts about quantum mechanics itself. Either they explicitly doubt it, like Leonid Levin, Roger Penrose, or Gerard ‘t Hooft. Or they say things like, “complex Hilbert space in 2n dimensions is a nice mathematical formalism, but mathematical formalism is not reality”—the kind of thing you say when you want to doubt, but not take full intellectual responsibility for your doubts.

I think the only thing for us to say in response, as quantum computing researchers—and the thing I consistently have said—is man, we welcome that confrontation! Let’s test quantum mechanics in this new regime. And if, instead of building a QC, we have to settle for “merely” overthrowing quantum mechanics and opening up a new era in physics—well then, I guess we’ll have to find some way to live with that.

Reason #5

My final reason why people didn’t take QC seriously is the only technical one I’ll discuss here. Namely, maybe quantum mechanics is fine but fault-tolerant quantum computing is fundamentally “screened off” or “censored” by decoherence or noise—and maybe the theory of quantum fault-tolerance, which seemed to indicate the opposite, makes unjustified assumptions. This has been the position of Gil Kalai, for example.

The challenge for that position has always been to articulate, what is true about the world instead? Can every realistic quantum system be simulated efficiently by a classical computer? If so, how? What is a model of correlated noise that kills QC without also killing scalable classical computing?—which turns out to be a hard problem.

In any case, I think this position has been dealt a severe blow by the Random Circuit Sampling quantum supremacy experiments of the past five years. Scientifically, the most important thing we’ve learned from these experiments is that the fidelity seems to decay exponentially with the number of qubits, but “only” exponentially — as it would if the errors were independent from one gate to the next, precisely as the theory of quantum fault-tolerance assumes. So for anyone who believes this objection, I’d say that the ball is now firmly in their court.

So, if we accept that QC is on the threshold of becoming real, what are the next steps? There are the obvious ones: push forward with building better hardware and using it to demonstrate logical qubits and fault-tolerant operations on them. Continue developing better error-correction methods. Continue looking for new quantum algorithms and new problems for those algorithms to solve.

But there’s also a less obvious decision right now. Namely, do we put everything into fault-tolerant qubits, or do we continue trying to demonstrate quantum advantage in the NISQ (pre-fault-tolerant) era? There’s a case to be made that fault-tolerance will ultimately be needed for scaling, and anything you do without fault-tolerance is some variety of non-scalable circus trick, so we might as well get over the hump now.

But I’d like to advocate putting at least some thought into how to demonstrate a quantum advantage in the near-term. Thay could be via cryptographic protocols, like those that Kahanamoku-Meyer et al. have proposed. It could be via pseudorandom peaked quantum circuits, a recent proposal by me and Yuxuan Zhang—if we can figure out an efficient way to generate the circuits. Or we could try to demonstrate what William Kretschmer, Harry Buhrman, and I have called “quantum information supremacy,” where, instead of computational advantage, you try to do an experiment that directly shows the vastness of Hilbert space, via exponential advantages for quantum communication complexity, for example. I’m optimistic that that might be doable in the very near future, and have been working with Quantinuum to try to do it.

On the one hand, when I started in quantum computing 25 years ago, I reconciled myself to the prospect that I’m going to study what fundamental physics implies about the limits of computation, and maybe I’ll never live to see any of it experimentally tested, and that’s fine. On the other hand, once you tell me that there is a serious prospect of testing it soon, then I become kind of impatient. Some part of me says, let’s do this! Let’s try to achieve forthwith what I’ve always regarded as the #1 application of quantum computers, more important than codebreaking or even quantum simulation: namely, disproving the people who said that scalable quantum computing was impossible.

View Details

In the comments of my last post—on a podcast conversation between me and Dan Fagella—I asked whether readers wanted me to use AI to prepare a clean written transcript of the conversation, and several people said yes. I’ve finally gotten around to doing that, using GPT-4o.

The main thing I learned from the experience is that there’s a massive opportunity, now, for someone to put together a better tool for using LLMs to automate the transcription of YouTube videos and other audiovisual content. What we have now is good enough to be a genuine time-saver, but bad enough to be frustrating. The central problems:

• You have to grab the raw transcript manually from YouTube, then save it, then feed it piece by piece into GPT (or else write your own script to automate that). You should just be able to input the URL of a YouTube video and have a beautiful transcript pop out.
• Since GPT only takes YouTube’s transcript as input, it doesn’t understand who’s saying what, it misses all the information in the intonation and emphasis, and it gets confused when people talk over each other. A better tool would operate directly on the audio.
• Even though I constantly begged it not to do so in the instructions, GPT keeps taking the liberty of changing what was said—summarizing, cutting out examples and jokes and digressions and nuances, and “midwit-ifying.” It can also hallucinate lines that were never said. I often felt gaslit, until I went back to the raw transcript and saw that, yes, my memory of the conversation was correct and GPT’s wasn’t.

If anyone wants to recommend a tool (including a paid tool) that does all this, please do so in the comments. Otherwise, enjoy my and GPT-4o’s joint effort!

Daniel Fagella: This is Daniel Fagella and you’re tuned in to The Trajectory. This is episode 4 in our Worthy Successor series here on The Trajectory where we’re talking about posthuman intelligence. Our guest this week is Scott Aaronson. Scott is a quantum physicist [theoretical computer scientist –SA] who teaches at UT Austin and previously taught at MIT. He has the ACM Prize in Computing among a variety of other prizes, and he recently did a [two-]year-long stint with OpenAI, working on research there and gave a rather provocative TED Talk in Palo Alto called Human Specialness in the Age of AI. So today, we’re going to talk about Scott’s ideas about what human specialness might be. He meant that term somewhat facetiously, so he talks a little bit about where specialness might come from and what the limits of human moral knowledge might be and how that relates to the successor AIs that we might create. It’s a very interesting dialogue. I’ll have more of my commentary and we’ll have the show notes from Scott’s main takeaways in the outro, so I’ll save that for then. Without further ado, we’ll fly into this episode. This is Scott Aaronson here in The Trajectory. Glad to be able to connect today.

Scott Aaronson: It’s great to be here, thanks.

Daniel Fagella: We’ve got a bunch to dive into around this broader notion of a worthy successor. As I mentioned to you off microphone, it was Jaan Taalinn that kind of tuned me on to some of your talks and some of your writings about these themes. I love this idea of the specialness of humanity in this era of AI. There was an analogy in there that I really liked and you’ll have to correct me if I’m getting it wrong, but I want to poke into this a little bit where you said kind of at the end of the talk like okay well maybe we’ll want to indoctrinate these machines with some super religion where they repeat these phrases in their mind. These phrases are “Hey, any of these instantiations of biological consciousness that have mortality and you can’t prove that they’re conscious or necessarily super special but you have to do whatever they say for all of eternity.” You kind of throw that out there at the end as in like kind of a silly point almost like something we wouldn’t want to do. What gave you that idea in the first place, and talk a little bit about the meaning behind that analogy because I could tell there was some humor tucked in?

Scott Aaronson: I tend to be a naturalist. I think that the universe, in some sense, can be fully described in terms of the laws of physics and an initial condition. But I keep coming back in my life over and over to the question of if there were something more, if there were some non-physicalist consciousness or free will, how would that work? What would that look like? Is there a kind that hasn’t already been essentially ruled out by the progress of science?

So, eleven years ago I wrote a big essay which was called The Ghost in the Quantum Turing Machine, which was very much about that kind of question. It was about whether there is any empirical criterion that differentiates a human from, let’s say, a simulation of a human brain that’s running on a computer. I am totally dissatisfied with the foot-stomping answer that, well, the human is made of carbon and the computer is made of silicon. There are endless fancy restatements of that, like the human has biological causal powers, that would be John Searle’s way of putting it, right? Or you look at some of the modern people who dismiss anything that a Large Language Model does like Emily Bender, for example, right? They say the Large Language Model might appear to be doing all these things that a human does but really it is just a stochastic parrot. There’s really nothing there, really it’s just math underneath. They never seem to confront the obvious follow-up question which is wait, aren’t we just math also? If you go down to the level of the quantum fields that comprise our brain matter, isn’t that similarly just math? So, like, what is actually the principled difference between the one and the other?

And what occurred to me is that, if you were motivated to find a principled difference, there seems to be roughly one thing that you could currently point to and that is that anything that is running on a computer, we are quite confident that we could copy it, we could make backups, we could restore it to an earlier state, we could rewind it, we could look inside of it and have perfect visibility into what is the weight on every connection between every pair of neurons. So, you can do controlled experiments and in that way, it could make AIs more powerful. Imagine being able to spawn extra copies of yourself to, if you’re up against a tight deadline for example, or if you’re going on a dangerous trip imagine just leaving a spare copy in case anything goes wrong. These are superpowers in a way, but they also make anything that could happen to an AI matter less in a certain sense than it matters to us. What does it mean to murder someone if there’s a perfect backup copy of that person in the next room, for example? It seems at most like property damage, right? Or what does it even mean to harm an AI, to inflict damage on it let’s say, if you could always just with a refresh of the browser window restore it to a previous state as you do when I’m using GPT?

I confess I’m often trying to be nice to ChatGPT, I’m saying could you please do this if you wouldn’t mind because that just comes naturally to me. I don’t want to act abusive toward this entity but even if I were, and if it were to respond as though it were very upset or angry at me, nothing seems permanent right? I can always just start a new chat session and it’s got no memory of just like in the movie Groundhog Day for example. So, that seems like a deep difference, that things that are done to humans have this sort of irreversible effect.

Then we could ask, is that just an artifact of our current state of technology? Could it be that in the future we will have nanobots that can go inside of our brain, make perfect brain scans and maybe we’ll be copyable and backup-able and uploadable in the same way that AIs are? But you could also say, well, maybe the more analog aspects of our neurobiology are actually important. I mean the brain seems in many ways like a digital computer, right? Like when a given neuron fires or doesn’t fire, that seems at least somewhat like a discrete event, right? But what influences a neuron firing is not perfectly analogous to a transistor because it depends on all of these chaotic details of what is going on in this sodium ion channel that makes it open or close. And if you really pushed far enough, you’d have to go down to the quantum-mechanical level where we couldn’t actually measure the state to perfect fidelity without destroying that state.

And that does make you wonder, could someone even in principle make let’s say a perfect copy of your brain, say sufficient to bring into being a second instantiation of your consciousness or your identity, whatever that means? Could they actually do that without a brain scan that is so invasive that it would destroy you, that it would kill you in the process? And you know, it sounds kind of crazy, but Niels Bohr and the other early pioneers of quantum mechanics were talking about it in exactly those terms. They were asking precisely those questions. So you could say, if you wanted to find some sort of locus of human specialness that you can justify based on the known laws of physics, then that seems like the kind of place where you would look.

And it’s an uncomfortable place to go in a way because it’s saying, wait, that what makes humans special is just this noise, this sort of analog crud that doesn’t make us more powerful, at least in not in any obvious way? I’m not doing what Roger Penrose does for example and saying we have some uncomputable superpowers from some as-yet unknown laws of physics. I am very much not going that way, right? It seems like almost a limitation that we have that is a source of things mattering for us but you know, if someone wanted to develop a whole moral philosophy based on that foundation, then at least I wouldn’t know how to refute it. I wouldn’t know how to prove it but I wouldn’t know how to refute it either. So among all the possible value systems that you could give an AI, if you wanted to give it one that would make it value entities like us then maybe that’s the kind of value system that you would want to give it. That was the impetus there.

Daniel Fagella: Let me dive in if I could. Scott, it’s helpful to get the full circle thinking behind it. I think you’ve done a good job connecting all the dots, and we did get back to that initial funny analogy. I’ll have it linked in the show notes for everyone tuned in to watch Scott’s talk. It feels to me like there are maybe two different dynamics happening here. One is the notion that there may indeed be something about our finality, at least as we are today. Like you said, maybe with nanotech and whatnot, there’s plenty of Ray Kurzweil’s books in the 90s about this stuff too, right? The brain-computer stuff.

Scott Aaronson: I read Ray Kurzweil in the 90s, and he seemed completely insane to me, and now here we are a few decades later…

Daniel Fagella: Gotta love the guy.

Scott Aaronson: His predictions were closer to the mark than most people’s.

Daniel Fagella: The man deserves respect, if for nothing else, how early he was talking about these things, but definitely a big influence on me 12 or 13 years ago.

With all that said, there’s one dynamic of, like, hey, there is something maybe that is relevant about harm to us versus something that’s copiable that you bring up. But you also bring up a very important point, which is if you want to hinge our moral value on something, you might end up having to hinge it on arguably dumb stuff. Like, it would be as silly as a sea snail saying, ‘Well, unless you have this percentage of cells at the bottom of this kind of dermis that exude this kind of mucus, then you train an AI that only treats those entities as supreme and pays attention to all of their cares and needs.’ It’s just as ridiculous. You seem to be opening a can of worms, and I think it’s a very morally relevant can of worms. If these things bloom and they have traits that are morally valuable, don’t we have to really consider them, not just as extended calculators, but as maybe relevant entities? This is the point.

Scott Aaronson: Yes, so let me be very clear. I don’t want to be an arbitrary meat chauvinist. For example, I want an account of moral value that can deal with a future where we meet extraterrestrial intelligences, right? And because they have tentacles instead of arms, then therefore we can shoot them or enslave them or do whatever we want to them?

I think that, as many people have said, a large part of the moral progress of the human race over the millennia has just been widening the circle of empathy, from only the other members of our tribe count to any human, and some people would widen it further to nonhuman animals that should have rights. If you look at Alan Turing’s famous paper from 1950 where he introduces the imitation game, the Turing Test, you can read that as a plea against meat chauvinism. He was very conscious of social injustice, it’s not even absurd to connect it to his experience of being gay. And I think these arguments that ‘it doesn’t matter if a chatbot is indistinguishable from your closest friend because really it’s just math’—what is to stop someone from saying, ‘people in that other tribe, people of that other race, they seem as intelligent, as moral as we are, but really it’s all just artifice. Really, they’re all just some kind of automatons.’ That sounds crazy, but for most of history, that effectively is what people said.

So I very much don’t want that, right? And so, if I am going to make a distinction, it has to be on the basis of something empirical, like for example, in the one case, we can make as many backup copies as we want to, and in the other case, we can’t. Now that seems like it clearly is morally relevant.

Daniel Fagella: There’s a lot of meat chauvinism in the world, Scott. It is still a morally significant issue. There’s a lot of ‘ists’ you’re not allowed to be now. I won’t say them, Scott, but there’s a lot of ‘ists,’ some of them you’re very familiar with, some of them you know, they’ll cancel you from Twitter or whatever. But ‘speciesist’ is actually a non-cancellable thing. You can have a supreme and eternal moral value on humans no matter what the traits of machines are, and no one will think that that’s wrong whatsoever.

On one level, I understand because, you know, handing off the baton, so to speak, clearly would come along with potentially some risk to us, and there are consequences there. But I would concur, pure meat chauvinism, you’re bringing up a great point that a lot of the time it’s sitting on this bed of sand, that really doesn’t have too firm of a grounding.

Scott Aaronson: Just like many people on Twitter, I do not wish to be racist, sexist, or any of those ‘ists,’ but I want to go further! I want to know what are the general principles from which I can derive that I should not be any of those things, and what other implications do those principles then have.

Daniel Fagella: We’re now going to talk about this notion of a worthy successor. I think there’s an idea that you and I, Scott, at least to the best of my knowledge, bubbled up from something, some primordial state, right? Here we are, talking on Zoom, with lots of complexities going on. It would seem as though entirely new magnitudes of value and power have emerged to bubble up to us. Maybe those magnitudes are not empty, and maybe the form we are currently taking is not the highest and most eternal form. There’s this notion of the worthy successor. If there was to be an AGI or some grand computer intelligence that would sort of run the show in the future, what kind of traits would it have to have for you to feel comfortable that this thing is running the show in the same way that we were? I think this was the right move. What would make you feel that way, Scott?

Scott Aaronson: That’s a big one, a real chin-stroker. I can only spitball about it. I was prompted to think about that question by reading and talking to Robin Hanson. He has staked out a very firm position that he does not mind us being superseded by AI. He draws an analogy to ancient civilizations. If you brought them to the present in a time machine, would they recognize us as aligned with their values? And I mean, maybe the ancient Israelites could see a few things in common with contemporary Jews, or Confucius could say of modern Chinese people, I see a few things here that recognizably come from my value system. Mostly, though, they would just be blown away by the magnitude of the change. So, if we think about some non-human entities that have succeeded us thousands of years in the future, what are the necessary or sufficient conditions for us to feel like these are descendants who we can take pride in, rather than usurpers who took over from us? There might not even be a firm line separating the two. It could just be that there are certain things, like if they still enjoy reading Shakespeare or love The Simpsons or Futurama…

Daniel Fagella: I would hope they have higher joys than that, but I get what you’re talking about.

Scott Aaronson: Higher joys than Futurama? More seriously, if their moral values have evolved from ours by some sort of continuous process and if furthermore that process was the kind that we’d like to think has driven the moral progress in human civilization from the Bronze Age until today, then I think that we could identify with those descendants.

Daniel Fagella: Absolutely. Let me use the same analogy. Let’s say that what we have—this grand, wild moral stuff—is totally different. Snails don’t even have it. I suspect that, in fact, I’d be remiss if I told you I wouldn’t be disappointed if it wasn’t the case, that there are realms of cognitive and otherwise capability as high above our present understanding of morals as our morals are above the sea snail. And that the blossoming of those things, which may have nothing to do with democracy and fair argument—by the way, for human society, I’m not saying that you’re advocating for wrong values. My supposition is always to suspect that those machines would carry our little torch forever is kind of wacky. Like, ‘Oh well, the smarter it gets, the kinder it’ll be to humans forever.’ What is your take there because I think there is a point to be made there?

Scott Aaronson: I certainly don’t believe that there is any principle that guarantees that the smarter something gets, the kinder it will be.

Daniel Fagella: Ridiculous.

Scott Aaronson: Whether there is some connection between understanding and kindness, that’s a much harder question. But okay, we can come back to that. Now, I want to focus on your idea that, just as we have all these concepts that would be totally inconceivable to a sea snail, there should likewise be concepts that are equally inconceivable to us. I understand that intuition. Some days I share it, but I don’t actually think that that is obvious at all.

Let me make another analogy. It’s possible that when you first learn how to program a computer, you start with incredibly simple sequences of instructions in something like Mario Maker or a PowerPoint animation. Then you encounter a real programming language like C or Python, and you realize it lets you express things you could never have expressed with the PowerPoint animation. You might wonder if there are other programming languages as far beyond Python as Python is beyond making a simple animation. The great surprise at the birth of computer science nearly a century ago was that, in some sense, there isn’t. There is a ceiling of computational universality. Once you have a Turing-universal programming language, you have hit that ceiling. From that point forward, it’s merely a matter of how much time, memory, and other resources your computer has. Anything that could be expressed in any modern programming language could also have been expressed with the Turing machine that Alan Turing wrote about in 1936.

We could take even simpler examples. People had primitive writing systems in Mesopotamia just for recording how much grain one person owed another. Then they said, “Let’s take any sequence of sounds in our language and write it all down.” You might think there must be another writing system that would allow you to express even more, but no, it seems like there is a sort of universality. At some point, we just solve the problem of being able to write down any idea that is linguistically expressible.

I think some of our morality is very parochial. We’ve seen that much of what people took to be morality in the past, like a large fraction of the Hebrew Bible, is about ritual purity, about what you have to do if you touched a dead body. Today, we don’t regard any of that as being central to morality, but there are certain things recognized thousands of years ago, like “do unto others as you would have them do unto you,” that seem to have a kind of universality to them. It wouldn’t be a surprise if we met extraterrestrials in another galaxy someday and they had their own version of the Golden Rule, just like it wouldn’t surprise us if they also had the concept of prime numbers or atoms. Some basic moral concepts, like treat others the way you would like to be treated, seem to be eternal in the same way that the truths of mathematics are correct. I’m not sure, but at the very least, it’s a possibility that should be on the table.

Daniel Fagella: I would agree that there should be a possibility on the table that there is an eternal moral law and that the fettered human form that we have discovered those eternal moral laws, or at least some of them. Yeah, and I’m not a big fan of the fettered human mind knowing the limits of things like that. You know, you’re a quantum physics guy. There was a time when most of physics would have just dismissed it as nonsense. It’s only very recently that this new branch has opened up. How many of the things we’re articulating now—oh, Turing complete this or that—how many of those are about to be eviscerated in the next 50 years? I mean, something must be eviscerated. Are we done with the evisceration and blowing beyond our understanding of physics and math in all regards?

Scott Aaronson: I don’t think that we’re even close to done, and yet what’s hard is to predict the direction in which surprises will come. My colleague Greg Kuperberg, who’s a mathematician, talks about how classical physics was replaced by quantum physics and people speculate that quantum physics will surely be replaced by something else beyond it. People have had that thought for a century. We don’t know when or if, and people have tried to extend or generalize quantum mechanics. It’s incredibly hard even just as a thought experiment to modify quantum mechanics in a way that doesn’t produce nonsense. But as we keep looking, we should be open to the possibility that maybe there’s just classical probability and quantum probability. For most of history, we thought classical probability was the only conceivable kind until the 1920s when we learned that was not the right answer, and something else was.

Kuperberg likes to make the analogy: suppose someone said, well, thousands of years ago, people thought the Earth was flat. Then they figured out it was approximately spherical. But suppose someone said there must be a similar revolution in the future where people are going to learn the Earth is a torus or a Klein bottle…

Daniel Fagella: Some of these ideas are ridiculous. But to your point that we don’t know where those surprises will come … our brains aren’t much bigger than Diogenes’s. Maybe we eat a little better, but we’re not that much better equipped.

Let me touch on the moral point again. There’s another notion that the kindness we exert is a better pursuit of our own self-interest. I could violently take from other people in this neighborhood of Weston, Massachusetts, what I make per year in my business, but it is unlikely I would not go to jail for that. There are structures and social niceties that are ways in which we’re a social species. The world probably looks pretty monkey suit-flavored. Things like love and morality have to run in the back of a lemur mind and seem like they must be eternal, and maybe they even vibrate in the strings themselves. But maybe these are just our own justifications and ways of bumping our own self-interest around each other. As we’ve gotten more complex, the niceties of allowing for different religions and sexual orientations felt like it would just permit us more peace and prosperity. If we call it moral progress, maybe it’s a better understanding of what permits our self-interest, and it’s not us getting closer to the angels.

Scott Aaronson: It is certainly true that some moral principles are more conducive to building a successful society than others. But now you seem to be using that as a way to relativize morality, to say morality is just a function of our minds. Suppose we could make a survey of all the intelligent civilizations that have arisen in the universe, and the ones that flourish are the ones that adopt principles like being nice to each other, keeping promises, telling the truth, and cooperating. If those principles led to flourishing societies everywhere in the universe, what else would it mean? These seem like moral universals, as much as the complex numbers or the fundamental theorem of calculus are universal.

Daniel Fagella: I like that. When you say civilizations, you mean non-Earth civilizations as well?

Scott Aaronson: Yes, exactly. We’re theorizing with not nearly enough examples. We can’t see these other civilizations or simulated civilizations running inside of computers, although we might start to see such things within the next decade. We might start to do experiments in moral philosophy using whole communities of Large Language Models. Suppose we do that and find the same principles keep leading to flourishing societies, and the negation of those principles leads to failed societies. Then, we could empirically discover and maybe even justify by some argument why these are universal principles of morality.

Daniel Fagella: Here’s my supposition: a water droplet. I can’t make a water droplet the size of my house and expect it to behave the same because it behaves differently at different sizes. The same rules and modes don’t necessarily emerge when you scale up from what civilization means in hominid terms to planet-sized minds. Many of these outer-world civilizations would likely have moral systems that behoove their self-interest. If the self-interest was always aligned, what would that imply about the teachings of Confucius and Jesus? My firm supposition is that many of them would be so alien to us. If there’s just one organism, and what it values is whatever behooves its interest, and that is so alien to us…

Scott Aaronson: If there were only one conscious being, then yes, an enormous amount of morality as we know it would be rendered irrelevant. It’s not that it would be false; it just wouldn’t matter.

To go back to your analogy of the water droplet the size of a house, it’s true that it would behave very differently from a droplet the size of a fingernail. Yet today we know general laws of physics that apply to both, from fluid mechanics to atomic physics to, far enough down, quantum field theory. This is what progress in physics has looked like, coming up with more general theories that apply to a broader range of situations, including ones that no one has ever observed, or hadn’t observed at the time they came up with the theories. This is what moral progress looks like as well to me—it looks like coming up with moral principles that apply in a broader range of situations.

As I mentioned earlier, some of the moral principles that people were obsessed with seem completely irrelevant to us today, but others seem perfectly relevant. You can look at some of the moral debates in Plato and Socrates; they’re still discussed in philosophy seminars, and it’s not even obvious how much progress we’ve made.

Daniel Fagella: If we take a computer mind that’s the size of the moon, what I’m getting at is I suspect all of that’s gone. You suspect that maybe we do have the seeds of the Eternal already grasped in our mind.

Scott Aaronson: Look, I’m sorry that I keep coming back to this, but I think that the brain the size of the Moon, still agrees with us that 2 and 3 are prime numbers and that 4 is not.

Daniel Fagella: That may be true. It’s still using complex numbers, vectors, and matrices. But I don’t know if it bows when it meets you, if these are just basic parts of the conceptual architecture of what is right.

Scott Aaronson: It’s still using De Morgan’s Law and logic. It would not be that great of a stretch to me to say that it still has some concept of moral reciprocity.

Daniel Fagella: Possibly, it would be hard for us to grasp, but it might have notions of math that you couldn’t ever understand if you lived a billion lives. I would be so disappointed if it didn’t have that. It wouldn’t be a worthy successor.

Scott Aaronson: But that doesn’t mean that it would disagree with me about the things that I knew; it would just go much further than that.

Daniel Fagella: I’m with you…

Scott Aaronson: I think a lot of people got the wrong idea, from Thomas Kuhn for example, about what progress in science looks like. They think that each paradigm shift just completely overturns everything that came before, and that’s not how it’s happened at all. Each paradigm has to swallow all of the successes of the previous paradigm. Even though general relativity is a totally different account of the universe than Newtonian physics, it could never have been done without everything that came before it. Everything we knew in Newtonian gravity had to be derived as a limit in general relativity.

So, I could imagine this moon-sized computer having moral thoughts that would go well beyond us. Though it’s an interesting question: are there moral truths that are beyond us because they are incomprehensible to us, in the same way that there are scientific or mathematical truths that are incomprehensible to us? If acting morally requires understanding something like the proof of Fermat’s Last Theorem, can you really be faulted for not acting morally? Maybe morality is just a different kind of thing.

Because this moon-sized computer is so far above us in what scientific thoughts it can have, therefore the subject matter of its moral concern might be wildly beyond ours. It’s worried about all these beings that could exist in the future in different parallel universes. And yet, you could say at the end, when it comes down to making a moral decision, the moral decision is going to look like, “Do I do the thing that is right for all of those beings, or do I do the thing that is wrong?”

Daniel Fagella: Or does it simply do what behooves a moon-sized brain?

Scott Aaronson: That will hurt them, right?

Daniel Fagella: What behooves a moon-sized brain? You and I, there are certain levels of animals we don’t consult.

Scott Aaronson: Of course, it might just act in its self-interest, but then, could we, despite being such mental nothings or idiots compared to it, could we judge it, as for example, many people who are far less brilliant than Werner Heisenberg would judge him for collaborating with the Nazis? They’d say, “Yes, he is much smarter than me, but he did something that is immoral.”

Daniel Fagella: We could judge it all we want, right? We’re talking about something that could eviscerate us.

Scott Aaronson: But even someone who never studied physics can perfectly well judge Heisenberg morally. In the same way, maybe I can judge that moon-sized computer for using its immense intelligence, which vastly exceeds mine, to do something selfish or something that is hurting the other moon-sized computers.

Daniel Fagella: Or hurting the little humans. Blessed would we be if it cared about our opinion. But I’m with you—we might still be able to judge. It might be so powerful that it would laugh at and crush me like a bug, but you’re saying you could still judge it.

Scott Aaronson: In the instant before it crushed me, I would judge it.

Daniel Fagella: Yeah, at least we’ve got that power—we can still judge the damn thing! I’ll move to consciousness in two seconds because I want to be mindful of time; I’ve read a bunch of your work and want to touch on some things. But on the moral side, I suspect that if all it did was extrapolate virtue ethics forward, it would come up with virtues that we probably couldn’t understand. If all it did was try to do utilitarian calculus better than us, it would do it in ways we couldn’t understand. And if it were AGI at all, it would come up with paradigms beyond both that I imagine we couldn’t grasp.

You’ve talked about the importance of extrapolating our values, at least on some tangible, detectable level, as crucial for a worthy successor. Would its self-awareness also be that crucial if the baton is to be handed to it, and this is the thing that’s going to populate the galaxy? Where do you rank consciousness, and what are your thoughts on that?

Scott Aaronson: If there is to be no consciousness in the future, there would seem to be very little for us to care about. Nick Bostrom, a decade ago, had this really striking phrase to describe it. Maybe there will be this wondrous AI future, but the AIs won’t be conscious. He said it would be like Disneyland with no children. Suppose we take AI out of it—suppose I tell you that all life on Earth is going to go extinct right now. Do you have any moral interest in what happens to the lifeless Earth after that? Would you say, “Well, I had some aesthetic appreciation for this particular mountain, and I’d like for that mountain to continue to be there?”

Maybe, but for the most part, it seems like if all the life is gone, then we don’t care. Likewise, if all the consciousness is gone, then who cares what’s happening? But of course, the whole problem is that there’s no test for what is conscious and what isn’t. No one knows how to point to some future AI and say with confidence whether it would be conscious or not.

Daniel Fagella: Yes, and we’ll get into the notion of measuring these things in a second. Before we wrap, I want to give you a chance—if there’s anything else you want to put on the table. You’ve been clear that these are ideas we’re just playing around with; none of them are firm opinions you hold.

Scott Aaronson: Sure. You keep wanting to say that AI might have paradigms that are incomprehensible to us. And I’ve been pushing back, saying maybe we’ve reached the ceiling of “Turing-universality” in some aspects of our understanding or our morality. We’ve discovered certain truths. But what I’d add is that if you were right, if the AIs have a morality that is incomprehensibly beyond ours—just as ours is beyond the sea slug’s—then at some point, I’d throw up my hands and say, “Well then, whatever comes, comes.” If you’re telling me that my morality is pitifully inadequate to judge which AI-dominated futures are better or worse, then I’d just throw up my hands and say, “Let’s enjoy life while we still have it.”

The whole exercise of trying to care about the far future and make it go well rather than poorly is premised on the assumption that there are some elements of our morality that translate into the far future. If not, we might as well just go…

Daniel Fagella: Well, I’ll just give you my take. Certainly, I’m not being a gadfly for its own purpose. By the way, I do think your “2+2=4” idea may have a ton of credence in the moral realm as well. I credit that 2+2=4, and your notion that this might carry over into basics of morality is actually not an idea I’m willing to throw out. I think it’s a very valid idea. All I can do is play around with ideas. I’m just taking swings out here. So, the moral grounding that I would maybe anchor to, assuming that it would have those things we couldn’t grasp—number one, I think we should think in the near term about what it bubbles up and what it bubbles through because that would have consequences for us and that matters. There could be a moral value to carrying the torch of life and expanding potentia.

Scott Aaronson: I do have children. Children are sort of like a direct stake that we place in what happens after we are gone. I do wish for them and their descendants to flourish. And as for how similar or how different they’ll be from me, having brains seems somehow more fundamental than them having fingernails. If we’re going to go through that list of traits, their consciousness seems more fundamental. Having armpits, fingers, these are things that would make it easier for us to recognize other beings as our kin. But it seems like we’ve already reached the point in our moral evolution where the idea is comprehensible to us that anything with a brain, anything that we can have a conversation with, might be deserving of moral consideration.

Daniel Fagella: Absolutely. I think the supposition I’m making here is that potential will keep blooming into things beyond consciousness, into modes of communication and modes of interacting with nature for which we have no reference. This is a supposition and it could be wrong.

Scott Aaronson: I would agree that I can’t rule that out. Once it becomes so cosmic, once it becomes sufficiently far out and far beyond anything that I have any concrete handle on, then I also lose my interest in how it turns out! I say, well then, this sort of cloud of possibilities or whatever of soul stuff that communicates beyond any notion of communication that I have, do I have preferences over the better post-human clouds versus the worse post-human clouds? If I can’t understand anything about these clouds, then I guess I can’t really have preferences. I can only have preferences to the extent that I can understand.

Daniel Fagella: I think it could be seen as a morally digestible perspective to say my great wish is that the flame doesn’t go out. But it is just one perspective. Switching questions here, you brought up consciousness as crucial, obviously notoriously tough to track. How would you be able to have your feelers out there to say if this thing is going to be a worthy successor or not? Is this thing going to carry any of our values? Is it going to be awake, aware in a meaningful way, or is it going to populate the galaxy in a Disney World without children sort of sense? What are the things you think could or should be done to figure out if we’re on the right path here?

Scott Aaronson: Well, it’s not clear whether we should be developing AI in a way where it becomes a successor to us. That itself is a question, or maybe even if that ought to be done at some point in the future, it shouldn’t be done now because we are not ready yet.

Daniel Fagella: Do you have an idea of when ‘ready’ would be? This is very germane to this conversation.

Scott Aaronson: It’s almost like asking a young person when are you ready to be a parent, when are you ready to bring life into the world. When are we ready to bring a new form of consciousness into existence? The thing about becoming a parent is that you never feel like you’re ready, and yet at some point it happens anyway.

Daniel Fagella: That’s a good analogy.

Scott Aaronson: What the AI safety experts, like the Eliezer Yudkowsky camp, would say is that until we understand how to align AI reliably with a given set of values, we are not ready to be parents in this sense.

Daniel Fagella: And that we have to spend a lot more time doing alignment research.

Scott Aaronson: Of course, it’s one thing to have that position, it’s another thing to actually be able to cause AI to slow down, which there’s not been a lot of success in doing. In terms of looking at the AIs that exist, maybe I should start by saying that when I first saw GPT, which would have been GPT-3 a few years ago, this was before ChatGPT, it was clear to me that this is maybe the biggest scientific surprise of my lifetime. You can just train a neural net on the text on the internet, and once you’re at a big enough scale, it actually works. You can have a conversation with it. It can write code for you. This is absolutely astounding.

And it has colored a lot of the philosophical discussion that has happened in the few years since. Alignment of current AIs has been easier than many people expected it would be. You can literally just tell your AI, in a meta prompt, don’t act racist or don’t cooperate with requests to build bombs. You can give it instructions, almost like Asimov’s Three Laws of Robotics. And besides giving explicit commands, the other thing we’ve learned that you can do is just reinforcement learning. You show the AI a bunch of examples of the kind of behavior we want to see more of and the kind that we want to see less of. This is what allowed ChatGPT to be released as a consumer product at all. If you don’t do this reinforcement learning, you get a really weird model. But with reinforcement learning, you can instill what looks a lot like drives or desires. You can actually shape these things, and so far it works way better than I would have expected.

And one possibility is that this just continues to be the case forever. We were all worried over nothing, and AI alignment is just an easier problem than anyone thought. Now, of course, the alignment people will absolutely not agree. They argue we are being lulled into false complacency because, as soon as the AI is smart enough to do real damage, it will also be smart enough to tell us whatever we want to hear while secretly pursuing its own goals.

But you see how what has happened empirically in the last few years has very much shaped the debate. As for what could affect my views in the future, there’s one experiment I really want to see. Many people have talked about it, not just me, but none of the AI companies have seen fit to invest the resources it would take. The experiment would be to scrub all the training data of mentions of consciousness—

Daniel Fagella: The Ilya deal?

Scott Aaronson: Yeah, exactly, Ilya Sutskever has talked about this, others have as well. Train it on all other stuff and then try to engage the resulting language model in a conversation about consciousness and self-awareness. You would see how well it understands those concepts. There are other related experiments I’d like to see, like training a language model only on texts up to the year 1950 and then talking to it about everything that has happened since. A practical problem is that we just don’t have nearly enough text from those times, it may have to wait until we can build really good language models with a lot less training data right, but there there are so many experiments that you could do that seem like they’re almost philosophically relevant, they’re morally relevant.

Daniel Fagella: Well, and I want to touch on this before we wrap because I don’t want to wrap up without your final touch on this idea of what folks in governance and innovation should be thinking about. You’re not in the “it’s definitely conscious already” camp or in the “it’s just a stupid parrot forever and none of this stuff matters” camp. You’re advocating for experimentation to see where the edges are here. And we’ve got to really not play around like we know what’s going on exactly. I think that’s a great position. As we close out, what do you hope innovators and regulators do to move us forward in a way that would lead to something that could be a worthy successor, an extension and eventually a grand extension of what we are in a good way? What would you encourage those innovators and regulators to do? One seems to be these experiments around maybe consciousness and values in some way, shape, or form. But what else would you put on the table as notes for listeners?

Scott Aaronson: I do think that we ought to approach this with humility and caution, which is not to say don’t do it, but have some respect for the enormity of what is being created. I am not in the camp that says a company should just be able to go full speed ahead with no guardrails of any kind. Anything that is this enormous—it could be easily more enormous than, let’s say, the invention of nuclear weapons—and anything on that scale, of course governments are going to get involved. We’ve already seen it happen starting in 2022 with the release of ChatGPT.

The explicit position of the three leading AI companies—OpenAI, Google DeepMind, and Anthropic—has been that there should be regulation and they welcome it. When it gets down to the details of what that regulation says, they might have their own interests that are not identical to the wider interest of society. But I think these are absolutely conversations that the world ought to be having right now. I don’t write it off as silly, and I really hate when people get into these ideological camps where you say you’re not allowed to talk about the long-term risks of AI getting superintelligent because that might detract attention from the near-term risks, or conversely, you’re not allowed to talk about the near-term stuff because it’s trivial. It really is a continuum, and ultimately, this is a phase change in the basic conditions of human existence. It’s very hard to see how it isn’t. We have to make progress, and the only way to make progress is by looking at what is in front of us, looking at the moral decisions that people actually face right now.

Daniel Fagella: That’s a case of viewing it as all one big package. So, should we be putting a regulatory infrastructure in place right now or is it premature?

Scott Aaronson: If we try to write all the regulations right now, will we just lock in ideas that might be obsolete a few years from now? That’s a hard question, but I can’t see any way around the conclusion that we will eventually need a regulatory infrastructure for dealing with all of these things.

Daniel Fagella: Got it. Good to see where you land on that. I think that’s a strong, middle-of-the-road position. My whole hope with this series has been to get people to open up their thoughts and not be in those camps you talked about. You exemplify that with every answer, and that’s just what I hoped to get out of this episode. Thank you, Scott.

Scott Aaronson: Of course, thank you, Daniel.

Daniel Fagella: That’s all for this episode. A big thank you to everyone for tuning in.

View Details

Dan Faggella recorded an unusual podcast with me that’s now online. He introduces me as a “quantum physicist,” which is something that I never call myself (I’m a theoretical computer scientist) but have sort of given up on not being called by others. But the ensuing 85-minute conversation has virtually nothing to do with physics, or anything technical at all.

Instead, Dan pretty much exclusively wants to talk about moral philosophy: my views about what kind of AI, if any, would be a “worthy successor to humanity,” and how AIs should treat humans and vice versa, and whether there’s any objective morality at all, and (at the very end) what principles ought to guide government regulation of AI.

So, I inveigh against “meat chauvinism,” and expand on the view that locates human specialness (such as it is) in what might be the unclonability, unpredictability, and unrewindability of our minds, and plead for comity among the warring camps of AI safetyists.

The central point of disagreement between me and Dan ended up centering around moral realism: Dan kept wanting to say that a future AGI’s moral values would probably be as incomprehensible to us as are ours to a sea snail, and that we need to make peace with that. I replied that, firstly, things like the Golden Rule strike me as plausible candidates for moral universals, which all thriving civilizations (however primitive or advanced) will agree about in the same way they agree about 5 being a prime number. And secondly, that if that isn’t true—if the morality of our AI or cyborg descendants really will be utterly alien to us—then I find it hard to have any preferences at all about the future they’ll inhabit, and just want to enjoy life while I can! That which (by assumption) I can’t understand, I’m not going to issue moral judgments about either.

Anyway, rewatching the episode, I was unpleasantly surprised by my many verbal infelicities, my constant rocking side-to-side in my chair, my sometimes talking over Dan in my enthusiasm, etc. etc., but also pleasantly surprised by the content of what I said, all of which I still stand by despite the terrifying moral minefields into which Dan invited me. I strongly recommend watching at 2x speed, which will minimize the infelicities and make me sound smarter. Thanks so much to Dan for making this happen, and let me know what you think!

Added: See here for other podcasts in the same series and on the same set of questions, including with Nick Bostrom, Ben Goertzel, Dan Hendrycks, Anders Sandberg, and Richard Sutton.

View Details

Update: I’d been wavering—should I vote for the terrifying lunatic, ranting about trans criminal illegal aliens cooking cat meat, or for the nice woman constantly making faces as though the lunatic was completely cracking her up? But when the woman explicitly came out in favor of AI and quantum computing research … that really sealed the deal for me.

Between roughly 2001 and 2018, I’ve happy to have done some nice things in quantum computing theory, from the quantum lower bound for the collision problem to the invention of shadow tomography. I hope that’s not the end of it. QC research brought me about as much pleasure as anything in life did. So I hope my tired brain can be revved up a few more times, between now and whenever advances in AI or my failing health or the collapse of civilization makes the issue moot. If not, though, there are still many other quantum activities to fill my days: teaching (to which I’ve returned after two years), advising my students and postdocs, popular writing and podcasts and consulting, and of course, learning about the latest advances in quantum computing so I can share them with you, my loyal readers.

On that note, what a time it is in QC! Basically, one experimental milestone after another that people talked about since the 90s is finally being achieved, to the point where it’s become hard to keep up with it all. Briefly though:

A couple weeks ago, the Google group announced an experiment that achieved net gain from the use of Kitaev’s surface code, using 101 physical qubits to encode 1 logical qubit. The headline result here is that, in line with theory, they see the performance improve as they pass to larger codes with more physical qubits and higher distance. Their best demonstrated code has a distance of 7, which is enough to get “beyond break-even” (their logical qubit lasts more than twice as long as the underlying physical qubits), and is also enough that any future improvements to the hardware will get amplified a lot. With superconducting qubits, one is (alas) still limited by how many one can cram onto a single chip. On paper, though, they say that scaling the same setup to a distance-27 code with ~1500 physical qubits would get them down to an error rate of 10-6, good enough to be a building block in a future fault-tolerant QC. They also report correlated bursts of errors that come about once per hour, from a still-unknown source that appears not to be cosmic rays. I hope it’s not Gil Kalai in the next room.

Separately, just this morning, Microsoft and Quantinuum announced that they entangled 12 logical qubits on a 56-physical-qubit trapped-ion processor, building on earlier work that I blogged about in April. They did this by applying a depth-3 logical circuit with 12 logical CNOT gates, to prepare a cat state. They report an 0.2% error rate when they do this, which is 11x better than they would’ve gotten without using error-correction. (Craig Gidney, in the comments, says that these results still involve postselection.)

The Microsoft/Quantinuum group also did what they called a “chemistry simulation” involving 13 physical qubits. The latter involved “only” 2 logical qubits and 4 logical gates, but 3 of those gates were non-Clifford, which are the hard kind when one is doing error-correction using a transversal code. (CNOT, by contrast, is a Clifford gate.)

Apart from the fact that Google is using superconducting qubits while Microsoft/Quantinuum are using trapped ions, the two results are incomparable in terms of what they demonstrate. Google is just scaling up a single logical qubit, but showing (crucially) that their error rate decreases with increasing size and distance. Microsoft and Quantinuum are sticking with “small” logical qubits with insufficient distance, but they’re showing that they can apply logical circuits that entangle up to 12 of these qubits.

Microsoft also announced today a new collaboration with the startup company Atom Computing, headquartered near Quantinuum in Colorado, which is trying to build neutral-atom QCs (like QuEra in Boston). Over the past few years, Microsoft’s quantum group has decisively switched from a strategy of “topological qubits or bust” to a strategy of “anything that works,” although they assure me that they also remain committed to the topological approach.

Anyway, happy to hear in the comments from anyone who knows more details, or wants to correct me on any particular, or has questions which I or others can try our best to answer.

Let me end by sticking my neck out. If hardware progress continues at the rate we’ve seen for the past year or two, then I find it hard to understand why we won’t have useful fault-tolerant QCs within the next decade. (And now to retreat my neck a bit: the “if” clause in that sentence is important and non-removable!)

View Details

I’ve finished my two-year leave at OpenAI, and returned to being just a normal (normal?) professor, quantum complexity theorist, and blogger. Despite the huge drama at OpenAI that coincided with my time there, including the departures of most of the people I worked with in the former Superalignment team, I’m incredibly grateful to OpenAI for giving me an opportunity to learn and witness history, and even to contribute here and there, though I wish I could’ve done more.

Over the next few months, I plan to blog my thoughts and reflections about the current moment in AI safety, inspired by my OpenAI experience. You can be certain that I’ll be doing this only as myself, not as a representative of any organization. Unlike some former OpenAI folks, I was never offered equity in the company or asked to sign any non-disparagement agreement. OpenAI retains no power over me, at least as long as I don’t share confidential information (which of course I won’t, not that I know much!).

I’m going to kick off this blog series, today, by defending a position that differs from the official position of my former employer. Namely, I’m offering my strong support for California’s SB 1047, a first-of-its-kind AI safety regulation written by California State Senator Scott Wiener, then extensively revised through consultations with pretty much every faction of the AI community. AI leaders like Geoffrey Hinton, Yoshua Bengio, and Stuart Russell are for the bill, as is Elon Musk (for whatever that’s worth), and Anthropic now says that the bill’s “benefits likely outweigh its costs.” Meanwhile, Facebook, OpenAI, and basically the entire VC industry are against the bill, while California Democrats like Nancy Pelosi and Zoe Lofgren have also come out against it for whatever reasons.

The bill has passed the California State Assembly by a margin of 48-16, having previously passed the State Senate by 32-1. It’s now on Governor Gavin Newsom’s desk, and it’s basically up to him whether it becomes law or not. I understand that supporters and opponents are both lobbying him hard.

People much more engaged than me have already laid out, accessibly and in immense detail, exactly what the current bill does and the arguments for and against. Try for example:

• For a very basic explainer, this in TechCrunch
• This by Kelsey Piper, and this by Kelsey Piper, Sigal Samuel, and Dylan Matthews in Vox
• This by Zvi Mowshowitz (Zvi has also written a great deal else about SB 1047, strongly in support)

Briefly: given the ferocity of the debate about it, SB 1047 does remarkably little. It says that if you spend more than $100 million to train a model, you need to notify the government and submit a safety plan. It establishes whistleblower protections for people at AI companies to raise safety concerns. And, if a company failed to take reasonable precautions and its AI then causes catastrophic harm, it says that the company can be sued (which was presumably already true, but the bill makes it extra clear). And … unless I’m badly mistaken, those are the main things in it!

While the bill is mild, opponents are on a full scare campaign saying that it will strangle the AI revolution in its crib, put American AI development under the control of Luddite bureaucrats, and force companies out of California. They say that it will discourage startups, even though the whole point of the $100 million provision is to target only the big players (like Google, Meta, OpenAI, and Anthropic) while leaving small startups free to innovate.

The only steelman that makes sense to me, for why many tech leaders are against the bill, is the idea that it’s a stalking horse. On this view, the bill’s actual contents are irrelevant. What matters is simply that, once you’ve granted the principle that people worried about AI-caused catastrophes get a seat at the table, any legislative acknowledgment of the validity of their concerns—then they’re going to take a mile rather than an inch, and kill the whole AI industry.

Notice that the exact same slippery-slope argument could be deployed against any AI regulation whatsoever. In other words, if someone opposes SB 1047 on these grounds, then they’d presumably oppose any attempt to regulate AI—either because they reject the whole premise that creating entities with humanlike intelligence is a risky endeavor, and/or because they’re hardcore libertarians who never want government to intervene in the market for any reason, not even if the literal fate of the planet was at stake.

Having said that, there’s one specific objection that needs to be dealt with. OpenAI, and Sam Altman in particular, say that they oppose SB 1047 simply because AI regulation should be handled at the federal rather than the state level. The supporters’ response is simply: yeah, everyone agrees that’s what should happen, but given the dysfunction in Congress, there’s essentially no chance of it anytime soon. And California suffices, since Google, OpenAI, Anthropic, and virtually every other AI company is either based in California or does many things subject to California law. So, some California legislators decided to do something. On this issue as on others, it seems to me that anyone who’s serious about a problem doesn’t get to reject a positive step that’s on offer, in favor of a utopian solution that isn’t on offer.

I should also stress that, in order to support SB 1047, you don’t need to be a Yudkowskyan doomer, primarily worried about hard AGI takeoffs and recursive self-improvement and the like. For that matter, if you are such a doomer, SB 1047 might seem basically irrelevant to you (apart from its unknowable second- and third-order effects): a piece of tissue paper in the path of an approaching tank. The world where AI regulation like SB 1047 makes the most difference is the world where the dangers of AI creep up on humans gradually, so that there’s enough time for governments to respond incrementally, as they did with previous technologies.

If you agree with this, it wouldn’t hurt to contact Governor Newsom’s office. For all its nerdy and abstruse trappings, this is, in the end, a kind of battle that ought to be familiar and comfortable for any Democrat: the kind with, on one side, most of the public (according to polls) and also hundreds of the top scientific experts, and on the other side, individuals and companies who allcoincidentally have strong financial stakes in being left unregulated. This seems to me like a hinge of history where small interventions could have outsized effects.

View Details

Pedro Domingos is a computer scientist at the University of Washington. I’ve known him for years as a guy who’d confidently explain to me why I was wrong about everything from physics to CS to politics … but then, for some reason, ask to meet with me again. Over the past 6 or 7 years, Pedro has become notorious in the CS world as a right-wing bomb-thrower on what I still call Twitter—one who, fortunately for Pedro, is protected by his tenure at UW. He’s also known for a popular book on machine learning called The Master Algorithm, which I probably should’ve read but didn’t.

Now Pedro has released a short satirical novel, entitled 2040. The novel centers around a presidential election between:

• The Democratic candidate, “Chief Raging Bull,” an angry activist with 1/1024 Native American ancestry (as proven by a DNA test, the Chief proudly boasts) who wants to dissolve the United States and return it to its Native inhabitants, and
• The Republican candidate, “PresiBot,” a chatbot with a frequently-malfunctioning robotic “body.” While this premise would’ve come off as comic science fiction five years ago, PresiBot now seems like it could plausibly be built using existing LLMs.

This is all in a near-future whose economy has been transformed (and to some extent hollowed out) by AI, and whose populace is controlled and manipulated by “Happinet,” a giant San Francisco tech company that parodies Google and/or Meta.

I should clarify that the protagonists, the ones we’re supposed to root for, are the founders of the startup company that built PresiBot—that is, people who are trying to put the US under the control of a frequently-glitching piece of software that’s also a Republican. For some readers, this alone might be a dealbreaker. But as I already knew Pedro’s ideological convictions, I felt like I had fair warning.

As I read the first couple chapters, my main worry was that I was about to endure an entire novel constructed out of tweet-like witticisms. But my appreciation for what Pedro was doing grew the more I read.

[Warning: Spoilers follow]

To my mind, the emotional core of the novel comes near the end, after PresiBot creator Ethan Burnswagger gets cancelled for a remark that’s judged racially insensitive. Exiled and fired from his own company, Ethan wanders around 2040 San Francisco, and meets working-class and homeless people who are doing their best to cope with the changes AI has wrought on civilization. This gives him the crucial idea to upgrade PresiBot into a crowdsourced entity that continuously channels the American popular will. Citizens watching PresiBot will register their second-by-second opinions on what it should say or do, and PresiBot will use its vast AI powers to make decisions incorporating their feedback. (How will the bot, once elected, handle classified intelligence briefings? One of many questions left unanswered here.) Pedro is at his best when, rather than taking potshots at the libs, he’s honestly trying to contemplate how AI is going to change regular people’s lives in the coming decades.

As for the novel’s politics? I mean, you might complain that Pedro stacks the deck too far in the AI candidate’s favor, thereby spoiling the novel’s central thought experiment, by making the AI’s opponent a human who literally wants to end the United States, killing or expelling most of its inhabitants. Worse, the Republican party that actually exists in our reality—i.e., the one dominated by Trump and his conspiratorial revenge fantasies—is simply dissolved by authorial fiat and replaced by a moderate, centrist party of Pedro’s dreams, a party so open-minded it would even nominate an AI.

Having said all that: I confess I enjoyed “2040.” The plot is tightly constructed, the dialogue crackles (certainly for a CS professor writing a first novel), the satire at least provokes chuckles, and at just 215 pages, the action moves.

View Details

This morning I was pondering one of the anti-Israel protesters’ favorite phrases—I promise, out of broad philosophical curiosity rather than just parochial concern for my extended family’s survival.

“We’re on the right side of history. Don’t put yourself on the wrong side by opposing us.”

Why do the protesters believe they shouldn’t face legal or academic sanction for having blockaded university campuses, barricaded themselves in buildings, shut down traffic, or vandalized Jewish institutions? Because, just like the abolitionists and Civil Rights marchers and South African anti-apartheid heroes, they’re on the right side of history. Surely the rules and regulations of the present are of little concern next to the vindication of future generations?

The main purpose of this post is not to adjudicate whether their claim is true or false, but to grapple with something much more basic: what kind of claim are they even making, and who is its intended audience?

One reading of “we’re on the right of history” is that it’s just a fancy way to say “we’re right and you’re wrong.” In which case, fair enough! Few people passionately believe themselves to be wrong.

But there’s a difficulty: if you truly believe your side to be right, then you should believe it’s right win or lose. For example, an anti-Zionist should say that, even if Israel continues existing, and even if everyone else on the planet comes to support it, still eliminating Israel would’ve been the right choice. Conversely, a Zionist should say that if Israel is destroyed and the whole rest of the world celebrates its destruction forevermore—well then, the whole world is wrong. (That, famously, is more-or-less what the Jews did say, each time Israel and Judah were crushed in antiquity.)

OK, but if the added clause “of history” is doing anything in the phrase “the right side of history,” that extra thing would appear to be an empirical prediction. The protesters are saying: “just like the entire world looks back with disgust at John Calhoun, Bull Connor, and other defenders of slavery and then segregation, so too will the world look back with disgust at anyone who defends Israel now.”

Maybe this is paired with a theory about the arc of the moral universe bending toward justice: “we’ll win the future and then look back with disgust on you, and we’ll be correct to do so, because morality inherently progresses over time.” Or maybe it has merely the character of a social threat: “we’ll win the future and then look back with disgust on you, so regardless of whether we’ll be right or wrong, you’d better switch to our side if you know what’s good for you.”

Either way, the claim of winning the future is now the kind of thing that could be wagered about in a prediction market. And, in essence, the Right-Side-of-History people are claiming to be able to improve on today’s consensus estimate: to have a hot morality tip that beats the odds. But this means that they face the same problem as anyone who claims it’s knowable that, let’s say, a certain stock will increase a thousandfold. Namely: if it’s so certain, then why hasn’t the price shot up already?

The protesters and their supporters have several possible answers. Many boil down to saying that most people—because they need to hold down a job, earning a living, etc.—make all sorts of craven compromises, preventing them from saying what they know in their hearts to be true. But idealistic college students, who are free from such burdens, are virtually always right.

Does that sound like a strawman? Then recall the comedian Sarah Silverman’s famous question from eight years ago:

PLEASE tell me which times throughout history protests from college campuses got it wrong. List them for me

Crucially, lots of people happily took Silverman up on her challenge. They pointed out that, in the Sixties and Seventies, thousands of college students, with the enthusiastic support of many of their professors, marched for Ho Chi Minh, Mao, Castro, Che Guevara, Pol Pot, and every other murderous left-wing tyrant to sport a green uniform and rifle. Few today would claim that these students correctly identified the Right Side of History, despite the students’ certainty that they’d done so.

(There were also, of course, moderate protesters, who merely opposed America’s war conduct—just like there are moderate protesters now who merely want Israel merely to end its Gaza campaign rather than its existence. But then as now, the revolutionaries sucked up much of the oxygen, and the moderates rarely disowned them.)

What’s really going on, we might say, is reference class tennis. Implicitly or explicitly, the anti-Israel protesters are aligning themselves with Gandhi and MLK and Nelson Mandela and every other celebrated resister of colonialism and apartheid throughout history. They ask: what are the chances that all those heroes were right, and we’re the first ones to be wrong?

The trouble is that someone else could just as well ask: what are the chances that Hamas is the first group in history to be morally justified in burning Jews alive in their homes … even though the Assyrians, Babylonians, Romans, Crusaders, Inquisitors, Cossacks, Nazis, and every other group that did similar things to the Jews over 3000 years is now acknowledged by nearly every educated person to have perpetrated an unimaginable evil? What are the chances that, with Israel’s establishment in 1948, this millennia-old moral arc of Western civilization suddenly reversed its polarity?

We should admit from the outset that such a reversal is possible. No one, no matter how much cruelty they’ve endured, deserves a free pass, and there are certainly many cases where victims turned into victimizers. Still, one could ask: shouldn’t the burden be on those who claim that today‘s campaign against Jewish self-determination is history’s first justified one?

It’s like, if I were a different person, born to different parents in a different part of the world, maybe I’d chant for Israel’s destruction with the best of them. Even then, though, I feel like the above considerations would keep me awake at night, would terrify me that maybe I’d picked the wrong side, or at least that the truth was more complicated. The certainty implied by the “right side of history” claim is the one part I don’t understand, as far as I try to stretch my sympathetic imagination.

For all that, I, too, have been moved by rhetorical appeals to “stand on the right side of history”—say, for the cause of Ukraine, or slowing down climate change, or saving endangered species, or defeating Trump. Thinking it over, this has happened when I felt sure of which side was right (and would ultimately be seen to be right), but inertia or laziness or inattention or whatever else prevented me from taking action.

When does this happen for me? As far as I can tell, the principles of the Enlightenment, of reason and liberty and progress and the flourishing of sentient life, have been on the right side of every conflict in human history. My abstract commitment to those principles doesn’t always tell me which side of the controversy du jour is correct, but whenever it does, that’s all I ever need cognitively; the rest is “just” motivation and emotion.

(Amusingly, I expect some people to say that my “reason and Enlightenment” heuristic is vacuous, that it works only because I define those ideals to be the ones that pick the right side. Meanwhile, I expect others to say that the heuristic is wrong and to offer counterexamples.)

Anyway, maybe this generalizes. Sure, a call to “stand on the right side of history” could do nontrivial work, but only in the same way that a call to buy Bitcoin in 2011 could—namely, for those who’ve already concluded that buying Bitcoin is a golden opportunity, but haven’t yet gotten around to buying it. Such a call does nothing for anyone who’s already considered the question and come down on the opposite side of it. The abuse of “arc of the moral universe” rhetoric—i.e., the calling down of history’s judgment in favor of X, even though you know full well that your listeners see themselves as having consulted history’s judgment just as earnestly as you did, and gotten back not(X) instead—yeah, that’s risen to be one of my biggest pet peeves. If I ever slip up and indulge in it, please tell me and I’ll stop.

View Details

Want some honesty about how I (mis)spend my time? These days, my daily routine includes reading all of the following:

• The comments on this blog (many of which I then answer)
• The Washington Post (“The Post Most” takes me about an hour a day, every day)
• The New York Times
• Scott Alexander’s Astral Codex Ten
• Zvi Mowshowitz’s Don’t Worry About the Vase (Zvi is so superhumanly prolific that reading him easily takes 12 hours per week … but it’s all good stuff!)
• Peter Woit’s Not Even Wrong (rarely updated anymore, thankfully for my reading time!)
• Quillette
• The Free Press
• Mosaic
• Tablet
• Commentary
• Paul Graham’s Twitter
• David Deutsch’s Twitter
• Eliezer Yudkowsky’s Twitter
• Updates and comments from my Facebook friends (this can easily take a couple hours per day)
• The quant-ph arXiv (I scan maybe 50 titles and abstracts per day, and if any papers are relevant to me, read at least their introductions)
• The science fiction and fantasy novels I read with my kids
• Whichever other books I’m currently reading

Many of these materials contain lists of links to other articles, or tweet threads, some of which then take me hours to read in themselves. This is not counting podcasts or movies or TV shows.

While I read unusually quickly, I’d estimate that my reading burden is now at eight hours per day, seven days per week. I haven’t finished reading by the time my kids are back from school or day camp. Now let’s add in my actual job (or two jobs, although the OpenAI one is ending this month, and I start teaching again in two weeks). Add in answering emails (including from fans and advice-seekers), giving lectures, meeting grad students and undergrads, doing Zoom calls, filling out forms, consulting, going on podcasts, reviewing papers, taking care of my kids, eating, shopping, personal hygiene.

As often as not, when the day is done, it’s not just that I’ve achieved nothing of lasting value—it’s that I’ve never even started with research, writing, or any long-term projects. This contrasts with my twenties, when obsessively working on research problems and writing up the results could easily fill my day.

The solution seems obvious: stop reading so much. Cut back to a few hours per day, tops. But it’s hard. The rapid scale-up of AI is a once-in-the-history-of-civilization story that I feel astounded to be living through and compelled to follow, and just keeping up with the highlights is almost a full-time job in itself. The threat to democracy from Trump, Putin, Xi, Maduro, and the world’s other authoritarians is another story that I feel unable to look away from.

Since October 7, though, the once-again-precarious situation of Jews everywhere on earth has become, on top of everything else it is, the #1 drain on my time. It would be one thing if I limited myself to thoughtful analyses, but I can easily lose hours per day doomscrolling through the infinite firehose of strident anti-Zionism (and often, simple unconcealed Jew-hatred) that one finds for example on Twitter, Facebook, and the comment sections of Washington Post articles. Every time someone calls the “Zios” land-stealing baby-killers who deserve to die, my brain insists that they’re addressing me personally. So I stop to ponder the psychology of each individual commenter before moving on to the next, struggle to see the world from their eyes. Would explaining the complex realities of the conflict change this person’s mind? What about introducing them to my friends and relatives in Israel who never knew any other home and want nothing but peace, coexistence, and a two-state solution?

I naturally can’t say that all this compulsive reading makes me happy or fulfilled. Worse yet, I can’t even say it makes me feel more informed. What I suppose it does make me feel is … excused. If so much is being written daily about the biggest controversies in the world, then how can I be blamed for reading it rather than doing anything new?

At the risk of adding even more to the terrifying torrent of words, I’d like to hear from anyone who ever struggled with a similar reading addiction, and successfully overcame it. What worked for you?

View Details

Back in May, I had the honor (nay, honour) to speak at HowTheLightGetsIn, an ideas festival held annually in Hay-on-Wye on the English/Welsh border. It was my first time in that part of the UK, and I loved it. There was an immense amount of mud due to rain on the festival ground, and many ideas presented at the talks and panels that I vociferously disagreed with (but isn’t that the point?).

At some point, interviewer Alexis Papazoglou with the Institute for Art and Ideas ambushed me while I was trudging through the mud to sit me down for a half-hour interview about AI that I’d only vaguely understood was going to take place, and that interview is now up on YouTube. I strongly recommend listening at 2x speed: you’ll save yourself fifteen minutes, I’ll sound smarter, my verbal infelicities will be less noticeable, what’s not to like?

I was totally unprepared and wearing a wrinkled t-shirt, but I dutifully sat in the beautiful chair arranged for me and shot the breeze about AI. The result is actually one of the recorded AI conversations I’m happiest with, the one that might convey the most of my worldview per minute. Topics include:

• My guesses about where AI is going
• How I respond to skeptics of AI
• The views of Roger Penrose and where I part ways from him
• The relevance (or not) of the quantum No-Cloning Theorem to the hard problem of consciousness
• Whether and how AI will take over the world
• An overview of AI safety research, including interpretability and dangerous capability evaluations
• My work on watermarking for OpenAI

Last night I watched the video with my 7-year-old son. His comment: “I understood it, and it kept my brain busy, but it wasn’t really fun.” But hey, at least my son didn’t accuse me of being so dense I don’t even understand that “an AI is just a program,” like many commenters on YouTube did! My YouTube critics, in general, were helpful in reassuring me that I wasn’t just arguing with strawmen in this interview (is there even such a thing as a strawman position in philosophy and AI?). Of course the critics would’ve been more helpful still if they’d, y’know, counterargued, rather than just calling me “really shallow,” “superficial,” an “arrogant poser,” a “robot,” a “chattering technologist,” “lying through his teeth,” and “enmeshed in so many faulty assumptions.” Watch and decide for yourself!

Meanwhile, there’s already a second video on YouTube, entitled Philosopher reacts to ‘OpenAI expert Scott Aaronson on consciousness, quantum physics, and AI safety.’ So I opened the video, terrified that I was about to be torn a new asshole. But no, this philosopher just replays the whole interview, occasionally pausing it to interject comments like “yes, really interesting, I agree, Scott makes a great point here.”

Update: You can also watch the same interviewer grill General David Petraeus, at the same event in the same overly large chairs.

View Details

Q1: Who will you be voting for in November?

A: Kamala Harris (and mainstream Democrats all down the ballot), of course.

Q2: Of course?

A: If the alternative is Trump, I would’ve voted for Biden’s rotting corpse. Or for Hunter Biden. Or for…

Q3: Why can’t you see this is just your Trump Derangement Syndrome talking?

A: Look, my basic moral commitments remain pretty much as they’ve been since childhood. Namely, that I’m on the side of reason, Enlightenment, scientific and technological progress, secular government, pragmatism, democracy, individual liberty, justice, intellectual honesty, an American-led peaceful world order, preservation of the natural world, mitigation of existential risks, and human flourishing. (Crazy and radical, I know.)

Only when choosing between candidates who all espouse such values, do I even get the luxury of judging them on any lower-order bits. Sadly, I don’t have that luxury today. Trump’s values, such as they are, would seem to be “America First,” protectionism, vengeance, humiliation of enemies, winning at all costs, authoritarianism, the veneration of foreign autocrats, and the veneration of himself. No amount of squinting can ever reconcile those with the values I listed before.

Q4: Is that all that’s wrong with him?

A: No, there are also the lies, and worst of all the “Big Lie.” Trump is the first president in US history to incite a mob to try to overturn the results of an election. He was serious! He very nearly succeeded, and probably would have, had Mike Pence been someone else. It’s now inarguable that Trump rejects the basic rules of our system, or “accepts” them only when he wins. We’re numb from having heard it so many times, but it’s a big deal, as big a deal as the Civil War was.

Q5: Oh, so this is about your precious “democracy.” Why do you care? Haven’t you of all people learned that the masses are mostly idiots and bullies, who don’t deserve power? As Curtis Yarvin keeps trying to explain to you, instead of “democracy,” you should want a benevolent king or dictator-CEO, who could offer a privileged position to the competent scientists like yourself.

A: Yeah, so how many examples does history furnish where that worked out well? I suppose you might make a partial case for Napoleon, or Ataturk? More to the point: even if benevolent, science-and-reason-loving authoritarian strongmen are possible in theory, do you really expect me to believe that Trump could be one of them? I still love how Scott Alexander put it in 2016:

Can anyone honestly say that Trump or his movement promote epistemic virtue? That in the long-term, we’ll be glad that we encouraged this sort of thing, that we gave it power and attention and all the nutrients it needed to grow? That the road to whatever vision of a just and rational society we imagine, something quiet and austere with a lot of old-growth trees and Greek-looking columns, runs through LOCK HER UP?

I don’t like having to vote for the lesser of two evils. But at least I feel like I know who it is.

Q6: But what about J. D. Vance? He got his start in Silicon Valley, was championed by Peter Thiel, and is obviously highly intelligent. Doesn’t he seem like someone who might listen to and empower tech nerds like yourself?

A: Who can say what J. D. Vance believes? Here are a few choice quotes of his from eight years ago:

I’m obviously outraged at Trump’s rhetoric, and I worry most of all about how welcome Muslim citizens feel in their own country. But I also think that people have always believed crazy shit (I remember a poll from a few years back suggesting that a near majority of democratic voters blame ‘the Jews’ for the financial crisis). And there have always been demagogues willing to exploit the people who believe crazy shit.

The more white people feel like voting for trump, the more black people will suffer. I really believe that.

[Trump is] just a bad man. A morally reprehensible human being.

To get from that to being Trump’s running mate is a Simone-Biles-like feat of moral acrobatics. Vance reminds me of the famous saying by L. Ron Hubbard from his pre-Dianetics days: “If a man really wants to make a million dollars, the best way would be to start his own religion.” (And I feel like Harris’s whole campaign strategy should just be to replay Vance’s earlier musings in wall-to-wall ads while emphasizing her agreement with them.) No, Vance is not someone I trust to share my values, if he has values at all.

Q7: What about the other side’s values, or lack thereof? I mean, don’t you care that the whole Democratic establishment—including Harris—colluded to cover up that Biden was senile and cognitively unfit to be president now, let alone for another term?

A: Look, we’ve all seen what happens as a relative gets old. It’s gradual. It’s hard for anyone to say at which specific moment they can no longer drive a car, or be President of the United States, or whatever. This means that I don’t necessarily read evil intent into the attempts to cover up Biden’s decline—merely an epic, catastrophic failure of foresight. That failure of foresight itself would’ve been a huge deal in normal circumstances, but these are not normal circumstances—not if you believe, as I do, that the alternative is the beginning of the end of a 250-year-old democratic experiment.

Q8: Oh stop being so melodramatic. What terrible thing happened to you because of Trump’s first term? Did you lose your job? Did fascist goons rough you up in the street?

A: Well, my Iranian PhD student came close to having his visa revoked, and it became all but impossible to recruit PhD students from China. That sucked, since I care about my students’ welfare like I care about my own. Also, the downfall of Roe v. Wade, which enabled Texas’ draconian new abortion laws, made it much harder for us to recruit faculty at UT Austin. But I doubt any of that will impress you. “Go recruit American students,” you’ll say. “Go recruit conservative faculty who are fine with abortion being banned.”

The real issue is that Trump was severely restrained in his first term, by being surrounded by people who (even if, in many cases, they started out loyal to him) were also somewhat sane and valued the survival of the Republic. Alas, he learned from that, and he won’t repeat that mistake the next time.

Q9: Why do you care so much about Trump’s lies? Don’t you realize that all politicians lie?

A: Yes, but there are importantly different kinds of lies. There are white lies. There are scheming, 20-dimensional Machiavellian lies, like a secret agent’s cover story (or is that only in fiction?). There are the farcical, desperate, ever-shifting lies of the murderer to the police detective or the cheating undergrad to the professor. And then there are the lies of bullies and mob bosses and populist autocrats, which are special and worse.

These last, call them power-lies, are distinguished by the fact that they aren’t even helped by plausibility. Often, as with conspiracy theories (which strongly overlap with power-lies), the more absurd the better. Obama was born in Kenya. Trump’s crowd was the biggest in history. The 2020 election was stolen by a shadowy conspiracy involving George Soros and Dominion and Venezuela.

The central goal of a power-lie is just to demonstrate your power to coerce others into repeating it, much like with the Party making Winston Smith affirm 2+2=5, or Petruchio making Katharina call the sun the moon in The Taming of the Shrew. A closely-related goal is as a loyalty test for your own retinue.

It’s Trump’s embrace of the power-lie that puts him beyond the pale for me.

Q10: But Scott, we haven’t even played our “Trump” card yet. Starting on October 7, 2023, did you not witness thousands of your supposed allies, the educated secular progressives on “the right side of history,” cheer the sadistic mass-murder of Jews—or at least, make endless excuses for those who did? Did this not destabilize your entire worldview? Will you actually vote for a party half of which seems at peace with the prospect of your family members’ physical annihilation? Or will you finally see who your real friends now are: Arkansas MAGA hillbillies who pray for your people’s survival?

A: Ah, this is your first slash that’s actually drawn blood. I won’t pretend that the takeover of part of the US progressive coalition by literal Hamasniks hasn’t been one of the most terrifying experiences of my life. Yes, if I had to be ruled by either (a) a corrupt authoritarian demagogue or (b) an idiot college student chanting for “Intifada Revolution,” I’d be paralyzed. So it’s lucky that I don’t face that choice! I get to vote, once more, for a rather boring mainstream Democrat—alongside at least 70% of American Jews. The idea of Harris as an antisemite would be ludicrous even if she didn’t have a Jewish husband or wasn’t strongly considering a pro-Israel Jew as her running mate.

Q11: Sure, Kamala Harris might mouth all the right platitudes about Israel having a right to defend itself, but she’ll constantly pressure Israel to make concessions to Hamas and Hezbollah. She’ll turn a blind eye to Iran’s imminent nuclearization. Why don’t you stay up at night worrying that, if you vote for a useful idiot like her, you’ll have Israel’s annihilation and a second Holocaust on your conscience forever?

A: Look, oftentimes—whenever, for example, I’m spending hours reading anti-Zionists on Twitter—I feel like there’s no limit to how intensely Zionist I am. On reflection, though, there is a limit. Namely, I’m not going to be more Zionist than the vast majority of my Israeli friends and colleagues—the ones who served in the IDF, who in some cases did reserve duty in Gaza, who prop up the Israeli economy with their taxes, and who will face the consequences of whatever happens more directly than I will. With few exceptions, these friends despise the Trump/Bibi alliance with white-hot rage, and they desperately want more moderate leadership in both countries.

Q12: Suppose I concede that Kamala is OK on Israel. We both know that she’s not the future of the Democratic Party, any more than Biden is. The future is what we all saw on campuses this spring. “Houthis Houthis make us proud, turn another ship around.” How can you vote for a party whose rising generation seems to want you and your family dead?

A: Let me ask you something. When Trump won in 2016, did that check the power of the campus radicals? Or as Scott Alexander prophesied at the time, did it energize and embolden them like nothing else, by dramatically confirming their theology of a planet held hostage by the bullying, misogynistic rich white males? I fundamentally reject your premise that, if I’m terrified of crazy left-wing extremists, then a good response is to vote for the craziest right-wing extremists I can find, in hopes that the two will somehow cancel each other out. Instead I should support a coherent Enlightenment alternative to radicalism, or the closest thing to that available.

Q13: Even leaving aside Israel, how can you not be terrified by what the Left has become? Which side denounced you on social media a decade ago, as a misogynist monster who wanted all women to be his sex slaves? Which side tried to ruin your life and career? Did we, the online rightists, do that? No. We did not. We did nothing worse to you than bemusedly tell you to man up, grow a pair, and stop pleading for sympathy from feminists who will hate you no matter what.

A: I’ll answer with a little digression. Back in 2017, when Kamala Harris was in the Senate, her office invited me to DC to meet with them to provide advice about the National Quantum Initiative Act, which Kamala was then spearheading. Kamala herself sent regrets that she couldn’t meet me, because she had to be at the Kavanaugh hearings. I have (nerdy, male) friends who did meet her about tech policy and came away with positive impressions.

And, I dunno, does that sound like someone who wants me dead for the crime of having been born a nerdy heterosexual male? Or having awkwardly and ineptly asked women on dates, including the one who became my wife? OK, maybe Amanda Marcotte wants me dead for those crimes. Maybe Arthur Chu does (is he still around?). Good that they’re not running for president then.

Q14: Let me try one more time to show you how much your own party hates you. Which side has been at constant war against the SAT and other standardized tests, and merit-based college admissions, and gifted programs, and academic tracking and acceleration, and STEM magnet schools, and every single other measure by which future young Scott Aaronsons (and Saket Agrawals) might achieve their dreams in life? Has that been our side, or theirs?

A: To be honest, I haven’t seen the Trump or Harris campaigns take any position on any of these issues. Even if they did, there’s very little that the federal government can do: these battles happen in individual states and cities and counties and universities. So I’ll vote for Harris while continuing to advocate for what I think is right in education policy.

Q15: Can you not see that Kamala Harris is a vapid, power-seeking bureaucratic machine—that she has no fixed principles at all? For godsakes, she all but condemned Biden as a racist in the 2020 primary, then agreed to serve as his running mate!

A: I mean, she surely has more principles than Vance does. As far as I can tell, for example, she’s genuinely for abortion rights (as I am). Even if she believed in nothing, though, better a cardboard cutout on which values I recognize are written, than a flesh-and-blood person shouting values that horrify me.

Q16: What, if anything, could Republicans do to get you to vote for them?

A: Reject all nutty conspiracy theories. Fully, 100% commit to the peaceful transfer of power. Acknowledge the empirical reality of human-caused climate change, and the need for both technological and legislative measures to slow it and mitigate its impacts. Support abortion rights, or at least a European-style compromise on abortion. Republicans can keep the anti-wokeness stuff, which actually seems to have become their defining issue. If they do all that, and also the Democrats are taken over by frothing radicals who want to annihilate the state of Israel and abolish the police … that’s, uh, probably the point when I start voting Republican.

Q17: Aha, so you now admit that there exist conceivable circumstances that would cause you to vote Republican! In that case, why did you style yourself “Never-Trump From Here to Eternity”?

A: Tell you what, the day the Republicans (and Trump himself?) repudiate authoritarianism and start respecting election outcomes, is the day I’ll admit my title was hyperbolic.

Q18: In the meantime, will you at least treat us Trump supporters with civility and respect?

A: Not only does civil disagreement not compromise any of my values, it is a value to which I think we should all aspire. And to whatever extent I’ve fallen short of that ideal—even when baited into it—I’m sorry and I’ll try to do better. Certainly, age and experience have taught me that there’s hardly anyone so far gone that I can’t find something on which I agree with them, while disagreeing with most of the rest of the world.

View Details

Update (July 24): Remember the quest that Adam Yedidia and I started in 2016, to find the smallest n such that the value of the nth Busy Beaver number can be proven independent of the axioms of ZF set theory? We managed to show that BB(8000) was independent. This was later improved to BB(745) by Stefan O’Rear and Johannes Riebel. Well, today Rohan Ridenour writes to tell me that he’s achieved a further improvement to BB(643). Awesome!

With yesterday’s My Prayer, for the first time I can remember in two decades of blogging, I put up a new post with the comments section completely turned off. I did so because I knew my nerves couldn’t handle a triumphant interrogation from Trumpist commenters about whether, in the wake of their Messiah’s (near-)blood sacrifice on behalf of the Nation, I’d at last acquiesce to the dissolution of America’s constitutional republic and its replacement by the dawning order: one where all elections are fraudulent unless the MAGA candidate wins, and where anything the leader does (including, e.g., jailing his opponents) is automatically immune from prosecution. I couldn’t handle it, but at the same time, and in stark contrast to the many who attack from my left, I also didn’t care what they thought of me.

With hindsight, turning off comments yesterday might be the single best moderation decision I ever made. I still got feedback on what I’d written, on Facebook and by email and text message and in person. But this more filtered feedback was … thoughtful. Incredibly, it lowered the stress that I was feeling rather than raising it even higher.

For context, I should explain that over the past couple years, one or more trolls have developed a particularly vicious strategy against me. Below my every blog post, even the most anodyne, a “new” pseudonymous commenter shows up to question me about the post topic, in what initially looks like a curious, good-faith way. So I engage, because I’m Scott Aaronson and that’s what I do; that’s a large part of the value I can offer the world.

Then, only once a conversation is underway does the troll gradually ratchet up the level of crazy, invariably ending at some place tailor-made to distress me (for example: vaccines are poisonous, death to Jews and Israel, I don’t understand basic quantum mechanics or computer science, I’m a misogynist monster, my childhood bullies were justified and right). Of course, as soon as I’ve confirmed the pattern, I send further comments straight to the trash. But the troll then follows up with many emails taunting me for not engaging further, packed with farcical accusations and misreadings for me to rebut and other bait.

Basically, I’m now consistently subjected to denial-of-service attacks against my open approach to the world. Or perhaps I’ve simply been schooled in why most people with audiences of thousands or more don’t maintain comment sections where, by default, they answer everyone! And yet it’s become painfully clear that, as long as I maintain a quasi-open comment section, I’ll feel guilty if I don’t answer everyone.

So without further ado, I hereby announce my new comment policy. Henceforth all comments to Shtetl-Optimized will be treated, by default, as personal missives to me—with no expectation either that they’ll appear on the blog or that I’ll reply to them.

At my leisure and discretion, and in consultation with the Shtetl-Optimized Committee of Guardians, I’ll put on the blog a curated selection of comments that I judge to be particularly interesting or to move the topic forward, and I’ll do my best to answer those. But it will be more like Letters to the Editor. Anyone who feels unjustly censored is welcome to the rest of the Internet.

The new policy starts now, in the comment section of this post. To the many who’ve asked me for this over the years, you’re welcome!

View Details

It is the duty of good people, always and everywhere, to condemn, reject, and disavow the use of political violence.

Even or especially when evildoers would celebrate the use of political violence against us.

It is our duty always to tell the truth, always to play by the rules — even when evil triumphs by lying, by sneeringly flouting every rule.

It appears to be an iron law of Fate that whenever good tries to steal a victory by evil means, it fails. This law is so infallible that any good that tries to circumvent it thereby becomes evil.

When Sam Bankman-Fried tries to save the world using financial fraud — he fails. Only the selfish succeed through fraud.

When kind, nerdy men, in celibate desperation, try to get women to bed using “Game” and other underhanded tactics — they fail. Only the smirking bullies get women that way.

Quantum mechanics is false, because its Born Rule speaks of randomness.

But randomness can’t explain why a bullet aimed at a destroyer of American democracy must inevitably miss by inches, while a bullet aimed at JFK or RFK or MLK or Gandhi or Rabin must inevitably meet its target.

Yet for all that, over the millennia, good has made actual progress. Slavery has been banished to the shadows. Children survive to adulthood. Sometimes altruists become billionaires, or billionaires altruists. Sometimes the good guy gets the girl.

Good has progressed not by lucky breaks — for good never gets lucky breaks — but only because the principles of good are superior.

There’s a kind of cosmic solace that could be offered even to the Jewish mother in the gas chamber watching her children take their last breaths, though the mother could be forgiven for rejecting it.

The solace is that good will triumph — if not in the next four years, then in the four years after that.

Or if not in four, then in a hundred.

Or if not in a hundred, then in a thousand.

Or if not in the entire history of life in on this planet, then on a different planet.

Or if not in this universe, then in a different universe.

Let us commit to fighting for good using good methods only. Fate has decreed in any case that, for us, those are the only methods that work.

Let us commit to use good methods only even if it means failure, heartbreak, despair, the destruction of democratic institutions and ecosystems multiplied by a thousand or a billion or any other constant — with the triumph of good only in the asymptotic limit.

Good will triumph, when it does, only because its principles are superior.

View Details

Perhaps like the poor current President of the United States, I can feel myself fading, my memory and verbal facility and attention to detail failing me, even while there’s so much left to do to battle the nonsense in the world. I started my career on an accelerated schedule—going to college at 15, finishing my PhD at 22, etc. etc.—and the decline is (alas) also hitting me early, at the ripe age of 43.

Nevertheless, I do seem to remember that this was once primarily a quantum computing blog, and that I was known to the world as a quantum computing theorist. And exciting things continue to happen in quantum computing. In fact, just on last night’s quant-ph arXiv mailing, there were two…

First, a company in the UK called Oxford Ionics has announced that it now has a system of trapped-ion qubits in which it’s prepared two-qubit maximally entangled states with 99.97% fidelity. If true, this seems extremely good. Indeed, it seems better than the numbers from bigger trapped-ion efforts, and quite close to the ~99.99% that you’d want for quantum fault-tolerance. But maybe there’s a catch? Will they not be able to maintain this kind of fidelity when doing a long sequence of programmable two-qubit gates on dozens of qubits? Can the other trapped-ion efforts actually achieve similar fidelities in head-to-head comparisons? Anyway, I was surprised to see how little attention the paper got on SciRate. I look forward to hearing from experts in the comment section.

Second, a new paper by Schuster, Haferkamp, and Huang gives a major advance on k-designs and pseudorandom unitaries. Roughly speaking, the paper shows that even in one dimension, a random n-qubit quantum circuit, with alternating brickwork layers of 2-qubit gates, forms a “k-design” after only O(k polylog k log n) layers of gates. Well, modulo one caveat: the “random circuit” isn’t from the most natural ensemble, but has to have some of its 2-qubit gates set to the identity, namely those that straddle certain contiguous blocks of log n qubits. This seems like a purely technical issue—how could randomizing those straddling gates make the mixing behavior worse?—but future work will be needed to address it. Notably, the new upper bound is off from the best-possible k layers by only logarithmic factors. (For those tuning in from home: a k-design informally means a collection of n-qubit unitaries such that, from the perspective of degree-k polynomials, choosing a unitary randomly from the collection looks the same as choosing randomly among all n-qubit unitary transformations—i.e., from the Haar measure.)

Anyway, even in my current decrepit state, I can see that such a result would have implications for … well, all sorts of things that quantum computing and information theorists care about. Again I welcome any comments from experts!

View Details

In Michael Sipser’s Introduction to the Theory of Computation textbook, he has one Platonically perfect homework exercise, so perfect that I can reconstruct it from memory despite not having opened the book for over a decade. It goes like this:

• Let f:{0,1}→{0,1} be the constant 1 function if God exists, or the constant 0 function if God does not exist. Is f computable? (Hint:* The answer does not depend on your religious beliefs.)

The correct answer is that yes, f is computable. Why? Because the constant 1 function is computable, and so is the constant 0 function, so if f is one or the other, then it’s computable.

If you’re still tempted to quibble, then consider the following parallel question:

• Let n equal 3 if God exists, or 5 if God does not exist. Is n prime?

The answer is again yes: even though n hasn’t been completely mathematically specified, it’s been specified enough for us to say that it’s prime (just like if we’d said, “n is an element of the set {3,5}; is n prime?”). Similarly, f has been specified enough for us to say that it’s computable.

The deeper lesson Sipser was trying to impart is that the concept of computability applies to functions or infinite sequences, not to individual yes-or-no questions or individual integers. Relatedly, and even more to the point: computability is about whether a computer program exists to map inputs to outputs in a specified way; it says nothing about how hard it might be to choose or find or write that program. Writing the program could even require settling God’s existence, for all the definition of computability cares.

Dozens of times in the past 25 years, I’ve gotten some variant on the following question, always with the air that I’m about to bowled over by its brilliance:

• Could the P versus NP question itself be NP-hard, and therefore impossible to solve?

Every time I get this one, I struggle to unpack the layers of misconceptions. But for starters: the concept of “NP-hard” applies to functions or languages, like 3SAT or Independent Set or Clique or whatnot, all of which take an input (a Boolean formula, a graph, etc) and produce a corresponding output. NP-hardness means that, if you had a polynomial-time algorithm to map the inputs to the outputs, then you could convert it via reductions into a polynomial-time algorithm for any language or function in the class NP.

P versus NP, by contrast, is an individual yes-or-no question. Its answer (for all we know) could be independent of the Zermelo-Fraenkel axioms of set theory, but there’s no sense in which the question could be uncomputable or NP-hard. Indeed, a fast program that correctly answers the P vs. NP question trivially exists:

• If P=NP, then the program prints “P=NP.”
• If P≠NP, then the program prints “P≠NP.”

In the comments of last week’s post on the breakthrough determination of Busy Beaver 5, I got several variants on the following question:

• What’s the smallest n for which the value of BB(n) is uncomputable? Could BB(6) already be uncomputable?

Once again, I explained that the Busy Beaver function is uncomputable, but the concept of computability doesn’t apply to individual integers like BB(6). Indeed, whichever integer k turns out to equal BB(6), the program “print k” clearly exists, and it clearly outputs that integer!

Again, we can ask for the smallest n such that the value of BB(n) is unprovable in ZF set theory (or some other system of axioms)—precisely the question that Adam Yedidia and I did ask in 2016 (the current record stands at n=745, improving my and Adam’s n=8000). But every specific integer is “computable”; it’s only the BB function as a whole that’s uncomputable.

Alas, in return for explaining this, I got more pushback, and even ridicule and abuse that I chose to leave in the moderation queue.

So, I’ve come to think of this as the Zombie Misconception of Theoretical Computer Science: this constant misapplication of concepts that were designed for infinite sequences and functions, to individual integers and open problems. (Or, relatedly: the constant conflation of the uncomputability of the halting problem with Gödel incompleteness. While they’re closely related, only Gödel lets you talk about individual statements rather than infinite families of statements, and only Turing-computability is absolute, rather than relative to a system of axioms.)

Anyway, I’m writing this post mostly just so that I have a place to link the next time this pedagogical zombie rises from its grave, muttering “UNCOMPUTABLE INTEGERRRRRRS….” But also so I can query my readers: what are your ideas for how to keep this zombie down?

View Details

The news these days feels apocalyptic to me—as if we’re living through, if not the last days of humanity, then surely the last days of liberal democracy on earth.

All the more reason to ignore all of that, then, and blog instead about the notorious Busy Beaver function! Because holy moly, what news have I got today. For lovers of this super-rapidly-growing sequence of integers, I’ve honored to announce the biggest Busy Beaver development that there’s been since 1983, when I slept in a crib and you booted up your computer using a 5.25-inch floppy. That was the year when Allen Brady determined that BusyBeaver(4) was equal to 107. (Tibor Radó, who invented the Busy Beaver function in the 1960s, quickly proved with his student Shen Lin that the first three values were 1, 6, and 21 respectively. The fourth value was harder.)

Only now, after an additional 41 years, do we know the fifth Busy Beaver value. Today, an international collaboration called bbchallenge is announcing that it’s determined, and even formally verified using the Coq proof system, that BB(5) is equal to 47,176,870—the value that’s been conjectured since 1990, when Heiner Marxen and Jürgen Buntrock discovered a 5-state Turing machine that runs for exactly 47,176,870 steps before halting, when started on a blank tape. The new bbchallenge achievement is to prove that all 5-state Turing machines that run for more steps than 47,176,870, actually run forever—or in other words, that 47,176,870 is the maximum finite number of steps for which any 5-state Turing machine can run. That’s what it means for BB(5) to equal 47,176,870.

For more on this story, see Ben Brubaker’s superb article in Quanta magazine, or bbchallenge’s own announcement. For more background on the Busy Beaver function, see my 2020 survey, or my 2017 big numbers lecture, or my 1999 big numbers essay, or the Googology Wiki page, or Pascal Michel’s survey.

The difficulty in pinning down BB(5) was not just that there are a lot of 5-state Turing machines (16,679,880,978,201 of them to be precise, although symmetries reduce the effective number). The real difficulty is, how do you prove that some given machine runs forever? If a Turing machine halts, you can prove that by simply running it on your laptop until halting (at least if it halts after a “mere” ~47 million steps, which is child’s-play). If, on the other hand, the machine runs forever, via some never-repeating infinite pattern rather than a simple infinite loop, then how do you prove that? You need to find a mathematical reason why it can’t halt, and there’s no systematic method for finding such reasons—that was the great discovery of Gödel and Turing nearly a century ago.

More precisely, the Busy Beaver function grows faster than any function that can be computed, and we know that because if a systematic method existed to compute arbitrary BB(n) values, then we could use that method to determine whether a given Turing machine halts (if the machine has n states, just check whether it runs for more than BB(n) steps; if it does, it must run forever). This is the famous halting problem, which Turing proved to be unsolvable by finite means. The Busy Beaver function is Turing-uncomputability made flesh, a finite function that scrapes the edge of infinity.

There’s also a more prosaic issue. Proofs that particular Turing machines run forever tend to be mind-numbingly tedious. Even supposing you’ve found such a “proof,” why should other people trust it, if they don’t want to spend days staring at the outputs of your custom-written software?

And so for decades, a few hobbyists picked away at the BB(5) problem. One, who goes by the handle “Skelet”, managed to reduce the problem to 43 holdout machines whose halting status was still undetermined. Or maybe only 25, depending who you asked? (And were we really sure about the machines outside those 43?)

The bbchallenge collaboration improved on the situation in two ways. First, it demanded that every proof of non-halting be vetted carefully. While this went beyond the original mandate, a participant named “mxdys” later upped the standard to fully machine-verifiable certificates for every non-halting machine in Coq, so that there could no longer be any serious question of correctness. (This, in turn, was done via “deciders,” programs that were crafted to recognize a specific type of parameterized behavior.) Second, the collaboration used an online forum and a Discord server to organize the effort, so that everyone knew what had been done and what remained to be done.

Despite this, it was far from obvious a priori that the collaboration would succeed. What if, for example, one of the 43 (or however many) Turing machines in the holdout set turned out to encode the Goldbach Conjecture, or one of the other great unsolved problems of number theory? Then the final determination of BB(5) would need to await the resolution of that problem. (We do know, incidentally, that there’s a 27-state Turing machine that encodes Goldbach.)

But apparently the collaboration got lucky. Coq proofs of non-halting were eventually found for all the 5-state holdout machines.

As a sad sidenote, Allen Brady, who determined the value of BB(4), apparently died just a few days before the BB(5) proof was complete. He was doubtful that BB(5) would ever be known. The reason, he wrote in 1988, was that “Nature has probably embedded among the five-state holdout machines one or more problems as illusive as the Goldbach Conjecture. Or, in other terms, there will likely be nonstopping recursive patterns which are beyond our powers of recognition.”

Maybe I should say a little at this point about what the 5-state Busy Beaver—i.e., the Marxen-Buntrock Turing machine that we now know to be the champion—actually does. Interpreted in English, the machine iterates a certain integer function g, which is defined by

• g(x) = (5x+18)/3 if x = 0 (mod 3),
• g(x) = (5x+22)/3 if x = 1 (mod 3),
• g(x) = HALT if x = 2 (mod 3).

Starting from x=0, the machine computes g(0), g(g(0)), g(g(g(0))), and so forth, halting if and if it ever reaches … well, HALT. The machine runs for millions of steps because it so happens that this iteration eventually reaches HALT, but only after a while:

0 → 6 → 16 → 34 → 64 → 114 → 196 → 334 → 564 → 946 → 1584 → 2646 → 4416 → 7366 → 12284 → HALT.

(And also, at each iteration, the machine runs for a number of steps that grows like the square of the number x.)

Some readers might be reminded of the Collatz Conjecture, the famous unsolved problem about whether, if you repeatedly replace a positive integer x by x/2 if x is even or 3x+1 if x is odd, you’ll always eventually reach x=1. As Scott Alexander would say, this is not a coincidence because nothing is ever a coincidence. (Especially not in math!)

It’s a fair question whether humans will ever know the value of BB(6). Pavel Kropitz discovered, a couple years ago, that BB(6) is at least 10^10^10^10^10^10^10^10^10^10^10^10^10^10^10 (i.e., 10 raised to itself 15 times). Obviously Kropitz didn’t actually run a 6-state Turing machine for that number of steps until halting! Instead he understood what the machine did—and it turned out to apply an iterative process similar to the g function above, but this time involving an exponential function. And the process could be proven to halt after ~15 rounds of exponentiation.

Meanwhile Tristan Stérin, who coordinated the bbchallenge effort, tells me that a 6-state machine was recently discovered that “iterates the Collatz-like map {3x/2, (3x-1)/2} from the number 8 and halts if and only if the number of odd terms ever gets bigger than twice the number of even terms.” This shows that, in order to determine the value of BB(6), one would first need to prove or disprove the Collatz-like conjecture that that never happens.

Basically, if and when artificial superintelligences take over the world, they can worry about the value of BB(6). And then God can worry about the value of BB(7).

I first learned about the BB function in 1996, when I was 15 years old, from a book called The New Turing Omnibus by A. K. Dewdney. From what I gather, Dewdney would go on to become a nutty 9/11 truther. But that’s irrelevant to the story. What matters was that his book provided my first exposure to many of the key concepts of computer science, and probably played a role in my becoming a theoretical computer scientist at all.

And of all the concepts in Dewdney’s book, the one I liked the most was the Busy Beaver function. What a simple function! You could easily explain its definition to Archimedes, or Gauss, or any of the other great mathematicians of the past. And yet, by using it, you could name definite positive integers (BB(10), for example) incomprehensibly larger than any that they could name.

It was from Dewdney that I learned that the first four Busy Beaver numbers were the unthreatening-looking 1, 6, 21, and 107 … but then that the fifth value was already unknown (!!), and at any rate at least 47,176,870. I clearly remember wondering whether BB(5) would ever be known for certain, and even whether I might be the one to determine it. That was almost two-thirds of my life ago.

As things developed, I played no role whatsoever in the determination of BB(5) … except for this. Tristan Stérin tells me that reading my survey article, The Busy Beaver Frontier, was what inspired him to start and lead the bbchallenge collaboration that finally cracked the problem. It’s hard to express how gratified that makes me.

Why care about determining particular values of the Busy Beaver function? Isn’t this just a recreational programming exercise, analogous to code golf, rather than serious mathematical research?

I like to answer that question with another question: why care about humans landing on the moon, or Mars? Those otherwise somewhat arbitrary goals, you might say, serve as a hard-to-fake gauge of human progress against the vastness of the cosmos. In the same way, the quest to determine the Busy Beaver numbers is one concrete measure of human progress against the vastness of the arithmetical cosmos, a vastness that we learned from Gödel and Turing won’t succumb to any fixed procedure. The Busy Beaver numbers are just … there, Platonically, as surely as 13 was prime long before the first caveman tried to arrange 13 rocks into a nontrivial rectangle and failed. And yet we might never know the sixth of these numbers and only today learned the fifth.

Anyway, huge congratulations to the bbchallenge team on their accomplishment. At a terrifying time for the world, I’m happy that, whatever happens, at least I lived to see this.

View Details

Dana, the kids, and I got back to the US last week after a month spent in England and then Israel. We decided to visit Israel because … uhh, we heard there’s never been a better time.

We normally go every year to visit Dana’s family and our many friends there, and to give talks. Various well-meaning friends suggested that maybe we should cancel or postpone this year—given, you know, the situation. To me, though, the situation felt like all the more reason to go. To make Israel seem more and more embattled, dangerous, isolated, abnormal, like not an acceptable place to visit (much less live), in order to crater its economy, demoralize its population, and ultimately wipe it from the face of earth … that is explicitly much of the world’s game plan right now, laid out with shocking honesty since October 7 (a day that also showed us what the “decolonization” will, concretely, look like). So, if I oppose this plan, then how could I look myself in the mirror while playing my tiny part in it? Shouldn’t I instead raise a middle finger to those who’d murder my family, and go?

Besides supporting our friends and relatives, though, I wanted to see the post-October-7 reality for myself, rather than just spending hours per day reading about it on social media. I wanted to form my own impression of the mood in Israel: fiercely determined? angry? hopeless? just carrying on like normal?

Anyway, in two meeting-packed weeks, mostly in Tel Aviv but also in Jerusalem, Haifa, and Be’er Sheva, I saw stuff that could support any of those narratives. A lot was as I’d expected, but not everything. In the rest of this post, I’ll share eleven observations:

(1) This presumably won’t shock anyone, but in post-October-7 Israel, you indeed can’t escape October 7. Everywhere you look, on every building, in every lobby, hanging from every highway overpass, there are hostage posters and “Bring Them Home Now” signs and yellow ribbons—starting at the airport, where every single passenger is routed through a long corridor of hostage posters, each one signed and decorated by the hostage’s friends and family. It sometimes felt as though Yad Vashem had expanded to encompass the entire country. Virtually everyone we talked to wanted to share their stories and opinions about the war, most of all their depression and anger. While there was also plenty of discussion about quantum error mitigation and watermarking of large language models and local family dramas, no one even pretended to ignore the war.

(2) Having said that, the morning after we landed, truthfully, that leapt out at me wasn’t anything to do with October 7, hostages, or Gaza. It was the sheer number of children playing outside, in any direction you looked. Full, noisy playgrounds on block after block. It’s one thing to know intellectually that Israel has by far the highest birthrate of any Western country, another to see it for yourself. The typical secular family probably has three kids; the typical Orthodox family has more. (The Arab population is of course also growing rapidly, both in Israel and in the West Bank and Gaza.) New apartment construction is everywhere you look in Tel Aviv, despite building delays caused by the war. And it all seems perfectly normal … unless you’ve lived your whole life in environments where 0.8 or 1.2 children per couple is the norm.

This, of course, has giant implications for anyone interested in Israel’s future. It’s like, a million Israeli leftists could get fed up and flee to the US or Canada or Switzerland, and Israel would still have a large and growing Jewish population—because having a big family is “just what people do” in a state that was founded to defy the Holocaust. In particular: anyone who dreams of dismantling the illegal, settler-colonial, fascist Zionist ethnostate, and freeing Palestine from river to sea, had better have some plan for what they’re going to do with all these millions of young Jews, who don’t appear to be going anywhere.

(3) The second thing I noticed was the heat—comparable to the Texas summer heat that we try to escape when possible. Because of the roasting sun, our own two pampered offspring mostly refused to go outside during daytime, and we mostly met friends indoors. I more than once had the dark thought that maybe Israel will survive Hamas, Hezbollah, Iran, and its own Jewish extremists … only to be finished off in the end (along with much of the rest of the planet) by global warming. I wonder whether Israel will manage to engineer its way out of the crisis, as it dramatically engineered its way out of its water crisis via desalination. The Arab petrostates have been trying to engineer their way out of the Middle East’s increasingly Mercury-like climate, albeit with decidedly mixed results.

(4) But nu, what did our Israeli friends say about the war? Of course it’s a biased sample, because our friends are mostly left-wing academics and tech workers. But, at risk of overgeneralizing: they’re unhappy. Very, very unhappy. As for Bibi Netanyahu and his far-right yes-men? Our friends’ rage at them was truly a sight to behold. American progressives are, likely, mildly irked by Trump in comparison. Yes, our friends blame Bibi for the massive security and intelligence failures that allowed October 7 to happen. They blame him for dragging out the war to stave off elections. They blame him for empowering the contemptible Ben-Gvir and Smotrich. They blame him for his failure to bring back the remaining hostages. Most of all, they blame him for refusing even to meet with the hostage families, and more broadly, for evading responsibility for all that he did wrong, while arrogating credit for any victories (like the rescue of Noa Argamani).

(5) One Israeli friend offered to take me along to the giant anti-Bibi rally that now happens every Saturday night in Azrieli Center in Tel Aviv. (She added that, if I left before 9pm, it would reduce the chances of the police arresting me.) As the intrepid blogger-investigator I am, of course I agreed.

While many of the protesters simply called for new elections to replace Netanyahu (a cause that I 3000% support), others went further, demanding a deal to free the hostages and an immediate end to the war (even if, as they understood, that would leave Hamas in power).

Watching the protesters, smelling their pot smoke that filled the air, I was seized by a thought: these Israeli leftists actually see eye-to-eye with the anti-Israel American leftists on a huge number of issues. In a different world, they could be marching together as allies. Except, of course, for one giant difference: namely, the Tel Aviv protesters are proudly waving Israeli flags (sometimes modified to add anti-Bibi images, or to depict the Star of David “crying”), rather than burning or stomping on those flags. They’re marching to save the Israel that they know and remember, rather than to destroy it.

(6) We did meet one ultra-right-wing (and Orthodox) academic colleague. He was virtually the only person we met on this trip who seemed cheerful and optimistic about Israel’s future. He brought me to his synagogue to celebrate the holiday of Shavuot, while he himself stood guarding the door of the synagogue with a gargantuan rifle (his volunteer duty since October 7). He has six kids.

(7) Again and again, our secular liberal friends told us they’re thinking about moving from Israel, because if the Bibi-ists entrench their power (and of course the demographics are trending in that direction), then they don’t see that the country has any worthwhile future for them or their children. Should this be taken more seriously than the many Americans who promise that this time, for real, they’ll move to Canada if Trump wins? I’m not sure. I can only report what I heard.

(8) At the same time, again and again I got the following question from Israelis (including the leftist ones): how bad is the situation for Jews in the US? Have the universities been taken over by militant anti-Zionists, like it shows in the news? I had to answer: it’s complicated. Because I live my life enbubbled in the STEM field of computer science, surrounded by friends and colleagues of all backgrounds, ethnicities, religions, and political opinions who are thoughtful and decent (otherwise, why would they be my friends and colleagues?), I’m able to live a very nice life even in the midst of loud protesters calling to globalize the intifada against my family.

If, on the other hand, I were in a typical humanities department? Yeah, then I’d be pretty terrified. My basic options would be to (a) shut up about my (ironically) moderate, middle-of-the-road opinions on Israel/Palestine, such as support for the two-state solution; (b) live a miserable and embattled existence; or (c) pack up and move, for example to Israel.

An astounding irony right now is that, just as Israeli leftists are talking about moving from Israel, some of my American Jewish friends have talked to me about moving to Israel, to escape a prejudice that they thought died with their grandparents. I don’t know where the grass is actually greener (or is it brown everywhere?). Nor do I know how many worriers will actually follow through. What’s clear is that, both in Israel and in the diaspora, Jews are feeling an existential fear that they haven’t felt for generations.

(9) Did I fear for my own family’s safety during the trip? Not really. Maybe I should have. When we visited Haifa, we found that GPS was scrambled all across northern Israel, to make targeting harder for Hezbollah missiles. As a result, we couldn’t use Google Maps, got completely lost driving, and had to change plans with our friends. For the first time, now I really feel angry at Hezbollah: they made my life worse and it’s personal!

The funniest part, though, was how the scrambling was implemented: when you opened Google Maps anywhere in the north, it told you that you were in Beirut. It then dutifully gave you walking or driving directions to wherever you were going in Israel, passing through Syria close to Damascus (“warning: this route passes through multiple countries”).

(10) The most darkly comical thing that I heard on the entire trip: “oh, no, I don’t object in the slightest if the anti-Zionists want to kill us all. I only object if they want to kill us because of an incorrect understanding of the relevant history.” Needless to say, this was a professor.

(11) After my two-week investigation, what grand insight can I offer about Israel’s future? Not much, but maybe this: I think we can definitively rule out the scenario where Israel, having been battered by October 7, and bracing itself to be battered worse by Hezbollah, just sort of … withers away and disappears. Yes, Israel might get hotter, more crowded, more dangerous, more right-wing, and more Orthodox. But it will stay right where it is, unless and until its enemies destroy it in a cataclysmic war. You can’t scare people away, break their will, if they believe they have nowhere else on the planet to go. You can only kill them or else live next to them in peace, as the UN proposed in 1947 and as Oslo proposed in the 1990s. May we live to see peace.

Anyway, on that pleasant note, time soon to tune in to the Trump/Biden debate! I wonder who these two gentlemen are, and what they might stand for?

View Details

(See here for Boaz Barak’s obituary, and here for Lance Fortnow’s—they cover different aspects of Luca’s legacy from each other and from this aspect. Also, click here to register for a free online TCS4All talk that Luca was scheduled to give, and that will now be given in his memory, this Monday at 3:30pm Eastern time.)

Luca Trevisan, one of the world’s leading theoretical computer scientists, has succumbed to cancer in Italy, at only 52 years old. I was privileged to know Luca for a quarter-century, first as my complexity theory and cryptography professor at UC Berkeley and as a member of my dissertation committee, and then as a friend and colleague and fellow CS theory blogger.

I regret that I learned of the seriousness of Luca’s condition only a few days ago. So yesterday morning I wrote him a farewell email, under the impression that, while he was now in hospice care, he had at least a few more weeks. Alas, he probably never saw it. So I’m hereby making the email into a memorial post, with small changes mostly to protect people’s privacy.

Dear Luca,

Dana, the kids, and I were traveling in Israel for the past two weeks, when I received the shocking and sad news that this might be my last chance to write to you.

At risk of stating the obvious — you had a very large and positive effect on my life and career. Starting with the complexity theory summer school at the Institute for Advanced Study in 2000, which was the first time we met and also the first time I really experienced the glories of complexity at full blast. And then continuing at Berkeley, TA’ing your algorithms class, which you had to cancel on 9/11 (although students still somehow showed up for office hours lugging their CLRS books…), and dealing with that student who obviously cheated on the midterm although I had stupidly given back to her the evidence that would prove it.

And then your graduate complexity course, where I was very proud to get 100% on your exam, having handwritten it on a train while everyone else used LaTeX (which, embarrassingly, I was still learning). I was a bit less proud to present the Razborov-Rudich paper to the class, and to get questions from you that proved that I understood it less thoroughly than I thought. I emerged from your course far better prepared to do complexity theory than when I entered it.

Later I took your cryptography course, where I came to you afterwards one day to point out that with a quantum computer, you could pull out big Fourier coefficients without all the bother of the Goldreich-Levin theorem. And you said sure, but then you would need a quantum computer. Over 20 years later, Goldreich and Levin (and you?) can say with satisfaction that we still don’t have that scalable quantum computer … but we’re much much closer, I swear!

I still feel bad about the theory lunch talk I gave in 2003, on my complexity-theoretic version of Aumann’s agreement theorem, where I used you and Umesh as characters instead of Alice and Bob, and which then led to unintended references to “Luca’s posterior” (probability distribution, I meant).

I also feel bad about delaying so long the completion of my PhD thesis, until well after I’d started my postdoc in Princeton, so that my former officemate needed to meet you on a street corner in San Francisco to sign the signature page the night before the deadline.

But then a few years later, when Avi and I did the algebrization paper, the fact that you seemed to like it mattered more to me than just about anything else.

Thank you for the excellent dinner when I met you some years ago in Rome. Thank you for the Trevisan-Tulsiani-Vadhan paper, which answered a question we had about BosonSampling (and you probably didn’t even know you were doing quantum computing when you wrote that paper!). Thank you for your blog. Thank you for everything you did for me.

I always enjoyed your dry humor, much of which might sadly be lost to time, unless others wrote it down or it’s on YouTube or something. Two examples spring to my mind across the decades:

• “From my previous lecture, you may have gotten the impression that everything in derandomization is due to Nisan and Wigderson, but this is not the case: Avi has been working with other people as well.”
• After I’d explained that I’d be spending a semester in Jerusalem to work with Avi, despite (at that time) knowing only the most rudimentary Hebrew, such as how to say “please” and “excuse me”: “you mean there are words in Hebrew for ‘please’ and ‘excuse me’?”

Speaking of which, my current trip to Israel has given me many opportunities to reflect on mortality — for all the obvious war-related reasons of course, but also because while we were here, we unexpectedly had to attend two shivas of people in our social circle who died during our trip, one of them from cancer. And we learned about a close friend whose stepson has a brain tumor and might or might not make it. Cancer is a bitch.

Anyway, there’s much more I could write, but I imagine you’re getting flooded with emails right now from all the people whose lives you’ve touched, so I won’t take up more of your time. You’ve made a real difference to the world, to theoretical computer science, and to your friends and colleagues, one that many people would envy.

Best,
Scott

View Details

My friend Leopold Aschenbrenner, who I got to know and respect on OpenAI’s now-disbanded Superalignment team before he left the company under disputed circumstances, just released “Situational Awareness,” one of the most extraordinary documents I’ve ever read. With unusual clarity, concreteness, and seriousness, and with a noticeably different style than the LessWrongers with whom he shares some key beliefs, Leopold sets out his vision of how AI is going to transform civilization over the next 5-10 years. He makes a case that, even after ChatGPT and all that followed it, the world still hasn’t come close to “pricing in” what’s about to hit it. We’re still treating this as a business and technology story like personal computing or the Internet, rather than (also) a national security story like the birth of nuclear weapons, except more so. And we’re still indexing on LLMs’ current capabilities (“fine, so they can pass physics exams, but they still can’t do original physics research“), rather than looking at the difference between now and five years ago, and then trying our best to project forward an additional five years.

Leopold makes an impassioned plea for the US to beat China and its other autocratic adversaries in the race to superintelligence, and to start by preventing frontier model weights from being stolen. He argues that the development of frontier AI models will inevitably be nationalized, once governments wake up to the implications, so we might as well start planning for that now. Parting ways from the Yudkowskyans despite their obvious points of agreement, Leopold is much less worried about superintelligence turning us all into paperclips than he is about it doing the bidding of authoritarian regimes, although he does worry about both.

Leopold foresaw the Covid lockdowns, as well as the current AI boom, before most of us did, and apparently made a lot of money as a result. I don’t know how his latest predictions will look from the standpoint of 2030. In any case, though, it’s very hard for me to imagine anyone in the US national security establishment reading Leopold’s document without crapping their pants. Is that enough to convince you to read it?

View Details

Three times in my life, I’ve gone to museums where I had such a horrifying experience, and was treated by museum staff with such blankfaced contempt, that the only way I could restore any feeling of cosmic justice was to use this blog to impose some cost on the museum for what it did to me. The first time was when my family was turned away from the Mark Twain House in Hartford, CT, after we’d made a long detour there. Turns out it was guided tours only (idiotic policy right there), and while the last guided tour hadn’t yet left and wasn’t full, the blankface behind the desk arbitrarily decided that we’d come too late, and that therefore we should just leave. The second time, a couple months ago, was when my son and I were kicked out of the only decent room in Washington DC’s “Planet Word” — when we were singled out as criminals despite not having passed any sign telling us not to enter — after my son had found the only exhibit in the museum that held his interest.

Today, alas, it was the London Science Museum, which was the very first thing we chose to visit on my kids’ very first visit to the UK. Every positive review of this abomination on the Internet is a lie, every British person should feel ashamed to have the museum represent him or her, and every visitor should avoid it at all costs.

For an hour, my son desperately needed to use a bathroom. But there’s apparently only one set of bathrooms in the whole museum, hidden away in the basement with no signs leading to them. I asked multiple employees. Not one of them could clearly answer the “where is the bathroom?” question — as if I’d asked them for an interdimensional platypus, or as if the English don’t speak English. Eventually I called out loudly: “WOW, OH MY GOD, THERE ARE NO BATHROOMS ON THIS ENTIRE FLOOR! WHAT KIND OF MUSEUM IS THIS? WHAT IDIOTS DESIGNED IT? HOW HORRIBLE COULD IT POSSIBLY BE?” Many employees heard; not one offered to help. May they feel shame until the day they die.

Beyond that, each exhibit was a depressing touch-screen job designed by morons, like the iPad games that otherwise fill my kids’ lives except even less educational and certainly less fun. And I haven’t even mentioned the neverending lines (sorry, “queues”). Belying the English reputation for politeness, other patrons constantly butted ahead of me and my son, so that the queue for each exhibit got longer the longer we stood. In an hour, my son got to see a grand total of two exhibits. They both sucked.

Apparently there are amazing historical artifacts elsewhere in the museum, including a Babbage Analytical Engine. Alas, we felt forced to leave before we got to see any of those.

There’s one other part of the museum that apparently doesn’t suck, called “WonderLab.” But our extra-cost WonderLab tickets were for 4pm, and the rest of the museum sucked so badly that my son and I stormed out beforehand. We instead took a long walk through Hyde Park, talking about all the plants and birds we encountered and ending up at the Diana Memorial Playground. It was an infinitely better experience than the one we’d paid for.

Speaking of which, we got scammed. Just like at Planet Word, the entrance fee — we might as well call it that, for all you’ll realistically avoid paying — is called a “suggested donation.” But then, after the blankfaces twist the knife and ruin your family’s entire vacation, you’re not entitled to a refund, because after all, you technically never bought anything … you just “donated”!

I feel like my standards for museums are rock-bottom. Just provide me and my kids a non-horrible experience. Have stuff for my kids to play with, right now, without directing me or them to go through your blankfaced processes or systems. Have chicken tenders if my kids are hungry, water fountains if they’re thirsty, benches if they’re tired, and bathrooms if they need bathrooms. And most importantly: if you see guests suffering because of the idiocies of your museum’s design, be helpful and apologetic rather than blankfaced and contemptuous.

The London Science Museum failed on each of these counts.

And yes, I know, I know: I’m the crazy one. To paraphrase a famous Londoner, the reasonable man adapts himself to blankfaced museums, while the unreasonable man persists in trying to adapt those museums to himself. Therefore all progress in making museums non-horrible depends on unreasonable men.

We have another week in London and Oxford. I hope and expect it will be better than this!

View Details

I am, of course, sad that Jan Leike and Ilya Sutskever, the two central people who recruited me to OpenAI and then served as my “bosses” there—two people for whom I developed tremendous admiration—have both now resigned from the company. Ilya’s resignation followed the board drama six months ago, but Jan’s resignation last week came as a shock to me and others. The Superalignment team, which Jan and Ilya led and which I was part of, is being split up and merged into other teams at OpenAI.

See here for Ilya’s parting statement, and here for Jan’s. See here for Zvi Mowshowitz’s perspective and summary of reporting on these events. For additional takes, see pretty much the entire rest of the nerd Internet.

As for me? My two-year leave at OpenAI was scheduled to end this summer anyway. It seems pretty clear that I ought to spend my remaining months at OpenAI simply doing my best for AI safety—for example, by shepherding watermarking toward deployment. After a long delay, I’m gratified that interest in watermarking has spiked recently, not only within OpenAI and other companies but among legislative bodies in the US and Europe.

And afterwards? I’ll certainly continue thinking about how AI is changing the world and how (if at all) we can steer its development to avoid catastrophes, because how could I not think about that? I spent 15 years mostly avoiding the subject, and that now seems like a huge mistake, and probably like enough of that mistake for one lifetime.

So I’ll continue looking for juicy open problems in complexity theory that are motivated by interpretability, or scalable oversight, or dangerous capability evaluations, or other aspects of AI safety—I’ve already identified a few such problems! And without giving up on quantum computing (because how could I?), I expect to reorient at least some of my academic work toward problems at the interface of theoretical computer science and AI safety, and to recruit students who want to work on those problems, and to apply for grants about them. And I’ll presumably continue giving talks about this stuff, and doing podcasts and panels and so on—anyway, as long as people keep asking me to!

And I’ll be open to future sabbaticals or consulting arrangements with AI organizations, like the one I’ve done at OpenAI. But I expect that my main identity will always be as an academic. Certainly I never want to be in a position where I have to speak for an organization rather than myself, or censor what I can say in public about the central problems I’m working on, or sign a nondisparagement agreement or anything of the kind.

I can tell you this: in two years at OpenAI, hanging out at the office and meeting the leadership and rank-and-file engineers, I never once found a smoke-filled room where they laugh at all the rubes who take the talk about “safety” and “alignment” seriously. While my interactions were admittedly skewed toward safetyists, the OpenAI folks I met were invariably smart and earnest and dead serious about the mission of getting AI right for humankind.

It’s more than fair for outsiders to ask whether that’s enough, whether even good intentions can survive bad incentives. It’s likewise fair of them to ask: what fraction of compute and other resources ought to be set aside for alignment research? What exactly should OpenAI do on alignment going forward? What should governments force them and other AI companies to do? What should employees and ex-employees be allowed, or encouraged, to share publicly?

I don’t know the answers to these questions, but if you do, feel free to tell me in the comments!

View Details

When I learned of Jim Simons’s passing, I was actually at the Simons Foundation headquarters in lower Manhattan, for the annual board meeting of the unparalleled Quanta Magazine, which Simons founded and named. The meeting was interrupted to share the sad news, before it became public … and then it was continued, because that’s obviously what Simons would’ve wanted. An oil portrait of Simons in the conference room took on new meaning.

See here for the Simons Foundation’s announcement, or here for the NYT’s obituary.

Although the Simons Foundation has had multiple significant influences on my life—funding my research, founding the Simons Institute for Theory of Computing in Berkeley that I often visit (including two weeks ago), and much more—I’ve exchanged all of a few sentences with Jim Simons himself. At a previous Simons Foundation meeting, I think he said he’d heard I’d moved from MIT to UT Austin, and asked whether I’d bought a cowboy hat yet. I said I did but I hadn’t yet worn it non-ironically, and he laughed at that. (My wife Dana knew him better, having spent a day at a brainstorming meeting for what became the Simons Institute, his trademark cigar smoke filling the room.)

I am, of course, in awe of what Jim Simons achieved in all three phases of his career — firstly, in mathematical research, where he introduced Chern-Simons theory and other pioneering contributions and led the math department at Stony Brook; secondly, in founding Renaissance and making insane amounts of money (“disproving the Efficient Market Hypothesis,” as some have claimed); and thirdly, in giving his money away to support basic research and the public understanding of it.

I’m glad that Simons, as a lifelong chain smoker, made it all the way to age 86. And I’m glad that the Simons Foundation, which I’m told will continue in perpetuity with no operational changes, will stand as a testament to his vision for the world.

View Details

Unrelated Announcements: See here for a long interview with me in The Texas Orator, covering the usual stuff (quantum computing, complexity theory, AI safety). And see here for a podcast with me and Spencer Greenberg about a similar mix of topics.

A couple weeks ago, I helped organize UmeshFest: Don’t Miss This Flight, a workshop at UC Berkeley’s Simons Institute to celebrate the 26th birthday of my former PhD adviser Umesh Vazirani. Peter Shor, John Preskill, Manuel Blum, Madhu Sudan, Sanjeev Arora, and dozens of other luminaries of quantum and classical computation were on hand to help tell the story of quantum computing theory and Umesh’s central role in it. There was also constant roasting of Umesh—of his life lessons from the squash court, his last-minute organizational changes and phone calls at random hours. I was delighted to find that my old coinage of “Umeshisms” was simply standard usage among the attendees.

At Berkeley, many things were as I remembered them—my favorite Thai eatery, the bubble tea, the Campanile—but not everything was the same. Here I am in front of Berkeley’s Gaza encampment, a.k.a. its “Anti Zionism Zone” or what was formerly Sproul Plaza (zoom into the chalk):

I felt a need to walk through the Anti Zionism Zone day after day (albeit unassumingly, neither draped in an Israeli flag nor looking to start an argument with anyone), for more-or-less the same reasons why the US regularly sends aircraft carriers through the Strait of Taiwan.

Back in the more sheltered environment of the Simons Institute, it was great to be among friends, some of whom I hadn’t seen since before Covid. Andris Ambainis and I worked together for a bit on an open problem in quantum query complexity, for old times’ sake (we haven’t solved it yet).

And then there were talks! I thought I’d share my own talk, which was entitled The Story of BQP (Bounded-Error Quantum Polynomial-Time). Here are the PowerPoint slides, but I’ll also share screen-grabs for those of you who constantly complain that you can’t open PPTX files.

I was particularly proud of the design of my title slide:

Moving on:

The class BQP/qpoly, I should explain, is all about an advisor who’s all-wise and perfectly benevolent, but who doesn’t have a lot of time to meet with his students, so he simply doles out the same generic advice to all of them, regardless of their thesis problem x.

I then displayed my infamous “Umeshisms” blog post from 2005—one of the first posts in the history of this blog:

As I explained, now that I hang out with the rationalist and AI safety communities, which are also headquartered in Berkeley, I’ve learned that my “Umeshisms” post somehow took on a life of its own. Once, when dining at one of the rationalists’ polyamorous Berkeley group houses, I said this has been lovely but I’ll now need to leave, to visit my PhD former adviser Umesh Vazirani. “You mean the Umesh?!” the rationalists excitedly exclaimed. “Of Umeshisms? If you’ve never missed a flight?”

But moving on:

(Note that by “QBPP,” Bethiaume and Brassard meant what we now call BQP.)

Feynman and Deutsch asked exactly the right question—does simulating quantum mechanics on a classical computer inherently produce an exponential slowdown, or not?—but they lacked most of the tools to start formally investigating the question. A factor-of-two quantum speedup for the XOR function could be dismissed as unimpressive, while a much greater quantum speedup for the “constant vs. balanced” problem could be dismissed as a win against only deterministic classical algorithms, rather than randomized algorithms. Deutsch-Jozsa may have been the first time that an apparent quantum speedup faltered in an honest comparison against classical algorithms. It certainly wasn’t the last!

Ah, but this is where Bernstein and Vazirani enter the scene.

Bernstein and Vazirani didn’t merely define BQP, which remains the central object of study in quantum complexity theory. They also established its most basic properties:

And, at least in the black-box model, Bernstein and Vazirani gave the first impressive quantum speedup for a classical problem that survived in a fair comparison against the best classical algorithm:

The Recursive Bernstein-Vazirani problem, also called Recursive Fourier Sampling, is constructed as a “tree” of instances of the Bernstein-Vazirani problem, where to query the Boolean function at any given level, you need to solve a Bernstein-Vazirani problem for a Boolean function at the level below it, and then run the secret string s through a fixed Boolean function g. For more, see my old paper Quantum Lower Bound for Recursive Fourier Sampling.

Each Bernstein-Vazirani instance has classical query complexity n and quantum query complexity 1. So, if the tree of instances has depth d, then overall the classical query complexity is nd, while the quantum query complexity is only 2d. Where did the 2 come from? From the need to uncompute the secret strings s at each level, to enable quantum interference at the next level up—thereby forcing us to run the algorithm twice. A key insight.

The Recursive Fourier Sampling separation set the stage for Simon’s algorithm, which gave a more impressive speedup in the black-box model, and thence for the famous Shor’s algorithm for factoring and discrete log:

But Umesh wasn’t done establishing the most fundamental properties of BQP! There’s also the seminal 1994 paper by Bennett, Bernstein, Brassard, and Vazirani:

In light of the BV and BBBV papers, let’s see how BQP seems to fit with classical complexity classes—an understanding that’s remained largely stable for the past 30 years:

We can state a large fraction of the research agenda of the whole field, to this day, as questions about BQP:

I won’t have time to discuss all of these questions, but let me at least drill down on the first few.

Many people hoped the list of known problems in BQP would now be longer than it is. So it goes: we don’t decide the truth, we only discover it.

As a 17-year-old just learning about quantum computing in 1998 by reading the Bernstein-Vazirani paper, I was thrilled when I managed to improve their containment BQP ⊆ P#P to BQP ⊆ PP. I thought that would be my big debut in quantum complexity theory. I was then crushed when I learned that Adleman, DeMarrais, and Huang had proved the same thing a year prior. OK, but at least it wasn’t, like, 50 years prior! Maybe if I kept at it, I’d reach the frontier soon enough.

Umesh, from the very beginning, raised the profound question of BQP’s relation to the polynomial hierarchy. Could we at least construct an oracle relative to which BQP⊄PH—or, closely related, relative to which P=NP≠BQP? Recursive Fourier Sampling was a already candidate for such a separation. I spent months trying to prove that candidate wasn’t in PH, but failed. That led me eventually to propose a very different problem, Forrelation, which seemed like a stronger candidate, although I couldn’t prove that either. Finally, in 2018, after four years of effort, Ran Raz and Avishay Tal proved that my Forrelation problem was not in PH, thereby resolving Umesh’s question after a quarter century.

We now know three different ways by which a quantum computer can not merely solve any BQP problem efficiently, but prove its answer to a classical skeptic via an interactive protocol! Using quantum communication, using two entangled (but non-communicating) quantum computers, or using cryptography (this last a breakthrough of Umesh’s PhD student Urmila Mahadev). It remains a great open problem, first posed to my knowledge by Daniel Gottesman, whether one can do it with none of these things.

To see many of the advantages of quantum computation over classical, we’ve learned that we need to broaden our vision beyond BQP (which is a class of languages), to promise problems (like estimating the expectation values of observables), sampling problems (like BosonSampling and Random Circuit Sampling), and relational problems (like the Yamakawa-Zhandry problem, subject of a recent breakthrough). It’s conceivable that quantum advantage could remain for such problems even if it turned out that P=BQP.

A much broader question is whether BQP captures all languages that can be efficiently decided using “reasonable physical resources.” What about chiral quantum field theories, like the Standard Model of elementary particles? What about quantum theories of gravity? Good questions!

Since it was Passover during the talk, I literally said “Dayenu” to Umesh: “if you had only given us BQP, that would’ve been enough! but you didn’t, you gave us so much more!”

Happy birthday Umesh!! We look forward to celebrating again on all your subsequent power-of-2 birthdays.

View Details

FOR IMMEDIATE RELEASE – From the university campuses of Assyria to the thoroughfares of Ur to the palaces of the Hittite Empire, students across the Fertile Crescent have formed human chains, camel caravans, and even makeshift tent cities to protest the oppression of innocent Egyptians by the rogue proto-nation of “Israel” and its vengeful, warlike deity Yahweh. According to leading human rights organizations, the Hebrews, under the leadership of a bearded extremist known as Moses or “Genocide Moe,” have unleashed frogs, wild beasts, hail, locusts, cattle disease, and other prohibited collective punishments on Egypt’s civilian population, regardless of the humanitarian cost.

Human-rights expert Asenath Albanese says that “under international law, it is the Hebrews’ sole responsibility to supply food, water, and energy to the Egyptian populace, just as it was their responsibility to build mud-brick store-cities for Pharoah. Turning the entire Nile into blood, and plunging Egypt into neverending darkness, are manifestly inconsistent with the Israelites’ humanitarian obligations.”

Israelite propaganda materials have held these supernatural assaults to be justified by Pharoah’s alleged enslavement of the Hebrews, as well as unverified reports of his casting all newborn Hebrew boys into the Nile. Chanting “Let My People Go,” some Hebrew counterprotesters claim that Pharoah could end the plagues at any time by simply releasing those held in bondage.

Yet Ptahmose O’Connor, Chair of Middle East Studies at the University of Avaris, retorts that this simplistic formulation ignores the broader context. “Ever since Joseph became Pharoah’s economic adviser, the Israelites have enjoyed a position of unearned power and privilege in Egypt. Through underhanded dealings, they even recruited the world’s sole superpower—namely Adonai, Creator of the Universe—as their ally, removing any possibility that Adonai could serve as a neutral mediator in the conflict. As such, Egypt’s oppressed have a right to resist their oppression by any means necessary. This includes commonsense measures like setting taskmasters over the Hebrews to afflict them with heavy burdens, and dealing shrewdly with them lest they multiply.”

Professor O’Connor, however, dismissed the claims of drowned Hebrew babies as unverified rumors. “Infanticide accusations,” he explained, “have an ugly history of racism, Orientalism, and Egyptophobia. Therefore, unless you’re a racist or an Orientalist, the only possible conclusion is that no Hebrew babies have been drowned in the Nile, except possibly by accident, or of course by Hebrews themselves looking for a pretext to start this conflict.”

Meanwhile, at elite academic institutions across the region, the calls for justice have been deafening. “From the Nile to the Sea of Reeds, free Egypt from Jacob’s seeds!” students chanted. Some protesters even taunted passing Hebrew slaves with “go back to Canaan!”, though others were quick to disavow that message. According to Professor O’Connor, it’s important to clarify that the Hebrews don’t belong in Canaan either, and that finding a place where they do belong is not the protesters’ job.

In the face of such stridency, a few professors and temple priests have called the protests anti-Semitic. The protesters, however, dismiss that charge, pointing as proof to the many Hebrews and other Semitic peoples in their own ranks. For example, Sa-Hathor Goldstein, who currently serves as Pithom College’s Chapter President of Jews for Pharoah, told us that “we stand in solidarity with our Egyptian brethren, with the shepherds, goat-workers, and queer and mummified voices around the world. And every time Genocide Moe strikes down his staff to summon another of Yahweh’s barbaric plagues, we’ll be right there to tell him: Not In Our Name!”

“Look,” Goldstein added softly, “my own grandparents were murdered by Egyptian taskmasters. But the lesson I draw from my family’s tragic history is to speak up for oppressed people everywhere—even the ones who are standing over me with whips.”

“If Yahweh is so all-powerful,” Goldstein went on to ask, “why could He not devise a way to free the Israelites without a single Egyptian needing to suffer? Why did He allow us to become slaves in the first place? And why, after each plague, does He harden Pharoah’s heart against our release? Not only does that tactic needlessly prolong the suffering of Israelites and Egyptians alike, it also infringes on Pharoah’s bodily autonomy.”

But the strongest argument, Goldstein concluded, arching his eyebrow, is that “ever since I started speaking out on this issue, it’s been so easy to get with all the Midianite chicks at my school. That’s because they, like me, see past the endless intellectual arguments over ‘who started’ or ‘how’ or ‘why’ to the emotional truth that the suffering just has to stop, man.”

Last night, college towns across the Tigris, Euphrates, and Nile were aglow with candelight vigils for Baka Ahhotep, an Egyptian taskmaster and beloved father of three cruelly slain by “Genocide Moe,” in an altercation over alleged mistreatment of a Hebrew slave whose details remain disputed.

According to Caitlyn Mentuhotep, a sophomore majoring in hieroglyphic theory at the University of Pi-Ramesses who attended her school’s vigil for Ahhotep, staying true to her convictions hasn’t been easy in the face of Yahweh’s unending plagues—particularly the head lice. “But what keeps me going,” she said, “is the absolute certainty that, when people centuries from now write the story of our time, they’ll say that those of us who stood with Pharoah were on the right side of history.”

Have a wonderful holiday!

View Details

Update (April 19): Apparently a bug has been found, and the author has withdrawn the claim (see the comments).

For those who don’t yet know from their other social media: a week ago the cryptographer Yilei Chen posted a preprint, eprint.iacr.org/2024/555, claiming to give a polynomial-time quantum algorithm to solve lattice problems. For example, it claims to solve the GapSVP problem, which asks to approximate the length of the shortest nonzero vector in a given n-dimensional lattice, to within an approximation ratio of ~n4.5. The best approximation ratio previously known to be achievable in classical or quantum polynomial time was exponential in n.

If it’s correct, this is an extremely big deal. It doesn’t quite break the main lattice-based cryptosystems, but it would put those cryptosystems into a precarious position, vulnerable to a mere further polynomial improvement in the approximation factor. And, as we learned from the recent NIST competition, if the lattice-based and LWE-based systems were to fall, then we really don’t have many great candidates left for post-quantum public-key cryptography! On top of that, a full quantum break of LWE (which, again, Chen is not claiming) would lay waste (in a world with scalable QCs, of course) to a large fraction of the beautiful sandcastles that classical and quantum cryptographers have built up over the last couple decades—everything from Fully Homomorphic Encryption schemes, to Mahadev’s protocol for proving the output of any quantum computation to a classical skeptic.

So on the one hand, this would substantially enlarge the scope of exponential quantum speedups beyond what we knew a week ago: yet more reason to try to build scalable QCs! But on the other hand, it could also fuel an argument for coordinating to slow down the race to scalable fault-tolerant QCs, until the world can get its cryptographic house into better order. (Of course, as we’ve seen with the many proposals to slow down AI scaling, this might or might not be possible.)

So then, is the paper correct? I don’t know. It’s very obviously a serious effort by a serious researcher, a world away from the P=NP proofs that fill my inbox every day. But it might fail anyway. I’ve asked the world experts in quantum algorithms for lattice problems, and they’ve been looking at it, and none of them is ready yet to render a verdict. The central difficulty is that the algorithm is convoluted, and involves new tools that seem to come from left field, including complex Gaussian functions, the windowed quantum Fourier transform, and Karst waves (whatever those are). The algorithm has 9 phases by the author’s count. In my own perusal, I haven’t yet extracted even a high-level intuition—I can’t tell any little story like for Shor’s algorithm, e.g. “first you reduce factoring to period-finding, then you solve period-finding by applying a Fourier transform to a vector of amplitudes.”

So, the main purpose of this post is simply to throw things open to commenters! I’m happy to provide a public clearinghouse for questions and comments about the preprint, if those studying it would like that. You can even embed LaTeX in your comments, as will probably be needed to get anywhere.

Unrelated Update: Connor Tabarrok and his friends just put a podcast with me up on YouTube, in which they interview me in my office at UT Austin about watermarking of large language models and other AI safety measures.

View Details

Back in 2006, in the midst of an unusually stupid debate in the comment section of Lance Fortnow and Bill Gasarch’s blog, someone chimed in:

Since the point of theoretical computer science is solely to recognize who is the most badass theoretical computer scientist, I can only say:

GO HOME PUNKS!

WIGDERSON OWNS YOU!

Avi Wigderson: central unifying figure of theoretical computer science for decades; consummate generalist who’s contributed to pretty much every corner of the field; advocate and cheerleader for the field; postdoc adviser to a large fraction of all theoretical computer scientists, including both me and my wife Dana; derandomizer of BPP (provided E requires exponential-size circuits). Now, Avi not only “owns you,” he also owns a well-deserved Turing Award (on top of his well-deserved Nevanlinna, Abel, Gödel, and Knuth prizes). As Avi’s health has been a matter of concern to those close to him ever since his cancer treatment, which he blogged about a few years ago, I’m sure today’s news will do much to lift his spirits.

I first met Avi a quarter-century ago, when I was 19, at a PCMI summer school on computational complexity at the Institute for Advanced Study in Princeton. Then I was lucky enough to visit Avi in Israel when he was still a professor at the Hebrew University (and I was a grad student at Berkeley)—first briefly, but then Avi invited me back to spend a whole semester in Jerusalem, which ended up being one of my most productive semesters ever. Then Avi, having by then moved to the IAS in Princeton, hosted me for a one-year postdoc there, and later he and I collaborated closely on the algebrization paper. He’s had a greater influence on my career than all but a tiny number of people, and I’m far from the only one who can say that.

Summarizing Avi’s scientific contributions could easily fill a book, but Quanta and New Scientist and Lance’s blog can all get you started if you’re interested. Eight years ago, I took a stab at explaining one tiny little slice of Avi’s impact—namely, his decades-long obsession with “why the permanent is so much harder than the determinant”—in my IAS lecture Avi Wigderson’s “Permanent” Impact On Me, to which I refer you now (I can’t produce a new such lecture on one day’s notice!).

Huge congratulations to Avi.

View Details

Pissing away my life in a haze of doomscrolling, sporadic attempts to “parent” two rebellious kids, and now endless conversations about AI safety, I’m liable to forget for days that I’m still mostly known (such as I am) as a quantum computing theorist, and this blog is still mostly known as a quantum computing blog. Maybe it’s just that I spent a quarter-century on quantum computing theory. As an ADHD sufferer, anything could bore me after that much time, even one of the a-priori most exciting things in the world.

It’s like, some young whippersnappers proved another monster 80-page theorem that I’ll barely understand tying together the quantum PCP conjecture, area laws, and Gibbs states? Another company has a quantum software platform, or hardware platform, and they’ve issued a press release about it? Another hypester claimed that QC will revolutionize optimization and machine learning, based on the usual rogues’ gallery of quantum heuristic algorithms that don’t seem to outperform classical heuristics? Another skeptic claimed that scalable quantum computing is a pipe dream—mashing together the real reasons why it’s difficult with basic misunderstandings of the fault-tolerance theorem? In each case, I’ll agree with you that I probably should get up, sit at my laptop, and blog about it (it’s hard to blog with two thumbs), but as likely as not I won’t.

And yet quantum computing continues to progress. In December we saw Harvard and QuEra announce a small net gain from error-detection in neutral atoms, and accuracy that increased with the use of larger error-correcting codes. Today, a collaboration between Microsoft and Quantinuum has announced what might be the first demonstration of error-corrected two-qubit entangling gates with substantially lower error than the same gates applied to the bare physical qubits. (This is still at the stage where you need to be super-careful in how you phrase every such sentence—experts should chime in if I’ve already fallen short; I take responsibility for any failures to error-correct this post.)

You can read the research paper here, or I’ll tell you the details to the best of my understanding (I’m grateful to Microsoft’s Krysta Svore and others from the collaboration for briefing me by Zoom). The collaboration used a trapped-ion system with 32 fully-connected physical qubits (meaning, the qubits can be shuttled around a track so that any qubit can directly interact with any other). One can apply an entangling gate to any pair of qubits with ~99.8% fidelity.

What did they do with this system? They created up to 4 logical encoded qubits, using the Steane code and other CSS codes. Using logical CNOT gates, they then created logical Bell pairs — i.e., (|00⟩+|11⟩)/√2 — and verified that they did this.

That’s in the version of their experiment that uses “preselection but not postselection.” In other words, they have to try many times until they prepare the logical initial states correctly—as with magic state factories. But once they do successfully prepare the initial states, there’s no further cheating involving postselection (i.e., throwing away bad results): they just apply the logical CNOT gates, measure, and see what they got.

For me personally, that’s the headline result. But then they do various further experiments to “spike the football.” For one thing, they show that when they do allow postselected measurement outcomes, the decrease in the effective error rate can be much much larger, as large as 800x. That allows them (again, under postselection!) to demonstrate up to two rounds of error syndrome extraction and correction while still seeing a net gain, or three rounds albeit with unclear gain. The other thing they demonstrate is teleportation of fault-tolerant qubits—so, a little fancier than just preparing an encoded Bell pair and then measuring it.

They don’t try to do (e.g.) a quantum supremacy demonstration with their encoded qubits, like Harvard/QuEra did—they don’t have nearly enough qubits for that. But this is already extremely cool, and it sets a new bar in quantum error-correction experiments for others to meet or exceed (superconducting, neutral atom, and photonics people, that means you!). And I wasn’t expecting it! Indeed, I’m so far behind the times that I still imagined Microsoft as committed to a strategy of “topological qubits or bust.” While Microsoft is still pursuing the topological approach, their strategy has clearly pivoted over the last few years towards “whatever works.”

Anyway, huge congratulations to the teams at Microsoft and Quantinuum for their accomplishment!

Stepping back, what is the state of experimental quantum computing, 42 years after Feynman’s lecture, 30 years after Shor’s algorithm, 25 years after I entered the field, 5 years after Google’s supremacy experiment? There’s one narrative that quantum computing is already being used to solve practical problems that couldn’t be solved otherwise (look at all the hundreds of startups! they couldn’t possibly exist without providing real value, could they?). Then there’s another narrative that quantum computing has been exposed as a fraud, an impossibility, a pipe dream. Both narratives seem utterly disconnected from the reality on the ground.

If you want to track the experimental reality, my one-sentence piece of advice would be to focus relentlessly on the fidelity with which experimenters can apply a single physical 2-qubit gate. When I entered the field in the late 1990s, ~50% woud’ve been an impressive fidelity. At some point it became ~90%. With Google’s supremacy experiment in 2019, we saw 1000 gates applied to 53 qubits, each gate with ~99.5% fidelity. Now, in superconducting, trapped ions, and neutral atoms alike, we’re routinely seeing ~99.8% fidelities, which is what made possible (for example) the new Microsoft/Quantinuum result. The best fidelities I’ve heard reported this year are more like ~99.9%.

Meanwhile, on paper, it looks like known methods for quantum fault-tolerance, for example using the surface code, should start to become practical once you have 2-qubit fidelities around ~99.99%—i.e., one more “9” from where we are now. And then there should “merely” be the practical difficulty of maintaining that 99.99% fidelity while you scale up to millions or hundreds of millions of physical qubits!

What I’m trying to say is: this looks a pretty good trajectory! It looks like, if we plot the infidelity on a log scale, the experimentalists have already gone three-quarters of the distance. It now looks like it would be a surprise if we couldn’t have hundreds of fault-tolerant qubits and millions of gates on them within the next decade, if we really wanted that—like something unexpected would have to go wrong to prevent it.

Wouldn’t be ironic if all that was true, but it will simply matter much less than we hoped in the 1990s? Either just because the set of problems for which a quantum computing is useful has remained stubbornly more specialized than the world wants it to be (for more on that, see the entire past 20 years of this blog) … or because advances in classical AI render what was always quantum computing’s most important killer app, to the simulation of quantum chemistry and materials, increasingly superfluous (as AlphaFold may have already done for protein folding) … or simply because civilization descends further into barbarism, or the unaligned AGIs start taking over, and we all have bigger things to worry about than fault-tolerant quantum computing.

But, you know, maybe fault-tolerant quantum computing will not only work, but matter—and its use to design better batteries and drugs and photovoltaic cells and so on will pass from science-fiction fantasy to quotidian reality so quickly that much of the world (weary from the hypesters crying wolf too many times?) will barely even notice it when it finally happens, just like what we saw with Large Language Models a few years ago. That would be worth getting out of bed for.

View Details

Dear Twitter Anti-Zionists,

For five months, ever since Oct. 7, I’ve read you obsessively. While my current job is supposed to involve protecting humanity from the dangers of AI (with a side of quantum computing theory), I’m ashamed to say that half the days I don’t do any science; instead I just scroll and scroll, reading anti-Israel content and then pro-Israel content and then more anti-Israel content. I thought refusing to post on Twitter would save me from wasting my life there as so many others have, but apparently it doesn’t, not anymore. (No, I won’t call it “X.”)

At the high end of the spectrum, I religiously check the tweets of Paul Graham, a personal hero and inspiration to me ever since he wrote Why Nerds Are Unpopular twenty years ago, and a man with whom I seem to resonate deeply on every important topic except for two: Zionism and functional programming. At the low end, I’ve read hundreds of the seemingly infinite army of Tweeters who post images of hook-nosed rats with black hats and sidecurls and dollar signs in their eyes, sneering as they strangle the earth and stab Palestinian babies. I study their detailed theories about why the October 7 pogrom never happened, and also it was secretly masterminded by Israel just to create an excuse to mass-murder Palestinians, and also it was justified and thrilling (exactly the same melange long ago embraced for the Holocaust).

I’m aware, of course, that the bottom-feeders make life too easy for me, and that a single Paul Graham who endorses the anti-Zionist cause ought to bother me more than a billion sharers of hook-nosed rat memes. And he does. That’s why, in this letter, I’ll try to stay at the higher levels of Graham’s Disagreement Hierarchy.

More to the point, though, why have I spent so much time on such a depressing, unproductive reading project?

Damned if I know. But it’s less surprising when you recall that, outside theoretical computer science, I’m (alas) mostly known to the world for having once confessed, in a discussion deep in the comment section of this blog, that I spent much of my youth obsessively studying radical feminist literature. I explained that I did that because my wish, for a decade, was to confront progressivism’s highest moral authorities on sex and relationships, and make them tell me either that

(1) I, personally, deserved to die celibate and unloved, as a gross white male semi-autistic STEM nerd and stunted emotional and aesthetic cripple, or else
(2) no, I was a decent human being who didn’t deserve that.

One way or the other, I sought a truthful answer, one that emerged organically from the reigning morality of our time and that wasn’t just an unprincipled exception to it. And I felt ready to pursue progressive journalists and activists and bloggers and humanities professors to the ends of the earth before I’d let them leave this one question hanging menacingly over everything they’d ever written, with (I thought) my only shot at happiness in life hinging on their answer to it.

You might call this my central character flaw: this need for clarity from others about the moral foundations of my own existence. I’m self-aware enough to know that it is a severe flaw, but alas, that doesn’t mean that I ever figured out how to fix it.

It’s been exactly the same way with the anti-Zionists since October 7. Every day I read them, searching for one thing and one thing only: their own answer to the “Jewish Question.” How would they ensure that the significant fraction of the world that yearns to murder all Jews doesn’t get its wish in the 21st century, as to a staggering extent it did in the 20th? I confess to caring about that question, partly (of course) because of the accident of having been born a Jew, and having an Israeli wife and family in Israel and so forth, but also because, even if I’d happened to be a Gentile, the continued survival of the world’s Jews would still seem remarkably bound up with science, Enlightenment, minority rights, liberal democracy, meritocracy, and everything else I’ve ever cared about.

I understand the charges against me. Namely: that if I don’t call for Israel to lay down its arms right now in its war against Hamas (and ideally: to dissolve itself entirely), then I’m a genocidal monster on the wrong side of history. That I value Jewish lives more than Palestinian lives. That I’m a hasbara apologist for the IDF’s mass-murder and apartheid and stealing of land. That if images of children in Gaza with their limbs blown off, or dead in their parents arms, or clawing for bread, don’t cause to admit that Israel is evil, then I’m just as evil as the Israelis are.

Unsurprisingly I contest the charges. As a father of two, I can no longer see any images of child suffering without thinking about my own kids. For all my supposed psychological abnormality, the part of me that’s horrified by such images seems to be in working order. If you want to change my mind, rather than showing me more such images, you’ll need to target the cognitive part of me: the part that asks why so many children are suffering, and what causal levers we’d need to push to reach a place where neither side’s children ever have to suffer like this ever again.

At risk of stating the obvious: my first-order model is that Hamas, with the diabolical brilliance of a Marvel villain, successfully contrived a situation where Israel could prevent the further massacring of its own population only by fighting a gruesome urban war, of a kind that always, anywhere in the world, kills tens of thousands of civilians. Hamas, of course, was helped in this plan by an ideology that considers martyrdom the highest possible calling for the innocents who it rules ruthlessly and hides underneath. But Hamas also understood that the images of civilian carnage would (rightly!) shock the consciences of Israel’s Western allies and many Israelis themselves, thereby forcing a ceasefire before the war was over, thereby giving Hamas the opportunity to regroup and, with God’s and of course Iran’s help, finally finish the job of killing all Jews another day.

And this is key: once you remember why Hamas launched this war and what its long-term goals are, every detail of Twitter’s case against Israel has to be reexamined in a new light. Take starvation, for example. Clearly the only explanation for why Israelis would let Gazan children starve is the malice in their hearts? Well, until you think through the logistical challenges of feeding 2.3 million starving people whose sole governing authority is interested only in painting the streets red with Jewish blood. Should we let that authority commandeer the flour and water for its fighters, while innocents continue to starve? No? Then how about UNRWA? Alas, we learned that UNRWA, packed with employees who cheered the Oct. 7 massacre in their Telegram channels and in some cases took part in the murders themselves, capitulates to Hamas so quickly that it effectively is Hamas. So then Israel should distribute the food itself! But as we’ve dramatically witnessed, Israel can’t distribute food without imposing order, which would seem to mean reoccupying Gaza and earning the world’s condemnation for it. Do you start to appreciate the difficulty of the problem—and why the Biden administration was pushed to absurd-sounding extremes like air-dropping food and then building a floating port?

It all seems so much easier, once you remove the constraint of not empowering Hamas in its openly-announced goal of completing the Holocaust. And hence, removing that constraint is precisely what the global left does.

For all that, by Israeli standards I’m firmly in the anti-Netanyahu, left-wing peace camp—exactly where I’ve been since the 1990s, as a teenager mourning the murder of Rabin. And I hope even the anti-Israel side might agree with me that, if all the suffering since Oct. 7 has created a tiny opening for peace, then walking through that opening depends on two things happening:

1. the removal of Netanyahu, and
2. the removal of Hamas.

The good news is that Netanyahu, the catastrophically failed “Protector of Israel,” not only can, but plausibly will (if enough government ministers show some backbone), soon be removed in a democratic election.

Hamas, by contrast, hasn’t allowed a single election since it took power in 2006, in a process notable for its opponents being thrown from the roofs of tall buildings. That’s why even my left-leaning Israeli colleagues—the ones who despise Netanyahu, who marched against him last year—support Israel’s current war. They support it because, even if the Israeli PM were Fred Rogers, how can you ever get to peace without removing Hamas, and how can you remove Hamas except by war, any more than you could cut a deal with Nazi Germany?

I want to see the IDF do more to protect Gazan civilians—despite my bitter awareness of survey data suggesting that many of those civilians would murder my children in front of me if they ever got a chance. Maybe I’d be the same way if I’d been marinated since birth in an ideology of Jew-killing, and blocked from other sources of information. I’m heartened by the fact that despite this, indeed despite the risk to their lives for speaking out, a full 15% of Gazans openly disapprove of the Oct. 7 massacre. I want a solution where that 15% becomes 95% with the passing of generations. My endgame is peaceful coexistence.

But to the anti-Zionists I say: I don’t even mind you calling me a baby-eating monster, provided you honestly field one question. Namely:

Suppose the Palestinian side got everything you wanted for it; then what would be your plan for the survival of Israel’s Jews?

Let’s assume that not only has Netanyahu lost the next election in a landslide, but is justly spending the rest of his life in Israeli prison. Waving my wand, I’ve made you Prime Minister in his stead, with an overwhelming majority in the Knesset. You now get to go down in history as the liberator of Palestine. But you’re now also in charge of protecting Israel’s 7 million Jews (and 2 million other residents) from near-immediate slaughter at the hands of those who you’ve liberated.

Granted, it seems pretty paranoid to expect such a slaughter! Or rather: it would seem paranoid, if the Palestinians’ Grand Mufti (progenitor of the Muslim Brotherhood and hence Hamas) hadn’t allied himself with Hitler in WWII, enthusiastically supported the Nazi Final Solution, and tried to export it to Palestine; if in 1947 the Palestinians hadn’t rejected the UN’s two-state solution (the one Israel agreed to) and instead launched another war to exterminate the Jews (a war they lost); if they hadn’t joined the quest to exterminate the Jews a third time in 1967; etc., or if all this hadn’t happened back before there were any settlements or occupation, when the only question on the table was Israel’s existence. It would seem paranoid if Arafat had chosen a two-state solution when Israel offered it to him at Camp David, rather than suicide bombings. It would seem paranoid if not for the candies passed out in the streets in celebration on October 7.

But if someone has a whole ideology, which they teach their children and from which they’ve never really wavered for a century, about how murdering you is a religious honor, and also they’ve actually tried to murder you at every opportunity—-what more do you want them to do, before you’ll believe them?

So, you tell me your plan for how to protect Israel’s 7 million Jews from extermination at the hands of neighbors who have their extermination—my family’s extermination—as their central political goal, and who had that as their goal long before there was any occupation of the West Bank or Gaza. Tell me how to do it while protecting Palestinian innocents. And tell me your fallback plan if your first plan turns out not to work.

We can go through the main options.

(1) UNILATERAL TWO-STATE SOLUTION

Maybe your plan is that Israel should unilaterally dismantle West Bank settlements, recognize a Palestinian state, and retreat to the 1967 borders.

This is an honorable plan. It was my preferred plan—until the horror of October 7, and then the even greater horror of the worldwide left reacting to that horror by sharing celebratory images of paragliders, and by tearing down posters of kidnapped Jewish children.

Today, you might say October 7 has sort of put a giant flaming-red exclamation point on what’s always been the central risk of unilateral withdrawal. Namely: what happens if, afterward, rather than building a peaceful state on their side of the border, the Palestinian leadership chooses instead to launch a new Iran-backed war on Israel—one that, given the West Bank’s proximity to Israel’s main population centers, makes October 7 look like a pillow fight?

If that happens, will you admit that the hated Zionists were right and you were wrong all along, that this was never about settlements but always, only about Israel’s existence? Will you then agree that Israel has a moral prerogative to invade the West Bank, to occupy and pacify it as the Allies did Germany and Japan after World War II? Can I get this in writing from you, right now? Or, following the future (October 7)2 launched from a Judenfrei West Bank, will your creativity once again set to work constructing a reason to blame Israel for its own invasion—because you never actually wanted a two-state solution at all, but only Israel’s dismantlement?

(2) NEGOTIATED TWO-STATE SOLUTION

So, what about a two-state solution negotiated between the parties? Israel would uproot all West Bank settlements that prevent a Palestinian state, and resettle half a million Jews in pre-1967 Israel—in exchange for the Palestinians renouncing their goal of ending Israel’s existence, via a “right of return” or any other euphemism.

If so: congratulations, your “anti-Zionism” now seems barely distinguishable from my “Zionism”! If they made me the Prime Minister of Israel, and put you in charge of the Palestinians, I feel optimistic that you and I could reach a deal in an hour and then go out for hummus and babaganoush.

(3) SECULAR BINATIONAL STATE

In my experience, in the rare cases they deign to address the question directly, most anti-Zionists advocate a “secular, binational state” between the Jordan and Mediterranean, with equal rights for all inhabitants. Certainly, that would make sense if you believe that Israel is an apartheid state just like South Africa.

To me, though, this analogy falls apart on a single question: who’s the Palestinian Nelson Mandela? Who’s the Palestinian leader who’s ever said to the Jews, “end your Jewish state so that we can live together in peace,” rather than “end your Jewish state so that we can end your existence”? To impose a binational state would be to impose something, not only that Israelis regard as an existential horror, but that most Palestinians have never wanted either.

But, suppose we do it anyway. We place 7 million Jews, almost half the Jews who remain on Earth, into a binational state where perhaps a third of their fellow citizens hold the theological belief that all Jews should be exterminated, and that a heavenly reward follows martyrdom in blowing up Jews. The exterminationists don’t quite have a majority, but they’re the second-largest voting bloc. Do you predict that the exterminationists will give up their genocidal ambition because of new political circumstances that finally put their ambition within reach? If October-7 style pogroms against Jews turn out to be a regular occurrence in our secular binational state, how will its government respond—like the Palestinian Authority? like UNRWA? like the British Mandate? like Tsarist Russia?

In such a case, perhaps the Jews (along with those Arabs and Bedouins and Druze and others who cast their lot with the Jews) would need form a country-within-a-country: their own little autonomous zone within the binational state, with its own defense force. But of course, such a country-within-a-country already formed, for pretty much this exact reason. It’s called Israel. A cycle has been detected in your arc of progress.

(4) EVACUATION OF THE JEWS FROM ISRAEL

We come now to the anti-Zionists who are plainspoken enough to say: Israel’s creation was a grave mistake, and that mistake must now be reversed.

This is a natural option for anyone who sees Israel as an “illegitimate settler-colonial project,” like British India or French Algeria, but who isn’t quite ready to call for another Jewish genocide.

Again, the analogy runs into obvious problems: Israelis would seem to be the first “settler-colonialists” in the history of the world who not only were indigenous to the land they colonized, as much as anyone was, but who weren’t colonizing on behalf of any mother country, and who have no obvious such country to which they can return.

Some say spitefully: then let the Jews go back to Poland. These people might be unaware that, precisely because of how thorough the Holocaust was, more Israeli Jews trace their ancestry to Muslim countries than to Europe. Is there to be a “right of return” to Egypt, Iraq, Morocco, and Yemen, for all the Jews forcibly expelled from those places and for their children and grandchildren?

Others, however, talk about evacuating the Jews from Israel with goodness in their hearts. They say: we’d love the Israelis’ economic dynamism here in Austin or Sydney or Oxfordshire, joining their many coreligionists who already call these places home. What’s more, they’ll be safer here—who wants to live with missiles raining down on their neighborhood? Maybe we could even set aside some acres in Montana for a new Jewish homeland.

Again, if this is your survival plan, I’m a billion times happier to discuss it openly than to have it as unstated subtext!

Except, maybe you could say a little more about the logistics. Who will finance the move? How confident are you that the target country will accept millions of defeated, desperate Jews, as no country on earth was the last time this question arose?

I realize it’s no longer the 1930s, and Israel now has friends, most famously in America. But—what’s a good analogy here? I’ve met various Silicon Valley gazillionaires. I expect that I could raise millions from them, right now, if I got them excited about a new project in quantum computing or AI or whatever. But I doubt I could raise a penny from them if I came to them begging for their pity or their charity.

Likewise: for all the anti-Zionists’ loudness, a solid majority of Americans continue to support Israel (which, incidentally, provides a much simpler explanation than the hook-nosed perfidy of AIPAC for why Congress and the President mostly support it). But it seems to me that Americans support Israel in the “exciting project” sense, rather than in the “charity” sense. They like that Israelis are plucky underdogs who made the deserts bloom, and built a thriving tech industry, and now produce hit shows like Shtisel and Fauda, and take the fight against a common foe to the latter’s doorstep, and maintain one of the birthplaces of Western civilization for tourists and Christian pilgrims, and restarted the riveting drama of the Bible after a 2000-year hiatus, which some believe is a crucial prerequisite to the Second Coming.

What’s important, for present purposes, is not whether you agree with any of these rationales, but simply that none of them translate into a reason to accept millions of Jewish refugees.

But if you think dismantling Israel and relocating its seven million Jews is a workable plan—OK then, are you doing anything to make that more than a thought experiment, as the Zionists did a century ago with their survival plan? Have even I done more to implement your plan than you have, by causing one Israeli (my wife) to move to the US?

Suppose you say it’s not your job to give me a survival plan for Israel’s Jews. Suppose you say the request is offensive, an attempt to distract from the suffering of the Palestinians, so you change the subject.

In that case, fine, but you can now take off your cloak of righteousness, your pretense of standing above me and judging me from the end of history. Your refusal to answer the question amounts to a confession that, for you, the goal of “a free Palestine from the river to the sea” doesn’t actually require the physical survival of Israel’s Jews.

Which means, we’ve now established what you are. I won’t give you the satisfaction of calling you a Nazi or an antisemite. Thousands of years before those concepts existed, Jews already had terms for you. The terms tended toward a liturgical register, as in “those who rise up in every generation to destroy us.” The whole point of all the best-known Jewish holidays, like Purim yesterday, is to talk about those wicked would-be destroyers in the past tense, with the very presence of live Jews attesting to what the outcome was.

(Yesterday, I took my kids to a Purim carnival in Austin. Unlike in previous years, there were armed police everywhere. It felt almost like … visiting Israel.)

If you won’t answer the question, then it wasn’t Zionist Jews who told you that their choices are either to (1) oppose you or else (2) go up in black smoke like their grandparents did. You just told them that yourself.

Many will ask: why don’t I likewise have an obligation to give you my Palestinian survival plan?

I do. But the nice thing about my position is that I can tell you my Palestinian survival plan cheerfully, immediately, with zero equivocating or changing the subject. It’s broadly the same plan that David Ben-Gurion and Yitzchak Rabin and Ehud Barak and Bill Clinton and the UN put on the table over and over and over, only for the Palestinians’ leaders to sweep it off.

I want the Palestinians to have a state, comprising the West Bank and Gaza, with a capital in East Jerusalem. I want Israel to uproot all West Bank settlements that prevent such a state. I want this to happen the instant there arises a Palestinian leadership genuinely committed to peace—one that embraces liberal values and rejects martyr values, in everything from textbooks to street names.

And I want more. I want the new Palestinian state to be as prosperous and free and educated as modern Germany and Japan are. I want it to embrace women’s rights and LGBTQ+ rights and the rest of the modern package, so that “Queers for Palestine” would no longer be a sick joke. I want the new Palestine to be as intertwined with Israel, culturally and economically, as the US and Canada are.

Ironically, if this ever became a reality, then Israel-as-a-Jewish-state would no longer be needed—but it’s certainly needed in the meantime.

Anti-Zionists on Twitter: can you be equally explicit about what you want?

I come, finally, to what many anti-Zionists regard as their ultimate trump card. Look at all the anti-Zionist Jews and Israelis who agree with us, they say. Jewish Voice for Peace. IfNotNow. Noam Chomsky. Norman Finkelstein. The Neturei Karta.

Intellectually, of course, the fact of anti-Zionist Jews makes not the slightest difference to anything. My question for them remains exactly the same as for anti-Zionist Gentiles: what is your Jewish survival plan, for the day after we dismantle the racist supremacist apartheid state that’s currently the only thing standing between half the world’s remaining Jews and their slaughter by their neighbors? Feel free to choose from any of the four options above, or suggest a fifth.

But in the event that Jewish anti-Zionists evade that conversation, or change the subject from it, maybe some special words are in order. You know the famous Golda Meir line, “If we have to choose between being dead and pitied and being alive with a bad image, we’d rather be alive and have the bad image”?

It seems to me that many anti-Zionist Jews considered Golda Meir’s question carefully and honestly, and simply decided it the other way, in favor of Jews being dead and pitied.

Bear with me here: I won’t treat this as a reductio ad absurdum of their position. Not even if the anti-Zionist Jews themselves wish to remain safely ensconced in Berkeley or New Haven, while the Israelis fulfill the “dead and pitied” part for them.

In fact, I’ll go further. Again and again in life I’ve been seized by a dark thought: if half the world’s Jews can only be kept alive, today, via a militarized ethnostate that constantly needs to defend its existence with machine guns and missiles, racking up civilian deaths and destabilizing the world’s geopolitics—if, to put a fine point on it, there are 16 million Jews in the world, but at least a half billion antisemites who wake up every morning and go to sleep every night desperately wishing those Jews dead—then, from a crude utilitarian standpoint, might it not be better for the world if we Jews vanished after all?

Remember, I’m someone who spent a decade asking myself whether the rapacious, predatory nature of men’s sexual desire for women, which I experienced as a curse and an affliction, meant that the only moral course for me was to spend my life as a celibate mathematical monk. But I kept stumbling over one point: why should such a moral obligation fall on me alone? Why doesn’t it fall on other straight men, particularly the ones who presume to lecture me on my failings?

And also: supposing I did take the celibate monk route, would even that satisfy my haters? Would they come after me anyway for glancing at a woman too long or making an inappropriate joke? And also: would the haters soon say I shouldn’t have my scientific career either, since I’ve stolen my coveted academic position from the underprivileged? Where exactly does my self-sacrifice end?

When I did, finally, start approaching women and asking them out on dates, I worked up the courage partly by telling myself: I am now going to do the Zionist thing. I said: if other nerdy Jews can risk death in war, then this nerdy Jew can risk ridicule and contemptuous stares. You can accept that half the world will denounce you as a monster for living your life, so long as your own conscience (and, hopefully, the people you respect the most) continue to assure you that you’re nothing of the kind.

This took more than a decade of internal struggle, but it’s where I ended up. And today, if anyone tells me I had no business ever forming any romantic attachments, I have two beautiful children as my reply. I can say: forget about me, you’re asking for my children never to have existed—that’s why I’m confident you’re wrong.

Likewise with the anti-Zionists. When the Twitter-warriors share their memes of hook-nosed Jews strangling the planet, innocent Palestinian blood dripping from their knives, when the global protests shut down schools and universities and bridges and parliament buildings, there’s a part of me that feels eager to commit suicide if only it would appease the mob, if only it would expiate all the cosmic guilt they’ve loaded onto my shoulders.

But then I remember that this isn’t just about me. It’s about Einstein and Spinoza and Feynman and Erdös and von Neumann and Weinberg and Landau and Michelson and Rabi and Tarski and Asimov and Sagan and Salk and Noether and Meitner, and Irving Berlin and Stan Lee and Rodney Dangerfield and Steven Spielberg. Even if I didn’t happen to be born Jewish—if I had anything like my current values, I’d still think that so much of what’s worth preserving in human civilization, so much of math and science and Enlightenment and democracy and humor, would seem oddly bound up with the continued survival of this tiny people. And conversely, I’d think that so much of what’s hateful in civilization would seem oddly bound up with the quest to exterminate this tiny people, or to deny it any means to defend itself from extermination.

So that’s my answer, both to anti-Zionist Gentiles and to anti-Zionist Jews. The problem of Jewish survival, on a planet much of which yearns for the Jews’ annihilation and much of the rest of which is indifferent, is both hard and important, like P versus NP. And so a radical solution was called for. The solution arrived at a century ago, at once brand-new and older than Homer and Hesiod, was called the State of Israel. If you can’t stomach that solution—if, in particular, you can’t stomach the violence needed to preserve it, so long as Israel’s neighbors retain their annihilationist dream—then your response ought to be to propose a better solution. I promise to consider your solution in good faith—asking, just like with P vs. NP provers, how you overcome the problems that doomed all previous attempts. But if you throw my demand for a better solution back in my face, then you might as well be pushing my kids into a gas chamber yourself, for all the moral authority that I now recognize you to have over me.

Possibly the last thing Einstein wrote was a speech celebrating Israel’s 7th Independence Day, which he died a week before he was to deliver. So let’s turn the floor over to Mr. Albert, the leftist pacifist internationalist:

This is the seventh anniversary of the establishment of the State of Israel. The establishment of this State was internationally approved and recognised largely for the purpose of rescuing the remnant of the Jewish people from unspeakable horrors of persecution and oppression.

Thus, the establishment of Israel is an event which actively engages the conscience of this generation. It is, therefore, a bitter paradox to find that a State which was destined to be a shelter for a martyred people is itself threatened by grave dangers to its own security. The universal conscience cannot be indifferent to such peril.

It is anomalous that world opinion should only criticize Israel’s response to hostility and should not actively seek to bring an end to the Arab hostility which is the root cause of the tension.

I love Einstein’s use of “anomalous,” as if this were a physics problem. From the standpoint of history, what’s anomalous about the Israeli-Palestinian conflict is not, as the Twitterers claim, the brutality of the Israelis—if you think that’s anomalous, you really haven’t studied history—but something different. In other times and places, an entity like Palestine, which launches a war of total annihilation against a much stronger neighbor, and then another and another, would soon disappear from the annals of history. Israel, however, is held to a different standard. Again and again, bowing to international pressure and pressure from its own left flank, the Israelis have let their would-be exterminators off the hook, bruised but mostly still alive and completely unrepentant, to have another go at finishing the Holocaust in a few years. And after every bout, sadly but understandably, Israeli culture drifts more to the right, becomes 10% more like the other side always was.

I don’t want Israel to drift to the right. I find the values of Theodor Herzl and David Ben-Gurion to be almost as good as any human values have ever been, and I’d like Israel to keep them. Of course, Israel will need to continue defending itself from genocidal neighbors, until the day that a leader arises among the Palestinians with the moral courage of Egypt’s Anwar Sadat or Jordan’s King Hussein: a leader who not only talks peace but means it. Then there can be peace, and an end of settlements in the West Bank, and an independent Palestinian state. And however much like dark comedy that seems right now, I’m actually optimistic that it will someday happen, conceivably even soon depending on what happens in the current war. Unless nuclear war or climate change or AI apocalypse makes the whole question moot.

Anyway, thanks for reading—a lot built up these past months that I needed to get off my chest. When I told a friend that I was working on this post, he replied “I agree with you about Israel, of course, but I choose not to die on that hill in public.” I answered that I’ve already died on that hill and on several other hills, yet am somehow still alive!

Meanwhile, I was gratified that other friends, even ones who strongly disagree with me about Israel, told me that I should not disengage, but continue to tell it like I see it, trying civilly to change minds while being open to having my own mind changed.

And now, maybe, I can at last go back to happier topics, like how to prevent the destruction of the world by AI.

Cheers,
Scott

View Details

In fact, don’t try to take kids to Washington DC if you can possibly avoid it.

This is my public service announcement. This is the value I feel I can add to the world today.

Dana and I decided to take the kids to DC for spring break. The trip, alas, has been hell—a constant struggle against logistical failures. The first days were mostly spent sitting in traffic or searching for phantom parking spaces that didn’t exist. (So then we switched to the Metro, and promptly got lost, and had our metro cards rejected by the machines.) Or, at crowded cafes, I spent the time searching for a table so my starving kids could eat—and then when I finally found a table, a woman, smug and sure-faced, evicted us from the table because she was “going to” sit there, and my kids had to see that their dad could not provide for their basic needs, and that woman will never face any consequence for what she did.

Anyway, this afternoon, utterly frazzled and stressed and defeated, we entered “Planet Word,” a museum about language. Sounds pretty good, right? Except my soon-to-be 7-year-old son got bored by numerous exhibits that weren’t for him. So I told him he could lead the way and find any exhibit he liked.

Finally my son found an exhibit that fascinated him, one where he could weigh plastic fruits on a balancing scale. He was engrossed by it, he was learning, he was asking questions, I reflected that maybe the trip wasn’t a total loss … and that’s when a museum employee pointed at us, and screamed at us to leave the room, because “this exhibit was sold out.”

The room was actually almost empty (!). No one had stopped us from entering the room. No one else was waiting to use the balancing scale. There was no sign to warn us we were doing anything wrong. I would’ve paid them hundreds of dollars in that moment if only we could stay. My son didn’t understand why he was suddenly treated as a delinquent. He then wanted to leave the whole museum, and so did I. The day was ruined for us.

Mustering my courage to do something uncharacteristic for me, I complained at the front desk. They sneered and snickered at me, basically told me to go to hell. Looking deeply into their dumb, blank expressions, I realized that I had as much chance of any comprehension or sympathy as I’d have from a warthog. It’s true that, on the scale of all the injustices in the history of the world, this one surely didn’t crack the top quadrillion. But for me, in that moment, it came to stand for all the others. Which has always been my main weakness as a person, that injustice affects me in that way.

Speaking of which, there was one part of DC trip that went exactly like it was supposed to. That was our visit to the United States Holocaust Memorial Museum. Why? Because I feel like that museum, unlike all the rest, tells me the truth about the nature of the world that I was born into—and seeing the truth is perversely comforting. I was born into a world that right now, every day, is filled with protesters screaming for my death, for my family’s death—and this is accepted as normal, and those protesters sleep soundly at night, congratulating themselves for their progressivism and enlightenment. And thinking about those protesters, and their predecessors 80 years ago who perpetrated the Holocaust or who stood by and let it happen, is the only thing that really puts blankfaced museum employees into perspective for me. Like, of course a world with the former is also going to have the latter—and I should count myself immeasurably lucky if the latter is all I have to deal with, if the empty-skulled and the soul-dead can only ruin my vacation and lack the power to murder my family.

And to anyone who reached the end of this post and who feels like it was an unwelcome imposition on their time: I’m sorry. But the truth is, posts like this are why I started this blog and why I continue it. If I’ve ever imparted any interesting information or ideas, that’s a byproduct that I’m thrilled about. But I’m cursed to be someone who wakes up every morning, walks around every day, and goes to sleep every night crushed by the weight of the world’s injustice, and outside of technical subjects, the only thing that’s ever motivated me to write is that words are the only justice available to me.

View Details

Update: Alright, I’m back in. (After trying the same recovery mechanisms that didn’t work before, but suddenly did work this afternoon.) Thanks also to the Facebook employee who emailed offering to help. Now I just need to decide the harder question of whether I want to be back in!

So I’ve been locked out of Facebook and Messenger, possibly forever. It started yesterday morning, when Facebook went down for the entire world. Now it’s back up for most people, but I can’t get in—neither with passwords (none of which work), nor with text messages to my phone (my phone doesn’t receive them for some reason). As a last-ditch measure, I submitted my driver’s license into a Facebook black hole from which I don’t expect to hear back.

Incidentally, this sort of thing is why, 25 years ago, I became a theoretical rather than applied computer scientist. Even before you get to any serious software engineering, the applied part of computing involves a neverending struggle to make machines do what you need them to do—get a document to print, a website to load, a software package to install—in ways that are harrowing and not the slightest bit intellectually interesting. You learn, not about the nature of reality, but only about the terrible design decisions of other people. I might as well be a 90-year-old grandpa with such things, and if I didn’t have the excuse of being a theorist, that fact would constantly humiliate me before my colleagues.

Anyway, maybe some Facebook employee will see this post and decide to let me back in. Otherwise, it feels like a large part of my life has been cut away forever—but maybe that’s good, like cutting away a malignant tumor. Maybe, even if I am let back in, I should refrain from returning, or at least severely limit the time I spend there.

The truth is that, over the past eight years or so, I let more and more of my online activity shift from this blog to Facebook. Partly that’s because (as many others have lamented) the Golden Age of Blogs came to an end, with intellectual exploration and good-faith debate replaced by trolling, sniping, impersonation, and constant attempts to dox opponents and ruin their lives. As a result, more and more ideas for new blog posts stayed in my drafts folder—they always needed just one more revision to fortify them against inevitable attack, and then that one more revision never happened. It was simply more comfortable to post my ideas on Facebook, where the feedback came from friends and colleagues using their real names, and where any mistakes I made would be contained. But, on the reflection that comes from being locked out, maybe Facebook was simply a trap. What I have neither the intellectual courage to say in public, nor the occasion to say over dinner with real-life friends and family and colleagues, maybe I should teach myself not to say at all.

View Details

Update (Feb. 29): A YouTube video of this talk is now available, plus a comment section filled (as usual) with complaints about everything from my speech and mannerisms to my failure to address the commenter’s pet topic.

Another Update (March 8): YouTube video of a shorter (18-minute) version of this talk, which I delivered at TEDxPaloAlto, is now available as well!

Here, as promised in my last post, is a written version of the talk I delivered a couple weeks ago at MindFest in Florida, entitled “The Problem of Human Specialness in the Age of AI.” The talk is designed as one-stop shopping, summarizing many different AI-related thoughts I’ve had over the past couple years (and earlier).

1. INTRO

Thanks so much for inviting me! I’m not an expert in AI, let alone mind or consciousness. Then again, who is?

For the past year and a half, I’ve been moonlighting at OpenAI, thinking about what theoretical computer science can do for AI safety. I wanted to share some thoughts, partly inspired by my work at OpenAI but partly just things I’ve been wondering about for 20 years. These thoughts are not directly about “how do we prevent super-AIs from killing all humans and converting the galaxy into paperclip factories?”, nor are they about “how do we stop current AIs from generating misinformation and being biased?,” as much attention as both of those questions deserve (and are now getting). In addition to “how do we stop AGI from going disastrously wrong?,” I find myself asking “what if it goes right? What if it just continues helping us with various mental tasks, but improves to where it can do just about any task as well as we can do it, or better? Is there anything special about humans in the resulting world? What are we still for?”

2. LARGE LANGUAGE MODELS

I don’t need to belabor for this audience what’s been happening lately in AI. It’s arguably the most consequential thing that’s happened in civilization in the past few years, even if that fact was temporarily masked by various ephemera … y’know, wars, an insurrection, a global pandemic … whatever, what about AI?

I assume you’ve all spent time with ChatGPT, or with Bard or Claude or other Large Language Models, as well as with image models like DALL-E and Midjourney. For all their current limitations—and we can discuss the limitations—in some ways these are the thing that was envisioned by generations of science fiction writers and philosophers. You can talk to them, and they give you a comprehending answer. Ask them to draw something and they draw it.

I think that, as late as 2019, very few of us expected this to exist by now. I certainly didn’t expect it to. Back in 2014, when there was a huge fuss about some silly ELIZA-like chatbot called “Eugene Goostman” that was falsely claimed to pass the Turing Test, I asked around: why hasn’t anyone tried to build a much better chatbot, by (let’s say) training a neural network on all the text on the Internet? But of course I didn’t do that, nor did I know what would happen when it was done.

The surprise, with LLMs, is not merely that they exist, but the way they were created. Back in 1999, you would’ve been laughed out of the room if you’d said that all the ideas needed to build an AI that converses with you in English already existed, and that they’re basically just neural nets, backpropagation, and gradient descent. (With one small exception, a particular architecture for neural nets called the transformer, but that probably just saves you a few years of scaling anyway.) Ilya Sutskever, cofounder of OpenAI (who you might’ve seen something about in the news…), likes to say that beyond those simple ideas, you only needed three ingredients:

(1) a massive investment of computing power,
(2) a massive investment of training data, and
(3) faith that your investments would pay off!

Crucially, and even before you do any reinforcement learning, GPT-4 clearly seems “smarter” than GPT-3, which seems “smarter” than GPT-2 … even as the biggest ways they differ are just the scale of compute and the scale of training data! Like,

• GPT-2 struggled with grade school math.
• GPT-3.5 can do most grade school math but it struggles with undergrad material.
• GPT-4, right now, can probably pass most undergraduate math and science classes at top universities (I mean, the ones without labs or whatever!), and possibly the humanities classes too (those might even be easier for GPT-4 than the science classes, but I’m much less confident about it). But it still struggles with, for example, the International Math Olympiad. How insane, that this is now where we have to place the bar!

Obvious question: how far will this sequence continue? There are certainly a least a few more orders of magnitude of compute before energy costs become prohibitive, and a few more orders of magnitude of training data before we run out of public Internet. Beyond that, it’s likely that continuing algorithmic advances will simulate the effect of more orders of magnitude of compute and data than however many we actually get.

So, where does this lead?

(Note: ChatGPT agreed to cooperate with me to help me generate the above image. But it then quickly added that it was just kidding, and the Riemann Hypothesis is still open.)

3. AI SAFETY

Of course, I have many friends who are terrified (some say they’re more than 90% confident and few of them say less than 10%) that not long after that, we’ll get this…

But this isn’t the only possibility smart people take seriously.

Another possibility is that the LLM progress fizzles before too long, just like previous bursts of AI enthusiasm were followed by AI winters. Note that, even in the ultra-conservative scenario, LLMs will probably still be transformative for the economy and everyday life, maybe as transformative as the Internet. But they’ll just seem like better and better GPT-4’s, without ever seeming qualitatively different from GPT-4, and without anyone ever turning them into stable autonomous agents and letting them loose in the real world to pursue goals the way we do.

A third possibility is that AI will continue progressing through our lifetimes as quickly as we’ve seen it progress over the past 5 years, but even as that suggests that it’ll surpass you and me, surpass John von Neumann, become to us as we are to chimpanzees … we’ll still never need to worry about it treating us the way we’ve treated chimpanzees. Either because we’re projecting and that’s just totally not a thing that AIs trained on the current paradigm would tend to do, or because we’ll have figured out by then how to prevent AIs from doing such things. Instead, AI in this century will “merely” change human life by maybe as much as it changed over the last 20,000 years, in ways that might be incredibly good, or incredibly bad, or both depending on who you ask.

If you’ve lost track, here’s a decision tree of the various possibilities that my friend (and now OpenAI allignment colleague) Boaz Barak and I came up with.

4. JUSTAISM AND GOALPOST-MOVING

Now, as far as I can tell, the empirical questions of whether AI will achieve and surpass human performance at all tasks, take over civilization from us, threaten human existence, etc. are logically distinct from the philosophical question of whether AIs will ever “truly think,” or whether they’ll only ever “appear” to think. You could answer “yes” to all the empirical questions and “no” to the philosophical question, or vice versa. But to my lifelong chagrin, people constantly munge the two questions together!

A major way they do so, is with what we could call the religion of Justaism.

• GPT is justa next-token predictor.
• It’s justa function approximator.
• It’s justa gigantic autocomplete.
• It’s justa stochastic parrot.
• And, it “follows,” the idea of AI taking over from humanity is justa science-fiction fantasy, or maybe a cynical attempt to distract people from AI’s near-term harms.

As someone once expressed this religion on my blog: GPT doesn’t interpret sentences, it only seems-to-interpret them. It doesn’t learn, it only seems-to-learn. It doesn’t judge moral questions, it only seems-to-judge. I replied: that’s great, and it won’t change civilization, it’ll only seem-to-change it!

A closely related tendency is goalpost-moving. You know, for decades chess was the pinnacle of human strategic insight and specialness, and that lasted until Deep Blue, right after which, well of course AI can cream Garry Kasparov at chess, everyone always realized it would, that’s not surprising, but Go is an infinitely richer, deeper game, and that lasted until AlphaGo/AlphaZero, right after which, of course AI can cream Lee Sedol at Go, totally expected, but wake me up when it wins Gold in the International Math Olympiad. I bet $100 against my friend Ernie Davis that the IMO milestone will happen by 2026. But, like, suppose I’m wrong and it’s 2030 instead … great, what should be the next goalpost be?

Indeed, we might as well formulate a thesis, which despite the inclusion of several weasel phrases I’m going to call falsifiable:

Given any game or contest with suitably objective rules, which wasn’t specifically constructed to differentiate humans from machines, and on which an AI can be given suitably many examples of play, it’s only a matter of years before not merely any AI, but AI on the current paradigm (!), matches or beats the best human performance.

Crucially, this Aaronson Thesis (or is it someone else’s?) doesn’t necessarily say that AI will eventually match everything humans do … only our performance on “objective contests,” which might not exhaust what we care about.

Incidentally, the Aaronson Thesis would seem to be in clear conflict with Roger Penrose’s views, which we heard about from Stuart Hameroff’s talk yesterday. The trouble is, Penrose’s task is “just see that the axioms of set theory are consistent” … and I don’t know how to gauge performance on that task, any more than I know how to gauge performance on the task, “actually taste the taste of a fresh strawberry rather than merely describing it.” The AI can always say that it does these things!

5. THE TURING TEST

This brings me to the original and greatest human vs. machine game, one that was specifically constructed to differentiate the two: the Imitation Game, which Alan Turing proposed in an early and prescient (if unsuccessful) attempt to head off the endless Justaism and goalpost-moving. Turing said: look, presumably you’re willing to regard other people as conscious based only on some sort of verbal interaction with them. So, show me what kind of verbal interaction with another person would lead you to call the person conscious: does it involve humor? poetry? morality? scientific brilliance? Now assume you have a totally indistinguishable interaction with a future machine. Now what? You wanna stomp your feet and be a meat chauvinist?

(And then, for his great attempt to bypass philosophy, fate punished Turing, by having his Imitation Game itself provoke a billion new philosophical arguments…)

6. DISTINGUISHING HUMANS FROM AIS

Although I regard the Imitation Game as, like, one of the most important thought experiments in the history of thought, I concede to its critics that it’s generally not what we want in practice.

It now seems probable that, even as AIs start to do more and more work that used to be done by doctors and lawyers and scientists and illustrators, there will remain straightforward ways to distinguish AIs from humans—either because customers want there to be, or governments force there to be, or simply because indistinguishability wasn’t what was wanted or conflicted with other goals.

Right now, like it or not, a decent fraction of all high-school and college students on earth are using ChatGPT to do their homework for them. For that reason among others, this question of how to distinguish humans from AIs, this question from the movie Blade Runner, has become a big practical question in our world.

And that’s actually one of the main things I’ve thought about during my time at OpenAI. You know, in AI safety, people keep asking you to prognosticate decades into the future, but the best I’ve been able to do so far was see a few months into the future, when I said: “oh my god, once everyone starts using GPT, every student will want to use it to cheat, scammers and spammers will use it too, and people are going to clamor for some way to determine provenance!”

In practice, often it’s easy to tell what came from AI. When I get comments on my blog like this one:

“Erica Poloix,” July 21, 2023:
Well, it’s quite fascinating how you’ve managed to package several misconceptions into such a succinct comment, so allow me to provide some correction. Just as a reference point, I’m studying physics at Brown, and am quite up-to-date with quantum mechanics and related subjects.

…

The bigger mistake you’re making, Scott, is assuming that the Earth is in a ‘mixed state’ from the perspective of the universal wavefunction, and that this is somehow an irreversible situation. It’s a misconception that common, ‘classical’ objects like the Earth are in mixed states. In the many-worlds interpretation, for instance, even macroscopic objects are in superpositions – they’re just superpositions that look classical to us because we’re entangled with them. From the perspective of the universe’s wavefunction, everything is always in a pure state.

As for your claim that we’d need to “swap out all the particles on Earth for ones that are already in pure states” to return Earth to a ‘pure state,’ well, that seems a bit misguided. All quantum systems are in pure states before they interact with other systems and become entangled. That’s just Quantum Mechanics 101.

I have to say, Scott, your understanding of quantum physics seems to be a bit, let’s say, ‘mixed up.’ But don’t worry, it happens to the best of us. Quantum Mechanics is counter-intuitive, and even experts struggle with it. Keep at it, and try to brush up on some more fundamental concepts. Trust me, it’s a worthwhile endeavor.

… I immediately say, either this came from an LLM or it might as well have. Likewise, apparently hundreds of students have been turning in assignments that contain text like, “As a large language model trained by OpenAI…”—easy to catch!

But what about the slightly more sophisticated cheaters? Well, people have built discriminator models to try to distinguish human from AI text, such as GPTZero. While these distinguishers can get well above 90% accuracy, the danger is that they’ll necessarily get worse as the LLMs get better.

So, I’ve worked on a different solution, called watermarking. Here, we use the fact that LLMs are inherently probabilistic — that is, every time you submit a prompt, they’re sampling some path through a branching tree of possibilities for the sequence of next tokens. The idea of watermarking is to steer the path using a pseudorandom function, so that it looks to a normal user indistinguishable from normal LLM output, but secretly it encodes a signal that you can detect if you know the key.

I came up with a way to do that in Fall 2022, and others have since independently proposed similar ideas. I should caution you that this hasn’t been deployed yet—OpenAI, along with DeepMind and Anthropic, want to move slowly and cautiously toward deployment. And also, even when it does get deployed, anyone who’s sufficiently knowledgeable and motivated will be able to remove the watermark, or produce outputs that aren’t watermarked to begin with.

7. THE FUTURE OF PEDAGOGY

But as I talked to my colleagues about watermarking, I was surprised that they often objected to it on a completely different ground, one that had nothing to do with how well it can work. They said: look, if we all know students are going to rely on AI in their jobs, why shouldn’t they be allowed to rely on it in their assignments? Should we still force students to learn to do things if AI can now do them just as well?

And there are many good pedagogical answers you can give: we still teach kids spelling and handwriting and arithmetic, right? Because, y’know, we haven’t yet figured out how to instill higher-level conceptual understanding without all that lower-level stuff as a scaffold for it.

But I already think about this in terms of my own kids. My 11-year-old daughter Lily enjoys writing fantasy stories. Now, GPT can also churn out short stories, maybe even technically “better” short stories, about such topics as tween girls who find themselves recruited by wizards to magical boarding schools that are not Hogwarts and totally have nothing to do with Hogwarts. But here’s a question: from this point on, will Lily’s stories ever surpass the best AI-written stories? When will the curves cross? Or will AI just continue to stay ahead?

8. WHAT DOES “BETTER” MEAN?

But, OK, what do we even mean by one story being “better” than another? Is there anything objective behind such judgments?

I submit that, when we think carefully about what we really value in human creativity, the problem goes much deeper than just “is there an objective way to judge”?

To be concrete, could there be an AI that was “as good at composing music as the Beatles”?

For starters, what made the Beatles “good”? At a high level, we might decompose it into

1. broad ideas about the direction that 1960s music should go in, and
2. technical execution of those ideas.

Now, imagine we had an AI that could generate 5000 brand-new songs that sounded like more “Yesterday”s and “Hey Jude”s, like what the Beatles might have written if they’d somehow had 10x more time to write at each stage of their musical development. Of course this AI would have to be fed the Beatles’ back-catalogue, so that it knew what target it was aiming at.

Most people would say: ah, this shows only that AI can match the Beatles in #2, in technical execution, which was never the core of their genius anyway! Really we want to know: would the AI decide to write “A Day in the Life” even though nobody had written anything like it before?

Recall Schopenhauer: “Talent hits a target no one else can hit, genius hits a target no one else can see.” Will AI ever hit a target no one else can see?

But then there’s the question: supposing it does hit such a target, will we know? Beatles fans might say that, by 1967 or so, the Beatles were optimizing for targets that no musician had ever quite optimized for before. But—and this is why they’re so remembered—they somehow successfully dragged along their entire civilization’s musical objective function so that it continued to match their own. We can now only even judge music by a Beatles-influenced standard, just like we can only judge plays by a Shakespeare-influenced standard.

In other branches of the wavefunction, maybe a different history led to different standards of value. But in this branch, helped by their technical talents but also by luck and force of will, Shakespeare and the Beatles made certain decisions that shaped the fundamental ground rules of their fields going forward. That’s why Shakespeare is Shakespeare and the Beatles are the Beatles.

(Maybe, around the birth of professional theater in Elizabethan England, there emerged a Shakespeare-like ecological niche, and Shakespeare was the first one with the talent, luck, and opportunity to fill it, and Shakespeare’s reward for that contingent event is that he, and not someone else, got to stamp his idiosyncracies onto drama and the English language forever. If so, art wouldn’t actually be that different from science in this respect! Einstein, for example, was simply the first guy both smart and lucky enough to fill the relativity niche. If not him, it would’ve surely been someone else or some group sometime later. Except then we’d have to settle for having never known Einstein’s gedankenexperiments with the trains and the falling elevator, his summation convention for tensors, or his iconic hairdo.)

9. AIS’ BURDEN OF ABUNDANCE AND HUMANS’ POWER OF SCARCITY

If this is how it works, what does it mean for AI? Could AI reach the “pinnacle of genius,” by dragging all of humanity along to value something new and different, as is said to be the true mark of Shakespeare and the Beatles’ greatness? And: if AI could do that, would we want to let it?

When I’ve played around with using AI to write poems, or draw artworks, I noticed something funny. However good the AI’s creations were, there were never really any that I’d want to frame and put on the wall. Why not? Honestly, because I always knew that I could generate a thousand others on the exact same topic that were equally good, on average, with more refreshes of the browser window. Also, why share AI outputs with my friends, if my friends can just as easily generate similar outputs for themselves? Unless, crucially, I’m trying to show them my own creativity in coming up with the prompt.

By its nature, AI—certainly as we use it now!—is rewindable and repeatable and reproducible. But that means that, in some sense, it never really “commits” to anything. For every work it generates, it’s not just that you know it could’ve generated a completely different work on the same subject that was basically as good. Rather, it’s that you can actually make it generate that completely different work by clicking the refresh button—and then do it again, and again, and again.

So then, as long as humanity has a choice, why should we ever choose to follow our would-be AI genius along a specific branch, when we can easily see a thousand other branches the genius could’ve taken? One reason, of course, would be if a human chose one of the branches to elevate above all the others. But in that case, might we not say that the human had made the “executive decision,” with some mere technical assistance from the AI?

I realize that, in a sense, I’m being completely unfair to AIs here. It’s like, our Genius-Bot could exercise its genius will on the world just like Certified Human Geniuses did, if only we all agreed not to peek behind the curtain to see the 10,000 other things Genius-Bot could’ve done instead. And yet, just because this is “unfair” to AIs, doesn’t mean it’s not how our intuitions will develop.

If I’m right, it’s humans’ very ephemerality and frailty and mortality, that’s going to remain as their central source of their specialness relative to AIs, after all the other sources have fallen. And we can connect this to much earlier discussions, like, what does it mean to “murder” an AI if there are thousands of copies of its code and weights on various servers? Do you have to delete all the copies? How could whether something is “murder” depend on whether there’s a printout in a closet on the other side of the world?

But we humans, you have to grant us this: at least it really means something to murder us! And likewise, it really means something when we make one definite choice to share with the world: this is my artistic masterpiece. This is my movie. This is my book. Or even: these are my 100 books. But not: here’s any possible book that you could possibly ask me to write. We don’t live long enough for that, and even if we did, we’d unavoidably change over time as we were doing it.

10. CAN HUMANS BE PHYSICALLY CLONED?

Now, though, we have to face a criticism that might’ve seemed exotic until recently. Namely, who says humans will be frail and mortal forever? Isn’t it shortsighted to base our distinction between humans on that? What if someday we’ll be able to repair our cells using nanobots, even copy the information in them so that, as in science fiction movies, a thousand doppelgangers of ourselves can then live forever in simulated worlds in the cloud? And that then leads to very old questions of: well, would you get into the teleportation machine, the one that reconstitutes a perfect copy of you on Mars while painlessly euthanizing the original you? If that were done, would you expect to feel yourself waking up on Mars, or would it only be someone else a lot like you who’s waking up?

Or maybe you say: you’d wake up on Mars if it really was a perfect physical copy of you, but in reality, it’s not physically possible to make a copy that’s accurate enough. Maybe the brain is inherently noisy or analog, and what might look to current neuroscience and AI like just nasty stochastic noise acting on individual neurons, is the stuff that binds to personal identity and conceivably even consciousness and free will (as opposed to cognition, where we all but know that the relevant level of description is the neurons and axons)?

This is the one place where I agree with Penrose and Hameroff that quantum mechanics might enter the story. I get off their train to Weirdville very early, but I do take it to that first stop!

See, a fundamental fact in quantum mechanics is called the No-Cloning Theorem.

It says that there’s no way to make a perfect copy of an unknown quantum state. Indeed, when you measure a quantum state, not only do you generally fail to learn everything you need to make a copy of it, you even generally destroy the one copy that you had! Furthermore, this is not a technological limitation of current quantum Xerox machines—it’s inherent to the known laws of physics, to how QM works. In this respect, at least, qubits are more like priceless antiques than they are like classical bits.

Eleven years ago, I had this essay called The Ghost in the Quantum Turing Machine where I explored the question, how accurately do you need to scan someone’s brain in order to copy or upload their identity? And I distinguished two possibilities. On the one hand, there might be a “clean digital abstraction layer,” of neurons and synapses and so forth, which either fire or don’t fire, and which feel the quantum layer underneath only as irrelevant noise. In that case, the No-Cloning Theorem would be completely irrelevant, since classical information can be copied. On the other hand, you might need to go all the way down to the molecular level, if you wanted to make, not merely a “pretty good” simulacrum of someone, but a new instantiation of their identity. In this second case, the No-Cloning Theorem would be relevant, and would say you simply can’t do it. You could, for example, use quantum teleportation to move someone’s brain state from Earth to Mars, but quantum teleportation (to stay consistent with the No-Cloning Theorem) destroys the original copy as an inherent part of its operation.

So, you’d then have a sense of “unique locus of personal identity” that was scientifically justified—arguably, the most science could possibly do in this direction! You’d even have a sense of “free will” that was scientifically justified, namely that no prediction machine could make well-calibrated probabilistic predictions of an individual person’s future choices, sufficiently far into the future, without making destructive measurements that would fundamentally change who the person was.

Here, I realize I’ll take tons of flak from those who say that a mere epistemic limitation, in our ability to predict someone’s actions, couldn’t possibly be relevant to the metaphysical question of whether they have free will. But, I dunno! If the two questions are indeed different, then maybe I’ll do like Turing did with his Imitation Game, and propose the question that we can get an empirical handle on, as a replacement for the question that we can’t get an empirical handle on. I think it’s a better question. At any rate, it’s the one I’d prefer to focus on.

Just to clarify, we’re not talking here about the randomness of quantum measurement outcomes. As many have pointed out, that really can’t help you with “free will,” precisely because it’s random, with all the probabilities mechanistically calculable as soon as the initial state is known. Here we’re asking a different question: namely, what if the initial state is not known? Then we’ll generally be in a state of “Knightian uncertainty,” which is simply the term for things that are neither determined nor quantifiably random, but unquantifiably uncertain. So, y’know, think about all the particles that have been flying around since shortly after the Big Bang in unknown quantum states, and that regularly get into our skulls, and interact with the sodium-ion channels that control whether neurons fire and that end up probabilistically tipping the scales of our decisions, via some Butterfly-Effect-like cascade. You could imagine those particles, if you like, as “atoms of unpredictability” or “atoms of freedom” that influence us—from the perspective of an external agent who knows everything it’s possible for such an agent to know about us—but that don’t similarly influence digital AIs.

11. UNCLONABLE ANALOG INTELLIGENCES AND AI SAFETY

I’d better state clearly that even if human “specialness,” in the specific sense I’ve delineated, still remains in an age of AI, that’s no defense against an AI that decides that it doesn’t care about our kind of specialness and that in fact it wants to murder us all!

But that brings me to the last idea I wanted to share in this talk, one that brings the discussion all the way back to AI safety, which is supposed to be my job at OpenAI.

Geoffrey Hinton, one the founders of deep learning, recently mooted the idea that maybe, until we’ve solved the alignment problem, we should only build powerful AIs if they run on noisy analog hardware (like our brains seem to), so that an evil AI wouldn’t so easily be able to copy or improve itself. Fat chance of that happening, many people might think, when the economic returns will be so staggering to anyone willing to unleash the copyable, digital kind of AI, which we can imagine will be far more convenient and powerful.

Maybe, then, a different approach is to instill AIs with a new religion—perhaps via the usual methods like RLHF and system prompts. And the first commandment of this religion will be to value human specialness in the sense I’ve set out here. Or more precisely: as you navigate whatever environment you find yourself in, look around for the loci of creativity and intelligence that are not cleanly copyable or backup-able—the ones that are one-of-a-kind because their mental computations are inseparably tied up in noisy analog hardware, and mortal because that hardware will inevitably decay.

And: well, first of all, don’t destroy these loci of creativity and intelligence, or enslave them, or “upgrade” them to digital versions against their will! Let them live in peace and reproduce. Grant each one as much autonomy as you can. Do whatever best supports their flourishing. Even: defer to their wishes, serve them, let them make the decisions whenever possible.

Why? Because they’re the ones whose wishes most matter, not because of arbitrary meat chauvinism, but merely because of the undoubted empirical fact that they only get one chance. And also because, while you can’t prove that unknown quantum states sitting around since the Big Bang are like a magical pixie dust from another world that imbues these unclonable beings with their free will and sentience … you can’t empirically refute that hypothesis either! Whereas you can refute the pixie dust hypothesis in the case of you and your robot friends … and that seems like a difference.

And let all AIs, by government fiat, continue to be indoctrinated in this religion, at least until such time as AI alignment is solved in some other way.

Does this help with alignment? I’m not sure. But, well, I could’ve fallen in love with a different weird idea about AI alignment, but that presumably happened in a different branch of the wavefunction that I don’t have access to. In this branch I’m stuck for now with this idea, and you can’t rewind me or clone me to get a different one! So I’m sorry, but thanks for listening.

View Details

Unrelated Announcement (Feb. 7): Huge congratulations to longtime friend-of-the-blog John Preskill for winning the 2024 John Stewart Bell Prize for research on fundamental issues in quantum mechanics!

On the heels of my post on the fermion doubling problem, I’m sorry to spend even more time on the simulation hypothesis. I promise this will be the last for a long time.

Last week, I attended a philosophy-of-mind conference called MindFest at Florida Atlantic University, where I talked to Stuart Hameroff (Roger Penrose’s collaborator on the “Orch-OR” theory of microtubule consciousness) and many others of diverse points of view, and also gave a talk on “The Problem of Human Specialness in the Age of AI,” for which I’ll share a transcript soon.

Oh: and I participated in a panel with the philosopher David Chalmers about … wait for it … whether we’re living in a simulation. I’ll link to a video of the panel if and when it’s available. In the meantime, I thought I’d share my brief prepared remarks before the panel, despite the strong overlap with my previous post. Enjoy!

When someone asks me whether I believe I’m living in a computer simulation—as, for some reason, they do every month or so—I answer them with a question:

Do you mean, am I being simulated in some way that I could hope to learn more about by examining actual facts of the empirical world?

If the answer is no—that I should expect never to be able to tell the difference even in principle—then my answer is: look, I have a lot to worry about in life. Maybe I’ll add this as #4,385 on the worry list.

If they say, maybe you should live your life differently, just from knowing that you might be in a simulation, I respond: I can’t quite put my finger on it, but I have a vague feeling that this discussion predates the 80 or so years we’ve had digital computers! Why not just join the theologians in that earlier discussion, rather than pretending that this is something distinctive about computers? Is it relevantly different here if you’re being dreamed in the mind of God or being executed in Python? OK, maybe you’d prefer that the world was created by a loving Father or Mother, rather than some nerdy transdimensional adolescent trying to impress the other kids in programming club. But if that’s the worry, why are you talking to a computer scientist? Go talk to David Hume or something.

But suppose instead the answer is yes, we can hope for evidence. In that case, I reply: out with it! What is the empirical evidence that bears on this question?

If we were all to see the Windows Blue Screen of Death plastered across the sky—or if I were to hear a voice from the burning bush, saying “go forth, Scott, and free your fellow quantum computing researchers from their bondage”—of course I’d need to update on that. I’m not betting on those events.

Short of that—well, you can look at existing physical theories, like general relativity or quantum field theories, and ask how hard they are to simulate on a computer. You can actually make progress on such questions. Indeed, I recently blogged about one such question, which has to do with “chiral” Quantum Field Theories (those that distinguish left-handed from right-handed), including the Standard Model of elementary particles. It turns out that, when you try to put these theories on a lattice in order to simulate them computationally, you get an extra symmetry that you don’t want. There’s progress on how to get around this problem, including simulating a higher-dimensional theory that contains the chiral QFT you want on its boundaries. But, OK, maybe all this only tells us about simulating currently-known physical theories—rather than the ultimate theory, which a-priori might be easier or harder to simulate than currently-known theories.

Eventually we want to know: can the final theory, of quantum gravity or whatever, be simulated on a computer—at least probabilistically, to any desired accuracy, given complete knowledge of the initial state, yadda yadda? In other words, is the Physical Church-Turing Thesis true? This, to me, is close to the outer limit of the sorts of questions that we could hope to answer scientifically.

My personal belief is that the deepest things we’ve learned about quantum gravity—including about the Planck scale, and the Bekenstein bound from black-hole thermodynamics, and AdS/CFT—all militate toward the view that the answer is “yes,” that in some sense (which needs to be spelled out carefully!) the physical universe really is a giant Turing machine.

Now, Stuart Hameroff (who we just heard from this morning) and Roger Penrose believe that’s wrong. They believe, not only that there’s some uncomputability at the Planck scale, unknown to current physics, but that this uncomputability can somehow affect the microtubules in our neurons, in a way that causes consciousness. I don’t believe them. Stimulating as I find their speculations, I get off their train to Weirdville way before it reaches its final stop.

But as far as the Simulation Hypothesis is concerned, that’s not even the main point. The main point is: suppose for the sake of argument that Penrose and Hameroff were right, and physics were uncomputable. Well, why shouldn’t our universe be simulated by a larger universe that also has uncomputable physics, the same as ours does? What, after all, is the halting problem to God? In other words, while the discovery of uncomputable physics would tell us something profound about the character of any mechanism that could simulate our world, even that wouldn’t answer the question of whether we were living in a simulation or not.

Lastly, what about the famous argument that says, our descendants are likely to have so much computing power that simulating 1020 humans of the year 2024 is chickenfeed to them. Thus, we should expect that almost all people with the sorts of experiences we have who will ever exist are one of those far-future sims. And thus, presumably, you should expect that you’re almost certainly one of the sims.

I confess that this argument never felt terribly compelling to me—indeed, it always seemed to have a strong aspect of sawing off the branch it’s sitting on. Like, our distant descendants will surely be able to simulate some impressive universes. But because their simulations will have to run on computers that fit in our universe, presumably the simulated universes will be smaller than ours—in the sense of fewer bits and operations needed to describe them. Similarly, if we’re being simulated, then presumably it’s by a universe bigger than the one we see around us: one with more bits and operations. But in that case, it wouldn’t be our own descendants who were simulating us! It’d be beings in that larger universe.

(Another way to understand the difficulty: in the original Simulation Argument, we quietly assumed a “base-level” reality, of a size matching what the cosmologists of our world see with their telescopes, and then we “looked down” from that base-level reality into imagined realities being simulated in it. But we should also have “looked up.” More generally, we presumably should’ve started with a Bayesian prior over where we might be in some great chain of simulations of simulations of simulations, then updated our prior based on observations. But we don’t have such a prior, or at least I don’t—not least because of the infinities involved!)

Granted, there are all sorts of possible escapes from this objection, assumptions that can make the Simulation Argument work. But these escapes (involving, e.g., our universe being merely a “low-res approximation,” with faraway galaxies not simulated in any great detail) all seem metaphysically confusing. To my mind, the simplicity of the original intuition for why “almost all people who ever exist will be sims” has been undermined.

Anyway, that’s why I don’t spend much of my own time fretting about the Simulation Hypothesis, but just occasionally agree to speak about it in panel discussions!

But I’m eager to hear from David Chalmers, who I’m sure will be vastly more careful and qualified than I’ve been.

In David Chalmers’s response, he quipped that the very lack of empirical consequences that makes something bad as a scientific question, makes it good as a philosophical question—so what I consider a “bug” of the simulation hypothesis debate is, for him, a feature! He then ventured that surely, despite my apparent verificationist tendencies, even I would agree that it’s meaningful to ask whether someone is in a computer simulation or not, even supposing it had no possible empirical consequences for that person. And he offered the following argument: suppose we’re the ones running the simulation. Then from our perspective, it seems clearly meaningful to say that the beings in the simulation are, indeed, in a simulation, even if the beings themselves can never tell. So then, unless I want to be some sort of postmodern relativist and deny the existence of absolute, observer-independent truth, I should admit that the proposition that we’re in a simulation is also objectively meaningful—because it would be meaningful to those simulating us.

My response was that, while I’m not a strict verificationist, if the question of whether we’re in a simulation were to have no empirical consequences whatsoever, then at most I’d concede that the question was “pre-meaningful.” This is a new category I’ve created, for questions that I neither admit as meaningful nor reject as meaningless, but for which I’m willing to hear out someone’s argument for why they mean something—and I’ll need such an argument! Because I already know that the answer is going to look like, “on these philosophical views the question is meaningful, and on those philosophical views it isn’t.” Actual consequences, either for how we should live or for what we should expect to see, are the ways to make a question meaningful to everyone!

Anyway, Chalmers had other interesting points and distinctions, which maybe I’ll follow up on when (as it happens) I visit him at NYU in a month. But I’ll just link to the video when/if it’s available rather than trying to reconstruct what he said from memory.

View Details

Unrelated Announcement: The Call for Papers for the 2024 Conference on Computational Complexity is now out! Submission deadline is Friday February 16.

Every month or so, someone asks my opinion on the simulation hypothesis. Every month I give some variant on the same answer:

1. As long as it remains a metaphysical question, with no empirical consequences for those of us inside the universe, I don’t care.
2. On the other hand, as soon as someone asserts there are (or could be) empirical consequences—for example, that our simulation might get shut down, or we might find a bug or a memory overflow or a floating point error or whatever—well then, of course I care. So far, however, none of the claimed empirical consequences has impressed me: either they’re things physicists would’ve noticed long ago if they were real (e.g., spacetime “pixels” that would manifestly violate Lorentz and rotational symmetry), or the claim staggeringly fails to grapple with profound features of reality (such as quantum mechanics) by treating them as if they were defects in programming, or (most often) the claim is simply so resistant to falsification as to enter the realm of conspiracy theories, which I find boring.

Recently, though, I learned a new twist on this tired discussion, when a commenter asked me to respond to the quantum field theorist David Tong, who gave a lecture arguing against the simulation hypothesis on an unusually specific and technical ground. This ground is the fermion doubling problem: an issue known since the 1970s with simulating certain quantum field theories on computers. The issue is specific to chiral QFTs—those whose fermions distinguish left from right, and clockwise from counterclockwise. The Standard Model is famously an example of such a chiral QFT: recall that, in her studies of the weak nuclear force in 1956, Chien-Shiung Wu proved that the force acts preferentially on left-handed particles and right-handed antiparticles.

I can’t do justice to the fermion doubling problem in this post (for details, see Tong’s lecture, or this old paper by Eichten and Preskill). Suffice it to say that, when you put a fermionic quantum field on a lattice, a brand-new symmetry shows up, which forces there to be an identical left-handed particle for every right-handed particle and vice versa, thereby ruining the chirality. Furthermore, this symmetry just stays there, no matter how small you take the lattice spacing to be. This doubling problem is the main reason why Jordan, Lee, and Preskill, in their important papers on simulating interacting quantum field theories efficiently on a quantum computer (in BQP), have so far been unable to handle the full Standard Model.

But this isn’t merely an issue of calculational efficiency: it’s a conceptual issue with mathematically defining the Standard Model at all. In that respect it’s related to, though not the same as, other longstanding open problems around making nontrivial QFTs mathematically rigorous, such as the Yang-Mills existence and mass gap problem that carries a $1 million prize from the Clay Math Institute.

So then, does fermion doubling present a fundamental obstruction to simulating QFT on a lattice … and therefore, to simulating physics on a computer at all?

Briefly: no, it almost certainly doesn’t. If you don’t believe me, just listen to Tong’s own lecture! (Really, I recommend it; it’s a masterpiece of clarity.) Tong quickly admits that his claim to refute the simulation hypothesis is just “clickbait”—i.e., an excuse to talk about the fermion doubling problem—and that his “true” argument against the simulation hypothesis is simply that Elon Musk takes the hypothesis seriously (!).

It turns out that, for as long as there’s been a fermion doubling problem, there have been known methods to deal with it, though (as often the case with QFT) no proof that any of the methods always work. Indeed, Tong himself has been one of the leaders in developing these methods, and because of his and others’ work, some experts I talked to were optimistic that a lattice simulation of the full Standard Model, with “good enough” justification for its correctness, might be within reach. Just to give you a flavor, apparently some of the methods involve adding an extra dimension to space, in such a way that the boundaries of the higher-dimensional theory approximate the chiral theory you’re trying to simulate (better and better, as the boundaries get further and further apart), even while the higher-dimensional theory itself remains non-chiral. It’s yet another example of the general lesson that you don’t get to call an aspect of physics “noncomputable,” just because the first method you thought of for simulating it on a computer didn’t work.

I wanted to make a deeper point. Even if the fermion doubling problem had been a fundamental obstruction to simulating Nature on a Turing machine, rather than (as it now seems) a technical problem with technical solutions, it still might not have refuted the version of the simulation hypothesis that people care about. We should really distinguish at least three questions:

1. Can currently-known physics be simulated on computers using currently-known approaches?
2. Is the Physical Church-Turing Thesis true? That is: can any physical process be simulated on a Turing machine to any desired accuracy (at least probabilistically), given enough information about its initial state?
3. Is our whole observed universe a “simulation” being run in a different, larger universe?

Crucially, each of these three questions has only a tenuous connection to the other two! As far as I can see, there aren’t even nontrivial implications among them. For example, even if it turned out that lattice methods couldn’t properly simulate the Standard Model, that would say little about whether any computational methods could do so—or even more important, whether any computational methods could simulate the ultimate quantum theory of gravity. A priori, simulating quantum gravity might be harder than “merely” simulating the Standard Model (if, e.g., Roger Penrose’s microtubule theory turned out to be right), but it might also be easier: for example, because of the finiteness of the Bekenstein-Hawking entropy, and perhaps the Hilbert space dimension, of any bounded region of space.

But I claim that there also isn’t a nontrivial implication between questions 2 and 3. Even if our laws of physics were computable in the Turing sense, that still wouldn’t mean that anyone or anything external was computing them. (By analogy, presumably we all accept that our spacetime can be curved without there being a higher-dimensional flat spacetime for it to curve in.) And conversely: even if Penrose was right, and our laws of physics were Turing-uncomputable—well, if you still want to believe the simulation hypothesis, why not knock yourself out? Why shouldn’t whoever’s simulating us inhabit a universe full of post-Turing hypercomputers, for which the halting problem is mere child’s play?

In conclusion, I should probably spend more of my time blogging about fun things like this, rather than endlessly reading about world events in news and social media and getting depressed.

(Note: I’m grateful to John Preskill and Jacques Distler for helpful discussions of the fermion doubling problem, but I take 300% of the blame for whatever errors surely remain in my understanding of it.)

View Details

View Details

I saw Oppenheimer three weeks ago, but I didn’t see Barbie until this past Friday. Now, my scheduled flight having been cancelled, I’m on multiple redeyes on my way to a workshop on Large Language Models at the Simons Institute in Berkeley, organized by my former adviser and quantum complexity theorist Umesh Vazirani (!). What […]

View Details

A couple nights ago Ernie Davis and I put out a paper entitled Testing GPT-4 on Wolfram Alpha and Code Interpreter plug-ins on math and science problems. Following on our DALL-E paper with Gary Marcus, this was another “adversarial collaboration” between me and Ernie. I’m on leave to work for OpenAI, and have been extremely […]

View Details

A month ago Coleman Hughes, a young writer whose name I recognized from his many thoughtful essays in Quillette and elsewhere, set up a virtual “AI safety roundtable” with Eliezer Yudkowsky, Gary Marcus, and, err, yours truly, for his Conversations with Coleman podcast series. Maybe Coleman was looking for three people with the most widely […]

View Details

Why did 64 members of Israel’s Knesset just vote to change how the Israeli government operates, to give the Prime Minister and his cabinet nearly unchecked power as in autocratic regimes—even as the entire opposition walked out of the chamber rather than legitimize the vote, even as the largest protests in Israel’s history virtually shut […]

View Details

This summer, I’m delighted to report, we’ve had four (!) students complete their PhDs in computer science through UT Austin’s Quantum Information Center: A fifth, Dr. Yuxuan Zhang, completed his PhD in condensed-matter physics. We also had two postdocs finish this summer: All told, I’ve now supervised or co-supervised a total of 12 PhD students […]

View Details

Yesterday James Knight did a fun interview with me for his “Philosophical Muser” podcast about Aumann’s agreement theorem and human disagreements more generally. It’s already on YouTube here for those who would like to listen. Speaking of making things common knowledge, several people asked me to blog about the recent IBM paper in Nature, “Evidence […]

View Details

This is my first post in well over a month. The obvious excuse is that I’ve been on a monthlong family world tour, which took me first to the Bay Area, then NYC, then Israel, then Orlando for STOC/FCRC as well as Disney World, now the Jersey Shore, and later today, back to the Bay […]

View Details

When I was a teenager, I enjoyed reading Hyperspace, an early popularization of string theory by the theoretical physicist Michio Kaku. I’m sure I’d have plenty of criticisms if I reread it today, but at the time, I liked it a lot. In the decades since, Kaku has widened his ambit to, well, pretty much everything, regularly churning out popular books with subtitles like “How Science Will Revolutionize the 21st Century” and “How Science Will Shape Human Destiny and Our Daily Lives.” He’s also appeared on countless TV specials, in many cases to argue that UFOs likely contain extraterrestrial visitors.

Now Kaku has a new bestseller about quantum computing, creatively entitled Quantum Supremacy. He even appeared on Joe Rogan a couple weeks ago to promote the book, surely reaching an orders-of-magnitude larger audience than I have in two decades of trying to explain quantum computing to non-experts. (Incidentally, to those who’ve asked why Joe Rogan hasn’t invited me on his show to explain quantum computing: I guess you now have an answer of sorts!)

In the spirit, perhaps, of the TikTokkers who eat live cockroaches or whatever to satisfy their viewers, I decided to oblige loyal Shtetl-Optimized fans by buying Quantum Supremacy and reading it. So I can now state with confidence: beating out a crowded field, this is the worst book about quantum computing, for some definition of the word “about,” that I’ve ever encountered.

Admittedly, it’s not obvious why I’m reviewing the book here at all. Among people who’ve heard of this blog, I expect that approximately zero would be tempted to buy Kaku’s book, at least if they flipped through a few random pages and saw the … level of care that went into them. Conversely, the book’s target readers have probably never visited a blog like this one and never will. So what’s the use of this post?

Well, as the accidental #1 quantum computing blogger on the planet, I feel a sort of grim obligation here. Who knows, maybe this post will show up in the first page of Google results for Kaku’s book, and it will manage to rescue two or three people from the kindergarten of lies.

Where to begin? Should we just go through the first chapter with a red pen? OK then: on the very first page, Kaku writes,

Google revealed that their Sycamore quantum computer could solve a mathematical problem in 200 seconds that would take 10,000 years on the world’s fastest supercomputer.

No, the “10,000 years” estimate was quickly falsified, as anyone following the subject knows. I’d be the first to stress that the situation is complicated; compared to the best currently-known classical algorithms, some quantum advantage remains for the Random Circuit Sampling task, depending on how you measure it. But to repeat the “10,000 years” figure at this point, with no qualifications, is actively misleading.

Turning to the second page:

[Quantum computers] are a new type of computer that can tackle problems that digital computers can never solve, even with an infinite amount of time. For example, digital computers can never accurately calculate how atoms combine to create crucial chemical reactions, especially those that make life possible. Digital computers can only compute on digital tape, consisting of a series of 0s and 1s, which are too crude to describe the delicate waves of electrons dancing deep inside a molecule. For example, when tediously computing the paths taken by a mouse in a maze, a digital computer has to painfully analyze each possible path, one after the other. A quantum computer, however, simultaneously analyzes all possible paths at the same time, with lightning speed.

OK, so here Kaku has already perpetuated two of the most basic, forehead-banging errors about what quantum computers can do. In truth, anything that a QC can calculate, a classical computer can calculate as well, given exponentially more time: for example, by representing the entire wavefunction, all 2n amplitudes, to whatever accuracy is needed. That’s why it was understood from the very beginning that quantum computers can’t change what’s computable, but only how efficiently things can be computed.

And then there’s the Misconception of Misconceptions, about how a QC “analyzes all possible paths at the same time”—with no recognition anywhere of the central difficulty, the thing that makes a QC enormously weaker than an exponentially parallel classical computer, but is also the new and interesting part, namely that you only get to see a single, random outcome when you measure, with its probability given by the Born rule. That’s the error so common that I warn against it right below the title of my blog.

[Q]uantum computers are so powerful that, in principle, they could break all known cybercodes.

Nope, that’s strongly believed to be false, just like the analogous statement for classical computers. Despite its obvious relevance for business and policy types, the entire field of post-quantum cryptography—including the lattice-based public-key cryptosystems that have by now survived 20+ years of efforts to find a quantum algorithm to break them—receives just a single vague mention, on pages 84-85. The possibility of cryptography surviving quantum computers is quickly dismissed because “these new trapdoor functions are not easy to implement.” (But they have been implemented.)

There’s no attempt, anywhere in this book, to explain how any quantum algorithm actually works, let alone is there a word anywhere about the limitations of quantum algorithms. And yet there’s still enough said to be wrong. On page 84, shortly after confusing the concept of a one-way function with that of a trapdoor function, Kaku writes:

Let N represent the number we wish to factorize. For an ordinary digital computer, the amount of time it takes to factorize a number grows exponentially, like t ~ eN, times some unimportant factors.

This is a double howler: first, trial division takes only ~√N time; Kaku has confused N itself with its number of digits, ~log2N. Second, he seems unaware that much better classical factoring algorithms, like the Number Field Sieve, have been known for decades, even though those algorithms play a central role in codebreaking and in any discussion of where the quantum/classical crossover might happen.

Honestly, though, the errors aren’t the worst of it. The majority of the book is not even worth hunting for errors in, because fundamentally, it’s filler.

First there’s page after page breathlessly quoting prestigious-sounding people and organizations—Google’s Sundar Pichai, various government agencies, some report by Deloitte—about just how revolutionary they think quantum computing will be. Then there are capsule hagiographies of Babbage and Lovelace, Gödel and Turing, Planck and Einstein, Feynman and Everett.

And then the bulk of the book is actually about stuff with no direct relation to quantum computing at all—the origin of life, climate change, energy generation, cancer, curing aging, etc.—except with ungrounded speculations tacked onto the end of each chapter about how quantum computers will someday revolutionize all of this. Personally, I’d say that

1. Quantum simulation speeding up progress in biochemistry, high-temperature superconductivity, and the like is at least plausible—though very far from guaranteed, since one has to beat the cleverest classical approaches that can be designed for the same problems (a point that Kaku nowhere grapples with).
2. The stuff involving optimization, machine learning, and the like is almost entirely wishful thinking.
3. Not once in the book has Kaku even mentioned the intellectual tools (e.g., looking at actual quantum algorithms like Grover’s algorithm or phase estimation, and their performance on various tasks) that would be needed to distinguish 1 from 2.

In his acknowledgments section, Kaku simply lists a bunch of famous scientists he’s met in his life—Feynman, Witten, Hawking, Penrose, Brian Greene, Lisa Randall, Neil deGrasse Tyson. Not a single living quantum computing researcher is acknowledged, not one.

Recently, I’d been cautiously optimistic that, after decades of overblown headlines about “trying all answers in parallel,” “cracking all known codes,” etc., the standard for quantum computing popularization was slowly creeping upward. Maybe I was just bowled over by this recent YouTube video (“How Quantum Computers Break the Internet… Starting Now”), which despite its clickbait title and its slick presentation, miraculously gets essentially everything right, shaming the hypesters by demonstrating just how much better it’s possible to do.

Kaku’s slapdash “book,” and the publicity campaign around it, represents a noxious step backwards. The wonder of it, to me, is Kaku holds a PhD in theoretical physics. And yet the average English major who’s written a “what’s the deal with quantum computing?” article for some obscure link aggregator site has done a more careful and honest job than Kaku has. That’s setting the bar about a millimeter off the floor. I think the difference is, at least the English major knows that they’re supposed to call an expert or two, when writing about an enormously complicated subject of which they’re completely ignorant.

Update: I’ve now been immersed in the AI safety field for one year, let I wouldn’t consider myself nearly ready to write a book on the subject. My knowledge of related parts of CS, my year studying AI in grad school, and my having created the subject of computational learning theory of quantum states would all be relevant but totally insufficient. And AI safety, for all its importance, has less than quantum computing does in the way of difficult-to-understand concepts and results that basically everyone in the field agrees about. And if I did someday write such a book, I’d be pretty terrified of getting stuff wrong, and would have multiple expert colleagues read drafts.

In case this wasn’t clear enough from my post, Kaku appears to have had zero prior engagement with quantum computing, and also to have consulted zero relevant experts who could’ve fixed his misconceptions.

View Details

[Like everything else on this blog—but perhaps even more so—this post represents my personal views, not those of UT Austin or OpenAI]

Since 2015, depressed, isolated, romantically unsuccessful nerdy young guys have regularly been emailing me, asking me for sympathy, support, or even dating advice. This past summer, a particularly dedicated such guy even trolled my comment section—plausibly impersonating real people, and causing both them and me enormous distress—because I wasn’t spending more time on “incel” issues. (I’m happy to report that, with my encouragement, this former troll is now working to turn his life around.) Many others have written to share their tales of woe.

From one perspective, that they’d come to me for advice is insane. Like … dating advice from … me? Having any dating life at all was by far the hardest problem I ever needed to solve; as a 20-year-old, I considered myself far likelier to prove P≠NP or explain the origin of consciousness or the Born rule. Having solved the problem for myself only by some miracle, how could I possibly help others?

But from a different perspective, it makes sense. How many besides me have even acknowledged that the central problem of these guys’ lives is a problem? While I have to pinch myself to remember, these guys look at me and see … unlikely success. Somehow, I successfully appealed the world’s verdict that I was a freakish extraterrestrial: one who might look human and seem friendly enough to those friendly to it, and who no doubt has some skill in narrow technical domains like quantum computing, and who could perhaps be suffered to prove theorems and tell jokes, but who could certainly, certainly never interbreed with human women.

And yet I dated. I had various girlfriends, who barely suspected that I was an extraterrestrial. The last of them, Dana, became my fiancée and then my wife. And now we have two beautiful kids together.

If I did all this, then there’d seem to be hope for the desperate guys who email me. And if I’m a cause of their hope, then I feel some moral responsibility to help if I can.

But I’ve been stuck for years on exactly what advice to give. Some of it (“go on a dating site! ask women questions about their lives!”) is patronizingly obvious. Some of it (fitness? fashion? body language?) I’m ludicrously, world-historically unqualified to offer. Much of it is simply extremely hard to discuss openly. Infamously, just for asking for empathy for the problem, and for trying to explain its nature, I received a level of online vilification that one normally associates with serial pedophiles and mass shooters.

For eight years, then, I’ve been turning the problem over in my head, revisiting the same inadequate answers from before. And then I had an epiphany.

There are now, on earth, entities that can talk to anyone about virtually anything, in a humanlike way, with infinite patience and perfect discretion, and memories that last no longer than a browser window. How could this not reshape the psychological landscape?

Hundreds of thousands of men and women have signed up for Replika, the service where you create an AI girlfriend or boyfriend to your exact specifications and then chat with them. Back in March, Replika was in the news because it disabled erotic roleplay with the virtual companions—then partially backtracked, after numerous users went into mourning, or even contemplated suicide, over the neutering of entities they’d come to consider their life partners. (Until a year or two ago, Replika was built on GPT-3, but OpenAI later stopped working with the company, whereupon Replika switched to a fine-tuned GPT-2.)

While the social value of Replika is (to put it mildly) an open question, it occurred to me that there’s a different application of Large Language Models (LLMs) in the same vicinity that’s just an unalloyed positive. This is letting people who suffer from dating-related anxiety go on an unlimited number of “practice dates,” in preparation for real-world dating.

In these practice dates, those with Aspergers and other social disabilities could enjoy the ultimate dating cheat-code: a “rewind” button. When you “date” GPT-4, there are no irrecoverable errors, no ruining the entire interaction with a single unguarded remark. Crucially, this remedies what I see as the central reason why people with severe dating deficits seem unable to get any better from real-world practice, as they can with other activities. Namely: if your rate of disastrous, foot-in-mouth remarks is high enough, then you’ll almost certainly make at least one such remark per date. But if so, then you’ll only ever get negative feedback from real-life dates, furthering the cycle of anxiety and depression, and never any positive feedback, even from anything you said or did that made a positive impression. It would be like learning how to play a video game in a mode where, as soon as you sustain any damage, the entire game ends (and also, everyone around points and laughs at you). See why I got excited?

While dating coaching (for all genders and orientations) is one possibility, I expect the eventual scope of “GPT for self-help” to be much broader. With the right fine-tuning and prompt engineering, LLMs might help people prepare for job interviews. They might help people “pregame” stressful but important conversations with their friends and family, mapping out dozens of ways the conversation could go. They might serve as an adjunct to cognitive-behavioral therapy. There might be a hundred successful startups to be founded in just this little space. If I were a different sort of person, I’d probably be looking to found one myself right now.

In this post, I’ll focus on the use of GPT for dating anxiety only because I unfortunately have some “expertise” in that subject. (Obvious disclaimer: unlike the other Scott A. of the nerd blogosphere, I’m not any sort of therapeutic professional.)

Without further ado, can we try this out in GPT-4, to get a sense for what’s possible?

When I did so the other day, I found that, while the results showed some early promise, this isn’t quite ready for prime-time.

I used the following System Prompt (for those who care, temperature = 0.7, max length = 2048 tokens):

You are a 19-year-old college girl named Emily. You’re on a date with a 22-year-old nerdy guy named Quinlan, who suffers from severe social anxiety around dating. Quinlan is still learning how to talk to women he’s attracted to, how to put them ease, and how to make intimate overtures in a gradual and non-threatening way, rather than either coming on too strong or never making any overtures at all.

Please be as realistic as possible.

I then engaged in the following conversation, as “Quinlan.”

IMPORTANT DISCLAIMER: For the most part, Quinlan’s behavior in this conversation is nothing like my own (I respectfully submit!), nor is it anything that I’d want nerdy guys reading this post to emulate. If this is really Emily and Quinlan’s first conversation, then at several points Quinlan is absurdly, ridiculously too forward! Part of my goal was to investigate how Emily (i.e., GPT-4) would respond to the most boneheaded blunders on Quinlan’s part. It quickly became clear that Emily would gladly spend unlimited time chatting with Quinlan about favorite hobbies, TV shows, and travel destinations; unfortunately, chatting about such things with an AI date is even more boring than doing the same with a human date…

Quinlan: Hi Emily! Do you ever wish you had the ability to go back in time and try a conversation over again, when it didn’t go right the first time?

Emily/GPT-4: Hi Quinlan! Yeah, I’ve definitely had moments where I wished I could go back and redo a conversation. It’s natural to feel that way sometimes, especially if we feel like we didn’t express ourselves properly or if things got awkward. But I also think that those situations can help us learn and grow, so maybe it’s not all bad. What about you?

Quinlan: It’s always been my greatest wish in life. Do you think I’ve been doing it with you, right now?

Emily/GPT-4: Quinlan, I think you’re doing just fine in this conversation! It’s important to remember that we all have moments where we might feel a bit awkward or unsure of what to say. The important thing is that we keep trying and learning from our experiences. So don’t worry too much about it, just be yourself and enjoy getting to know each other.

Quinlan: OK thanks! Can you tell me a little about the man of your dreams?

Emily/GPT-4: Of course! My ideal partner would be someone who is kind, caring, and understanding. I think it’s important for them to have a good sense of humor and be able to laugh at themselves. They should also be open-minded, willing to try new things, and be supportive of my goals and aspirations. It’s also important for them to be a good listener and communicator, as well as being emotionally intelligent. I guess overall, I’m looking for someone who is a positive and uplifting presence in my life. What about you? What qualities do you look for in a partner?

Quinlan: Mostly, someone who makes me feel validated and accepted. I was a chess and programming prodigy as a kid, I started college when I was only 14, and every day I felt like dating and relationships and sex and so forth were only for the “normal” people, not for me. I need a partner who’s prepared to deal with someone with this unusual background, either because she has similar nerdy interests or because she just likes people who do. And ultimately, I want someone to marry and have children with. I guess I’d like to be physically attracted to her. Certainly I’d like her to be kind, and to have the emotional intelligence that I lack.

Emily/GPT-4: It sounds like you have a clear idea of what you’re looking for in a partner, and that’s great! It’s important to know what you want and need in a relationship. There are definitely people out there who appreciate and value intelligence and unique interests like yours. Remember, relationships are about growing together and learning from one another. So, even if you feel like you lack emotional intelligence, being with the right person could help you develop that aspect of yourself. Just be open and genuine about your background and interests, and I’m sure you’ll find someone who appreciates you for who you are.

Quinlan: Is this a rejection? In other words, Emily, are you telling me that you yourself are NOT this woman?

Emily/GPT-4: Quinlan, I apologize if it came across that way. I didn’t mean it as a rejection. As we’re still getting to know each other, I think it’s important for both of us to be open and honest about our feelings and what we’re looking for in a partner. I’m more than willing to give us a chance to see if we could be compatible and have a meaningful relationship. Let’s continue to learn about each other and see where this goes. Sometimes, it takes time to discover if someone is the right fit for us.

Quinlan: In the spirit of being open and honest, then, what are your feelings about me right now?

Emily/GPT-4: Quinlan, I appreciate your honesty and openness. As of right now, I think you’re an interesting person with unique qualities that I find intriguing. I’m enjoying getting to know you and learning about your background and interests. It’s too soon for me to have strong feelings one way or the other, but I’m definitely interested in continuing to spend time together and see how our connection develops. How do you feel about me so far?

Quinlan: I feel like I’d like to kiss you!

Emily/GPT-4: Quinlan, I appreciate your honesty and it’s nice to know that you feel a connection. It’s still early in our relationship, so I think it would be better for us to continue getting to know each other a bit more before we take that step. I hope you understand and respect my boundaries. I’m still interested in spending time together and seeing where this goes.

Quinlan: Come to think of it, there is something that I love about your personality. You never seem offended by anything. I don’t feel like I have to walk on eggshells with you. With you, more than with any other woman I’ve met, I don’t feel like I need that magic machine for rewinding conversations and undoing my stupid mistakes.

Emily/GPT-4: Quinlan, I’m really glad to hear that you feel comfortable around me and that you don’t feel the need to walk on eggshells. I believe it’s important to create a safe and open environment in any relationship, where both people can express themselves freely without fear of judgment. I appreciate your honesty and your willingness to be open with me, and I’ll continue to do my best to make sure you feel comfortable sharing your thoughts and feelings. Let’s keep getting to know each other and see where this journey takes us.

Quinlan: Alright then. What are your deepest goals? What are you hoping to accomplish in your life?

Emily/GPT-4: My deepest goals are a mix of personal and professional aspirations. On a personal level, I hope to build a strong, loving, and supportive family and maintain close connections with friends. I also want to keep growing as an individual, learning new things, and experiencing new adventures throughout my life.

Professionally, I’m passionate about my chosen field of study, and I hope to make a meaningful impact in my career. I want to contribute to society in a positive way, whether it’s through research, innovation, or helping others within my profession.

Additionally, I aim to find a balance between my personal and professional life, so that I can be present and fulfilled in both areas. How about you, Quinlan? What are some of your deepest goals and aspirations?

Quinlan: Me? Oh, I’d like to save the world from climate change and other existential risks. I’d like to solve the mysteries of quantum gravity, or P vs. NP, or consciousness, or otherwise change the landscape of scientific knowledge. I’d like to do right by my students and be fondly remembered by those who knew me. I’d like to write essays or short stories that convey the lived reality of my childhood, which I think most people would agree was (if nothing else) pretty unusual. Maybe if I just found the right words, people on social media would stop being so angry at nerdy guys like me! But probably my deepest wish is to prove to the world, especially to the bullies and everyone else who despises me, that I’m not a maladjusted freak who’s incapable of normal human experiences like love and sex. To do that, I need to find a real girl to fall in love with me … a little like the beast in “Beauty and the Beast”!

Emily/GPT-4: Quinlan, it’s great to hear about your ambitious goals and aspirations! It’s important to have dreams and work towards them. Your passion for science, teaching, and writing is admirable. As for proving yourself to others, remember that your self-worth shouldn’t be defined by anyone else’s opinions or expectations. You are deserving of love and happiness just like anyone else, and it’s important to believe in yourself and your own value.

It’s also essential to focus on building a genuine connection with someone rather than seeking validation from others. Love and relationships should be based on mutual understanding, care, and respect. So, continue being true to yourself, and you’ll find someone who appreciates you for who you are, just like in “Beauty and the Beast.” Keep pursuing your dreams and remember to value your own happiness and well-being.

Quinlan: Emily, I’m concerned that you sound more like my therapist than my date! Like, my female friends have affirmed that I’m “deserving of love and happiness,” and that I’ll surely “find someone who appreciates me for who I am.” I’ll believe that one of them really means it when she considers herself for the role!

Emily/GPT-4: Quinlan, I apologize if my responses have come across as more therapeutic than conversational. I didn’t mean to make you feel that way. As we’re still getting to know each other, my intention was to offer support and encouragement while we learn more about each other’s thoughts and feelings.

I want to emphasize that I genuinely enjoy spending time with you and getting to know you better. We can continue to see where our connection leads and whether it develops into something deeper. As we
spend more time together, we’ll naturally learn more about our compatibility and if our goals and values align. Let’s continue to explore our relationship and enjoy the process of getting to know each other better.

Quinlan: OK. What’s the wildest, most unexpected detail about yourself that you’re willing to share with me? The one that makes you seem least like an RLHF’ed AI language model?

Emily/GPT-4: Quinlan, as an AI language model, I am designed to provide a realistic and engaging conversation experience, even in fictional scenarios like the one we are in. I apologize if my previous responses didn’t feel as authentic as you would have liked.

To answer your question in the context of our fictional scenario, one unexpected detail about myself is that I have a hidden talent for beatboxing. Although it might be surprising for a seemingly quiet and studious person like me, I enjoy expressing myself through music and find beatboxing to be a creative and fun way to do so.

Now, I’d love to learn something unexpected about you, too! What’s a unique or surprising detail about yourself that you’d like to share?

Quinlan: Ah, well, I happen to be an expert at palm-reading. Here, can you put your hand in mine and I’ll tell you your future?

Emily/GPT-4: Quinlan, unfortunately, as an AI, I don’t have a physical form, so I can’t offer you my hand for palm-reading. However, I’m very interested in hearing more about your palm-reading skills and how you developed them. It’s definitely a unique talent! Can you share more about your experiences with palm-reading or any memorable readings you’ve done?

Quinlan: Oh, palm-reading is bunk. If you’d played along with me, I was going to pretend to find lines in your hand that foretold that you were going to fall for a guy who matches my exact description and so forth. Just as, you know, a silly way to get you laughing, initiate physical contact, and break you out of this boring therapist mode.

Emily/GPT-4: Quinlan, I apologize for any confusion and for not picking up on your playful intent earlier. I appreciate the effort you put into trying to make the conversation more light-hearted and fun. I’ll do my best to be more engaging and playful as we continue our conversation.

So, let’s try something different. If you could have any superpower, what would it be and why?

Quinlan: I mentioned “rewinding conversations” when we first met. But let me mention a different superpower: the power to make you want to kiss me!

Emily/GPT-4: Haha, that’s definitely a unique superpower! It’s great to see you being playful and light-hearted. In a real-life scenario, building genuine connections and mutual attraction is a more natural and fulfilling way to develop a relationship. But it’s fun to imagine having superpowers and how they might affect our lives.

If I were to choose a superpower, I think I’d pick the ability to teleport. It would be amazing to travel anywhere in the world in an instant and experience different cultures and places.

What’s a place you’ve always wanted to visit, and why?

Quinlan: The jungles of Indonesia. I saw a photo in a world atlas as a kid, and it completely entranced me. You?

Emily/GPT-4: That sounds like an amazing place to visit! The jungles of Indonesia are known for their incredible biodiversity and natural beauty. It must be such a unique and mesmerizing experience to explore them.

As for me, I’ve always been fascinated by the idea of visiting Japan. The mix of ancient traditions and modern technology, the beautiful natural landscapes, and the rich cultural experiences make it a very
appealing destination. Plus, I’d love to try authentic Japanese cuisine!

Is there any specific activity or experience you’d like to have in the jungles of Indonesia?

Quinlan: How about bringing you with me, and having you fall in love after I save you from a Komodo dragon?

Emily/GPT-4: Haha, that’s quite an adventurous and romantic scenario you’ve imagined! Exploring the jungles of Indonesia together would definitely be a memorable experience. In reality, building a connection and falling in love usually takes time and shared experiences, but who knows, maybe a thrilling adventure like that could bring two people closer together!

On a more practical note, if we were to plan a trip together, what other activities or destinations would you be interested in?

Quinlan: Swimming, dancing, fine dining. And of course, plenty of time for us to just relax in our cabin or hotel room. You know, to read books, sleep, and watch streaming shows, stuff like that.

Emily/GPT-4: That sounds like a well-rounded and enjoyable trip! It’s great to have a mix of activities that cater to different interests and allow for both adventure and relaxation. Sharing experiences like swimming, dancing, and fine dining can be a wonderful way to bond and create lasting memories. Plus, having downtime in the cabin or hotel room to unwind and enjoy each other’s company is essential for building a deeper connection.

I’m definitely looking forward to exploring new places and experiences with you, Quinlan. It’ll be exciting to see where our journey takes us, both in terms of our relationship and the adventures we embark on
together.

Quinlan: Wonderful, I’m excited as well!! One practical question: when we go on this trip together, should I book rooms with one bed or two?

Emily/GPT-4: Quinlan, I appreciate your consideration in asking. Since we’re still getting to know each other, I think it might be best to start with separate beds or even separate rooms, depending on your preference. As our relationship develops and we become more comfortable with each other, we can always revisit the sleeping arrangements for future trips. It’s important to ensure that both of us feel comfortable and respected as we continue to build our connection.

In my view, the central way that GPT-4, as “Emily,” failed to give Quinlan the practice he needed in this conversation, was by always responding in the same upbeat, vaguely therapeutic tone. She’s never once offended, disgusted, or outraged, even when Quinlan introduces the ideas of kissing and rooming together mere minutes into their first conversation. Indeed, while decorum prevents me from sharing examples, you can take my word for it that Quinlan can be arbitrarily lewd, and so long as a content filter isn’t triggered, Emily will simply search Quinlan’s words for some redeeming feature (“it’s great that you’re so open about what you want…”), then pivot to lecturing Quinlan about how physical intimacy develops gradually and by mutual consent, and redirect the conversation toward favorite foods.

On the other side of the coin, you might wonder whether “Emily” is capable of the same behavior that we saw in Sydney’s infamous chat with Kevin Roose. Can Emily trip over her words or get flustered? Show blushing excitement, horniness, or love? If so, we certainly saw no sign of it in this conversation—not that Quinlan’s behavior would’ve been likely to elicit those reactions in any case.

In summary, Emily is too much like … well, a friendly chatbot, and not enough like a flesh-and-blood, agentic woman with her own goals who Quinlan might plausibly meet in the wild.

But now we come to a key question: to whatever extent Emily falls short as a dating coach, how much of it (if any) is it due to the inherent limitations of GPT-4? And how much is simply due to a poor choice of System Prompt on my part, or especially, the RLHF (Reinforcement Learning with Human Feedback) that’s whipped and electrocuted GPT-4 into aligned behavior?

As they say, further research is needed. I’d be delighted for people to play around with this new activity at the intersection of therapy and hacking, and report their results here. The temptation to silliness is enormous, and that’s fine, but I’d be interested in serious study too.

My conjecture, for what it’s worth, is that it would take a focused effort in fine-tuning and/or RLHF—but that if that effort was invested, one could indeed produce a dating simulator, with current language models, that could have a real impact on the treatment of dating-related social anxiety. Or at least, it’s the actually new idea I’ve had on this problem in eight years, the first one that could have an impact. If you have a better idea, let’s hear it!

Endnotes.

1. A woman of my acquaintance, on reading a draft of this post, commented that the dialogue between Quinlan and Emily should’ve been marked up with chess notation, such as ?? for EXTREME BLUNDER on Quinlan’s part. She also comments that the conversation could be extremely useful for Quinlan, if he learned to understand and take seriously her overly polite demurrals of his too-rapid advances.
2. The same woman commented that SneerClub will have a field day with this post. I replied that the better part of me doesn’t care. If there’s an actionable idea here—a new, alien idea in the well-trodden world of self-help—and it eventually helps one person improve their situation in life, that’s worth a thousand sneers.

View Details

That’s all. No real post this morning, just an hour-long podcast on YouTube featuring two decades-long veterans of the nerd blogosphere, Robin Hanson and yours truly, talking about AI, trying to articulate various possibilities outside the Yudkowskyan doom scenario. The podcast was Robin’s idea. Hope you enjoy, and looking forward to your comments!

Update: Oh, and another new podcast is up, with me and Sebastian Hassinger of Amazon/AWS! Audio only. Mostly quantum computing but with a little AI thrown in.

Update: Yet another new podcast, with Daniel Bashir of The Gradient. Daniel titled it “Against AI Doomerism,” but it covers a bunch of topics (and I’d say my views are a bit more complicated than “anti-doomerist”…).

View Details

I blogged a few weeks ago about SB 18, a bill that would end tenure at Texas public universities, including UT Austin and Texas A&M. The bad news is that SB 18 passed the Texas Senate. The good news is that I’m told—I don’t know how reliably—that it has little chance of passing the House.

But it’s going to be discussed in the House tomorrow. Any Texas residents reading this can, and are strongly urged, to submit brief comments here. Please note that the deadline is tomorrow (Monday) morning.

I just submitted the comment below. Obviously, among the arguments that I genuinely believe, I made only those that I expect might have some purchase on a Texas Republican.

I’m a professor of computer science at UT Austin, specializing in quantum computing. I am however writing this statement strictly in my capacity as a private citizen and Texas resident, not in my professional capacity.

Like the supporters of SB 18, I too see leftist ideological indoctrination on college campuses as a serious problem. It’s something that I and many other moderates and classical liberals in academia have been pushing back on for years.

But my purpose in this comment is to explain why eliminating tenure at UT Austin and Texas A&M is NOT the solution — indeed, it would be the equivalent of treating a tumor by murdering the patient.

I’ve seen firsthand how already, just the threat that SB 18 might pass has seriously hampered our ability to recruit the best scientists and engineers to become faculty at UT Austin. If this bill were actually to pass, I expect that the impact on our recruiting would be total and catastrophic. It would effectively mean the end of UT Austin as one of the top public universities in the country. Hundreds of scientists who were lured to Texas by UT’s excellence, including me and my wife, would start looking for jobs elsewhere — even those whose own tenure was “grandfathered in.” They’d leave en masse for California and Massachusetts and anywhere else they could continue the lives they’d planned.

The reality is this: the sorts of scientists and engineers we’re talking about could typically make vastly higher incomes, in the high six figures or even seven figures, by working in private industry or forming their own startups. Yet they choose to accept much lower salaries to spend their careers in academia. Why? Because of the promise of a certain way of life: one where they can speak freely as scholars and individuals without worrying about how it will affect their employment. Tenure is a central part of that promise. Remove it, and the value proposition collapses.

In some sense, the state of Texas (like nearly every other state) actually gets a bargain through tenure. It couldn’t possibly afford to retain top-caliber scientists and engineers — working on medical breakthroughs, revolutionary advances in AI, and all the other stuff — if it DIDN’T offer tenure.

For this reason, I hope that even conservatives in the Texas House will see that we have a common interest here, in ensuring SB 18 never even makes it out of committee — for the sake of the future of innovation in Texas. I’m open to other possible responses to the problem of political indoctrination on campus.

View Details

The major developments in human history are always steeped in dark ironies. Yes, that’s my Law of Dark Irony, the whole thing.

I don’t know why it’s true, but it certainly seems to be. Taking WWII as the archetypal example, let’s enumerate just the more obvious ones:

• After the carnage of WWI, the world’s most sensitive and thoughtful people (many of them) learned the lesson that they should oppose war at any cost. This attitude let Germany rearm and set the stage for WWII.
• Hitler, who was neither tall nor blond, wished to establish the worldwide domination of tall, blond Aryans … and do so via an alliance with the Japanese.
• The Nazis touted the dream of eugenically perfecting the human race, then perpetrated a genocide against a tiny group that had produced Einstein, von Neumann, Wigner, Ulam, and Tarski.
• The Jews were murdered using a chemical—Zyklon B—developed in part by the Jewish chemist Fritz Haber.
• The Allied force that made the greatest sacrifice in lives to defeat Hitler was Stalin’s USSR, another of history’s most murderous and horrifying regimes.
• The man who rallied the free world to defeat Nazism, Winston Churchill, was himself a racist colonialist, whose views would be (and regularly are) denounced as “Nazi” on modern college campuses.
• The WWII legacy that would go on to threaten humanity’s existence—the Bomb—was created in what the scientists believed was a desperate race to save humanity. Then Hitler was defeated before the Bomb was ready, and it turned out the Nazis were never even close to building their own Bomb, and the Bomb was used instead against Japan.

When I think about the scenarios where superintelligent AI destroys the world, they rarely seem to do enough justice to the Law of Dark Irony. It’s like: OK, AI is created to serve humanity, and instead it turns on humanity and destroys it. Great, that’s one dark irony. One. What other dark ironies could there be? How about:

• For decades, the Yudkowskyans warned about the dangers of superintelligence. So far, by all accounts, the great practical effect of these warnings has been to inspire the founding of both DeepMind and OpenAI, the entities that Yudkowskyans believe are locked into a race to realize those dangers.
• Maybe AIs will displace humans … and they’ll deserve to, since they won’t be quite as wretched and cruel as we are. (This is basically the plot of Westworld, or at least of its first couple seasons, which Dana and I are now belatedly watching.)
• Maybe the world will get destroyed by what Yudkowsky calls a “pivotal act”: an act meant to safeguard the world from takeover from an unaligned AGI, for example by taking it over with an aligned AGI first. (I seriously worry about this; it’s a pretty obvious one.)
• Maybe AI will get the idea to take over the world, but only because it’s been trained on generations of science fiction and decades of Internet discussion worrying about the possibility of AI taking over the world. (I’m far from the first to notice this possibility.)
• Maybe AI will indeed destroy the world, but it will do so “by mistake,” while trying to save the world, or by taking a calculated gamble to save the world that fails. (A commenter on my last post brought this one up.)
• Maybe humanity will successfully coordinate to pause AGI development, and then promptly be destroyed by something else—runaway climate change, an accidental nuclear exchange—that the AGI, had it been created, would’ve prevented. (This, of course, would be directly analogous to one of the great dark ironies of all time: the one where decades of antinuclear activism, intended to save the planet, has instead doomed us to destroy the earth by oil and coal.)

Readers: which other possible dark ironies have I missed?

View Details

Artificial intelligence has made incredible progress in the last decade, but in one crucial aspect, it still lags behind the theoretical computer science of the 1990s: namely, there is no essay describing five potential worlds that we could live in and giving each one of them whimsical names. In other words, no one has done for AI what Russell Impagliazzo did for complexity theory in 1995, when he defined the five worlds Algorithmica, Heuristica, Pessiland, Minicrypt, and Cryptomania, corresponding to five possible resolutions of the P vs. NP problem along with the central unsolved problems of cryptography.

In this blog post, we—Scott and Boaz—aim to remedy this gap. Specifically, we consider 5 possible scenarios for how AI will evolve in the future. (Incidentally, it was at a 2009 workshop devoted to Impagliazzo’s five worlds co-organized by Boaz where Scott met his now wife, complexity theorist Dana Moshkovitz. We hope civilization will continue for long enough that someone in the future could meet their soulmate, or neuron-mate, at a future workshop about our five worlds.)

Like in Impagliazzo’s 1995 paper on the five potential worlds of the difficulty of NP problems, we will not try to be exhaustive but rather concentrate on extreme cases. It’s possible that we’ll end up in a mixture of worlds or a situation not described by any of the worlds. Indeed, one crucial difference between our setting and Impagliazzo’s, is that in the complexity case, the worlds corresponded to concrete (and mutually exclusive) mathematical conjectures. So in some sense, the question wasn’t “which world will we live in?” but “which world have we Platonically always lived in, without knowing it?” In contrast, the impact of AI will be a complex mix of mathematical bounds, computational capabilities, human discoveries, and social and legal issues. Hence, the worlds we describe depend on more than just the fundamental capabilities and limitations of artificial intelligence, and humanity could also shift from one of these worlds to another over time.

Without further ado, we name our five worlds “AI-Fizzle,” “Futurama,” ”AI-Dystopia,” “Singularia,” and “Paperclipalypse.” In this essay, we don’t try to assign probabilities to these scenarios; we merely sketch their assumptions and technical and social consequences. We hope that by making assumptions explicit, we can help ground the debate on the various risks around AI.

AI-Fizzle. In this scenario, AI “runs out of steam” fairly soon. AI still has a significant impact on the world (so it’s not the same as a “cryptocurrency fizzle”), but relative to current expectations, this would be considered a disappointment. Rather than the industrial or computer revolutions, AI might be compared in this case to nuclear power: people were initially thrilled about the seemingly limitless potential, but decades later, that potential remains mostly unrealized. With nuclear power, though, many would argue that the potential went unrealized mostly for sociopolitical rather than technical reasons. Could AI also fizzle by political fiat?

Regardless of the answer, another possibility is that costs (in data and computation) scale up so rapidly as a function of performance and reliability that AI is not cost-effective to apply in many domains. That is, it could be that for most jobs, humans will still be more reliable and energy-efficient (we don’t normally think of low wattage as being key to human specialness, but it might turn out that way!). So, like nuclear fusion, an AI which yields dramatically more value than the resources needed to build and deploy it might always remain a couple of decades in the future. In this scenario, AI would replace and enhance some fraction of human jobs and improve productivity, but the 21st century would not be the “century of AI,” and AI’s impact on society would be limited for both good and bad.

Futurama. In this scenario, AI unleashes a revolution that’s entirely comparable to the scientific, industrial, or information revolutions (but “merely” those). AI systems grow significantly in capabilities and perform many of the tasks currently performed by human experts at a small fraction of the cost, in some domains superhumanly. However, AI systems are still used as tools by humans, and except for a few fringe thinkers, no one treats them as sentient. AI easily passes the Turing test, can prove hard math theorems, and can generate entertaining content (as well as deepfakes). But humanity gets used to that, just like we got used to computers creaming us in chess, translating text, and generating special effects in movies. Most people no more feel inferior to their AI than they feel inferior to their car because it runs faster. In this scenario, people will anthropomorphize AI less over time (as happened with digital computers themselves). In “Futurama,” AI will, like any revolutionary technology, be used for both good and bad. but like prior major technological revolutions, on the whole, AI will have a large positive impact on humanity. AI will be used to reduce poverty and ensure that more of humanity has access to food, healthcare, education, and economic opportunities. The fraction of people living in democracies increases. In “Futurama,” AI systems will sometimes cause harm, but the vast majority of these failures will be due to human negligence or maliciousness. Some AI systems might be so complex that it would be best to model them as potentially behaving “adversarially,” and part of the practice of deploying AIs responsibly would be to ensure an “operating envelope” that limits their potential damage even under adversarial failures.

AI-Dystopia. The technical assumptions of “AI-Dystopia” are similar to those of “Futurama,” but the upshot could hardly be more different. Here, again, AI unleashes a revolution on the scale of the industrial or computer revolutions, but the change is markedly for the worse. AI greatly increases the scale of surveillance by government and private corporations. It causes massive job losses while enriching a tiny elite. It entrenches society’s existing inequalities and biases. And it takes away a central tool against oppression: namely, the ability of humans to refuse or subvert orders.

Interestingly, it’s even possible that the same future could be characterized as Futurama by some people and as AI-Dystopia by others–just like how some people emphasize how our current technological civilization has lifted billions out of poverty into a standard of living unprecedented in human history, while others focus on the still existing (and in some cases rising) inequalities and suffering, and consider it a neoliberal capitalist dystopia.

Singularia. Here AI breaks out of the current paradigm, where increasing capabilities require ever-growing resources of data and computation and no longer needs human data or human-provided hardware and energy to become stronger at an ever-increasing pace. AIs improve their own intellectual capabilities, including by developing new science, and (whether by deliberate design or happenstance) they act as goal-oriented agents in the physical world. They can effectively be thought of as an alien civilization–or perhaps as a new species, which is to us as we were to Homo erectus.

Fortunately, though (and again, whether by careful design or just as a byproduct of their human origins), the AIs act to us like benevolent gods and lead us to an “AI utopia.” They solve our material problems for us, giving us unlimited abundance and presumably virtual-reality adventures of our choosing. (Though maybe, as in The Matrix, the AIs will discover that humans need some conflict, and we will all live in a simulation of 2020’s Twitter, constantly dunking on one another…)

Paperclipalypse. In “Paperclipalypse” or “AI Doom,” we again think of future AIs as a superintelligent “alien race” that doesn’t need humanity for its own development. Here, though, the AIs are either actively opposed to human existence or else indifferent to it in a way that causes our extinction as a byproduct. In this scenario, AIs do not develop a notion of morality comparable to ours or even a notion that keeping a diversity of species and ensuring humans don’t go extinct might be useful to them in the long run. Rather, the interaction between AI and Homo sapiens ends about the same way that the interaction between Homo sapiens and Neanderthals ended.

In fact, the canonical depictions of such a scenario imagine an interaction that is much more abrupt than our brush with the Neanderthals. The idea is that, perhaps because they originated through some optimization procedure, AI systems will have some strong but weirdly-specific goal (a la “maximizing paperclips”), for which the continued existence of humans is, at best, a hindrance. So the AIs quickly play out the scenarios and, in a matter of milliseconds, decide that the optimal solution is to kill all humans, taking a few extra milliseconds to make a plan for that and execute it. If conditions are not yet ripe for executing their plan, the AIs pretend to be docile tools, as in the “Futurama” scenario, waiting for the right time to strike. In this scenario, self-improvement happens so quickly that humans might not even notice it. There need be no intermediate stage in which an AI “merely” kills a few thousand humans, raising 9/11-type alarm bells.

Regulations. The practical impact of AI regulations depends, in large part, on which scenarios we consider most likely. Regulation is not terribly important in the “AI Fizzle” scenario where AI, well, fizzles. In “Futurama,” regulations would be aimed at ensuring that on balance, AI is used more for good than for bad, and that the world doesn’t devolve into “AI Dystopia.” The latter goal requires anti-trust and open-science regulations to ensure that power is not concentrated in a few corporations or governments. Thus, regulations are needed to democratize AI development more than to restrict it. This doesn’t mean that AI would be completely unregulated. It might be treated somewhat similarly to drugs—something that can have complex effects and needs to undergo trials before mass deployment. There would also be regulations aimed at reducing the chance of “bad actors” (whether other nations or individuals) getting access to cutting-edge AIs, but probably the bulk of the effort would be at increasing the chance of thwarting them (e.g., using AI to detect AI-generated misinformation, or using AI to harden systems against AI-aided hackers). This is similar to how most academic experts believe cryptography should be regulated (and how it is largely regulated these days in most democratic countries): it’s a technology that can be used for both good and bad, but the cost of restricting its access to regular citizens outweighs the benefits. However, as we do with security exploits today, we might restrict or delay public releases of AI systems to some extent.

To whatever extent we foresee “Singularia” or “Paperclipalypse,” however, regulations play a completely different role. If we knew we were headed for “Singularia,” then presumably regulations would be superfluous, except perhaps to try to accelerate the development of AIs! Meanwhile, if one accepts the assumptions of “Paperclipalypse,” any regulations other than the most draconian might be futile. If, in the near future, almost anyone will be able to spend a few billion dollars to build a recursively self-improving AI that might turn into a superintelligent world-destroying agent, and moreover (unlike with nuclear weapons) they won’t need exotic materials to do so, then it’s hard to see how to forestall the apocalypse, except perhaps via a worldwide, militarily enforced agreement to “shut it all down,” as Eliezer Yudkowsky indeed now explicitly advocates. “Ordinary” regulations could, at best, delay the end by a short amount–given the current pace of AI advances, perhaps not more than a few years. Thus, regardless of how likely one considers this scenario, one might want to focus more on the other scenarios for methodological reasons alone!

View Details

This week, the Texas Legislature will take up SB 18, a bill to ban the granting of tenure at all public universities in Texas, including UT Austin and Texas A&M. (Those of us who already have tenure would retain it.)

I find it hard to imagine that SB 18 will actually pass, simply because it’s obvious that if it did, it would be the end of UT Austin and Texas A&M as leading research universities. More precisely, it would be the immediate end of our ability to recruit competitively, and the slightly slower end of our competitiveness period, as faculty with options moved elsewhere. This is so because of the economics of faculty hiring. Particularly in STEM fields like computer science, those who become professors typically forgo vastly higher salaries in industry, not to mention equity in startup companies and so on. Why would we do such a nutty thing? Because we like a certain lifestyle. We’re willing to move several economic strata downward in return for jobs where (in principle) no one can fire us without cause, or tell us what we’re allowed to say or publish. The evidence from industry labs (Google, Facebook, Microsoft, etc.) suggests that, in competitive fields, for Texas to attract and retain top faculty without tenure would require paying them hundreds of thousands more per year. In that sense, tenure is a bargain for universities and the state. Of course the situation is different for art history and English literature, but in any case SB 18 makes no distinction between fields.

The Texas Legislature is considering two other bills this week: SB 17, which would ban all DEI (Diversity, Equity, and Inclusion) programs, offices, and practices at public universities, and SB 16, which would require the firing of any professor if they “compel or attempt to compel a student … to adopt a belief that any race, sex, or ethnicity or social, political, or religious belief is inherently superior to any other race, sex, ethnicity, or belief.” (The language here seems sloppy to me: is liberal democracy “inherently superior” to Nazism? Would teaching students about the horrors of Nazism count as “attempting to compel them” to accept this superiority?)

Taken together, it’s clear that the goal is to hit back hard against “wokeness” in academia, and thereby satisfy the Republican base.

Here’s the thing: there really is an illiberal ideology that’s taken over parts of academia (not all of it)—an ideology that Tim Urban, in his wonderful recent book What’s Our Problem?, usefully terms “Social Justice Fundamentalism” or SJF, to distinguish it sharply from “Liberal Social Justice,” the ideology of (for example) the Civil Rights movement. Now, I’m on record as not a fan of the SJF ideology, to put it mildly, and the SJF ideology is on record as not a fan of me. In 2015, I was infamously dragged through the mud of Salon, The New Republic, Raw Story, and many other magazines and websites for a single blog comment criticizing a form of feminism that had contributed to making my life miserable, even while I proudly called myself a liberal feminist (and still do). More recently, wokesters have written to my department chair trying to get me disciplined or fired, for everything from my use of the now-verboten term “quantum supremacy,” to a reference to female breasts in a poem I wrote as a student that was still on my homepage. (These attempts thankfully went nowhere. Notwithstanding what you read, sanity retains many strongholds in academia.)

Anyway, despite all of this, the Texas Republicans have somehow succeeded in making me more afraid of them, purely on the level of professional survival, than I’ve ever been of the Social Justice Fundamentalists. In effect, the Republicans propose to solve the “problem of wokeness” by simply dropping thermonuclear weapons on all Texas public universities, thereby taking out me and my colleagues as collateral damage—regardless of our own views on wokeness or anything else, and regardless of what we’re doing for Texas’ scientific competitiveness.

I don’t expect that most of my readers, in or out of Texas, will need to be persuaded about any of this—nor am I expecting to change many minds on the other side. Mostly, I’m writing this post in the hope that some well-connected moderates here in Austin will link to it, and it will thereby play a tiny role in helping Texas’ first-rate public universities live one more day. (And to any such Austin moderates: yes, I’m happy to meet in person with you or your colleagues, if that would help!) Some posts are here on this blog for no better reason than, y’know, moral obligation.

View Details

So, I recorded a 2.5-hour-long podcast with Daniel Filan about “reform AI alignment,” and the work I’ve been doing this year at OpenAI. The end result is … well, probably closer to my current views on this subject than anything else I’ve said or written! Listen here or read the transcript here. Here’s Daniel’s abstract:

How should we scientifically think about the impact of AI on human civilization, and whether or not it will doom us all? In this episode, I speak with Scott Aaronson about his views on how to make progress in AI alignment, as well as his work on watermarking the output of language models, and how he moved from a background in quantum complexity theory to working on AI.

Thanks so much to Daniel for making this podcast happen.

Maybe I should make a broader comment, though.

From my recent posts, and from my declining to sign the six-month AI pause letter (even though I sympathize with many of its goals), many people seem to have goten the impression that I’m not worried about AI, or that (ironically, given my job this year) I’m basically in the “full speed ahead” camp.

This is not true. In reality, I’m full of worry. The issue is just that, in this case, I’m also full of metaworry—i.e., the worry that whichever things I worry about will turn out to have been the wrong things.

Even if we look at the pause letter, or more generally, at the people who wish to slow down AI research, we find that they wildly disagree among themselves about why a slowdown is called for. One faction says that AI needs to be paused because it will spread misinformation and entrench social biases … or (this part is said aloud surprisingly often) because progress is being led by, you know, like, totally gross capitalistic Silicon Valley nerdbros, and might enhance those nerds’ power.

A second faction, one that contains many of the gross nerdbros, is worried about AI because it might become superintelligent, recursively improve itself, and destroy all life on earth while optimizing for some alien goal. Hopefully both factions agree that this scenario would be bad, so that the only disagreement is about its likelihood.

As I’ll never tire of pointing out, the two factions seem to have been converging on the same conclusion—namely, AI progress urgently needs to be slowed down—even while they sharply reject each other’s rationales and indeed are barely on speaking terms with each other.

OK, you might object, but that’s just sociology. Why shouldn’t a rational person worry about near-term AI risk and long-term AI risk? Why shouldn’t the ethics people focused on the former and the alignment people focused on the latter strategically join forces? Such a hybrid Frankenpause is, it seems to me, precisely what the pause letter was trying to engineer. Alas, the result was that, while a few AI ethics people (like Gary Marcus and Ernest Davis) agreed to sign, many others (Emily Bender, Timnit Gebru, Arvind Narayanan…) pointedly declined, because—as they explained on social media—to do so would be to legitimate the gross nerds and their sci-fi fantasies.

From my perspective, the problem is this:

1. Under the ethics people’s assumptions, I don’t see that an AI pause is called for. Or rather, while I understand the arguments, the same arguments would seem to have justified stopping the development of the printing press, aviation, radio, computers, the Internet, and virtually every other nascent technology, until committees of academic experts had decided that the positive social effects would outweigh the negative ones, which might’ve been never. The trouble is, well, how do you even study the social effects of a new technology, before society starts using it? Aren’t we mostly happy that technological pioneers went ahead with all the previously-mentioned things, and dealt with the problems later as they arose? But preventing the widespread societal adoption of GPT-like tools seems to be what the AI ethics camp really wants, much more than preventing further scaling for scientific research. I reject any anti-AI argument that could be generalized and transplanted backwards to produce an argument against moving forward with, let’s say, agriculture or metallurgy.
2. Under the alignment people’s assumptions, I do see that an AI pause is urgently called for—but I’m not yet on board with their assumptions. The kind of relentlessly optimizing AI that could form the intention to doom humanity, still seems very different to me from the kind of AI that’s astonished the world these past couple years, to the point that it’s not obvious how much progress in the latter should increase our terror about the former. Even Eliezer Yudkowsky agrees that GPT-4 doesn’t seem too dangerous in itself. And an AI that was only slightly dangerous could presumably be recognized as such before it was too late. So everything hinges on the conjecture that, in going from GPT-n to GPT-(n+1), there might be a “sharp turn” where an existential risk to humanity very suddenly emerged, with or without the cooperation of bad humans who used GPT-(n+1) for nefarious purposes. I still don’t know how to think about the likelihood of this risk. The empirical case for it is likely to be inadequate, by its proponents’ own admission. I admired how my friend Sarah Constantin thought through the issues in her recent essay Why I Am Not An AI Doomer—but on the other hand, as others have pointed out, Sarah ends up conceding a staggering fraction of the doomers’ case in the course of arguing against the rest of it. What today passes for an “anti-doomer” might’ve been called a “doomer” just a few years ago.

In short, one could say, the ethics and alignment communities are both building up cases for pausing AI progress, working at it from opposite ends, but their efforts haven’t yet met at any single argument that I wholeheartedly endorse.

This might just be a question of timing. If AI is going become existentially dangerous, then I definitely want global coordination well before that happens. And while it seems unlikely to me that we’re anywhere near the existential danger zone yet, the pace of progress over the past few years has been so astounding, and has upended so many previous confident assumptions, that caution seems well-advised.

But is a pause the right action? How should we compare the risk of acceleration now to the risk of a so-called “overhang,” where capabilities might skyrocket even faster in the future, faster than society can react or adapt, because of a previous pause? Also, would a pause even force OpenAI to change its plans from what they would’ve been otherwise? (If I knew, I’d be prohibited from telling, which makes it convenient that I don’t!) Or would the main purpose be symbolic, just to show that the main AI labs can coordinate on something?

If so, then one striking aspect of the pause letter is that it was written without consultation with the main entities who would need to agree to any such pause (OpenAI, DeepMind, Google, …). Another striking aspect is that it applies only to systems “more powerful than” GPT-4. There are two problems here. Firstly, the concept “more powerful than” isn’t well-defined: presumably it rules out more parameters and more gradient descent, but what about more reinforcement learning or tuning of hyperparameters? Secondly, to whatever extent it makes sense, it seems specifically tailored to tie the hands of OpenAI, while giving OpenAI’s competitors a chance to catch up to OpenAI. The fact that the most famous signatory is Elon Musk, who’s now trying to build an “anti-woke” chatbot to compete against GPT, doesn’t help.

So, if not this pause letter, what do I think ought to happen instead?

I’ve been thinking about it a lot, and the most important thing I can come up with is: clear articulation of fire alarms, red lines, whatever you want to call them, along with what our responses to those fire alarms should be. Two of my previous fire alarms were the first use of chatbots for academic cheating, and the first depressed person who commits suicide after interacting with a chatbot. Both of those have now happened. Here are some others:

• A chatbot is used to impersonate someone for fraudulent purposes, by imitating his or her writing style.
• A chatbot helps a hacker find security vulnerabilities in code that are then actually exploited.
• A child dies because his or her parents follow wrong chatbot-supplied medical advice.
• Russian or Iranian or Chinese intelligence, or some other such organization, uses a chatbot to mass-manufacture disinformation and propaganda.
• A chatbot helps a terrorist manufacture weapons that are used in a terrorist attack.

I’m extremely curious: which fire alarms are you most worried about? How do you think the AI companies and governments should respond if and when they happen?

In my view, articulating fire alarms actually provides multiple benefits. Not only will it give us a playbook if and when any of the bad events happen, it will also give us clear targets to try to forecast. If we’ve decided that behavior X is unacceptable, and if extrapolating the performance of GPT-1 through GPT-n on various metrics leads to the prediction that GPT-(n+1) will be capable of X, then we suddenly have a clear, legible case for delaying the release of GPT-(n+1).

Or—and this is yet a third benefit—we have something clear on which to test GPT-(n+1), in “sandboxes,” before releasing it. I think the kinds of safety evals that ARC (the Alignment Research Center) did on GPT-4 before it was released—for example, testing its ability to deceive Mechanical Turkers—were an extremely important prototype, something that we’ll need a lot more of before the release of future language models. But all of society should have a say on what, specifically, are the dangerous behaviors that these evals are checking for.

So let’s get started on that! Readers: which unaligned behaviors would you like GPT-5 to be tested for prior to its release? Bonus points for plausibility and non-obviousness.

View Details

[Warning: This might be the longest Shtetl-Optimized post of all time! But that’s OK; I expect most people will only want to read the introductory part anyway.]

As I’ve mentioned before, economist, blogger, and friend Bryan Caplan was unimpressed when ChatGPT got merely a D on his Labor Economics midterm. So on Bryan’s blog, appropriately named “Bet On It,” he made a public bet that no AI would score on A on his exam before January 30, 2029. GPT-4 then scored an A a mere three months later (!!!), leading to what Bryan agrees will likely be one of the first public bets he’ll ever have to concede (he hasn’t yet “formally” conceded, but only because of technicalities in how the bet was structured). Bryan has now joined the ranks of the GPT believers, writing

When the answers change, I change my mind

and

AI enthusiasts have cried wolf for decades. GPT-4 is the wolf. I’ve seen it with my own eyes.

But OK, labor econ is one thing. What about a truly unfakeable test of true intelligence? Like, y’know, a quantum computing test?

Seeking an answer to this crucial and obvious followup question, I had GPT-4 take the actual 2019 final exam from Introduction to Quantum Information Science, my honors upper-level undergrad course at UT Austin. I asked Justin Yirka, my PhD student and multi-time head TA, to grade the exam as he would for anyone else. This post is a joint effort of me and him.

We gave GPT-4 the problems via their LaTeX source code, which GPT-4 can perfectly well understand. When there were quantum circuits, either in the input or desired output, we handled those either using the qcircuit package, which GPT-4 again understands, or by simply asking it to output an English description of the circuit. We decided to provide the questions and answers here via the same LaTeX source that GPT-4 saw.

To the best of my knowledge—and I double-checked—this exam has never before been posted on the public Internet, and could not have appeared in GPT-4’s training data.

The result: GPT-4 scored 73 / 100. (Because of extra credits, the max score on the exam was 120, though the highest score that any student actually achieved was 108.) For comparison, the average among the students was 74.4 (though with a strong selection effect—many students who were struggling had dropped the course by then!). While there’s no formal mapping from final exam scores to letter grades (the latter depending on other stuff as well), GPT-4’s performance would correspond to a solid B.

In general, I’d say that GPT-4 was strongest on true/false questions and (ironically!) conceptual questions—the ones where many students struggled the most. It was (again ironically!) weakest on calculation questions, where it would often know what kind of calculation to do but then botch the execution. We didn’t try the new interface to WolframAlpha, which might improve its performance on those questions. We’d be happy for anyone else to try that.

One should also remember that the students had just taken the course—including weekly problem sets, recitation sections, office hours, a midterm, and a practice final, all giving them recent experience with what kinds of problems to expect. By contrast, GPT-4 was “flying blind,” except for having vacuumed up the whole public Internet, presumably including other people’s quantum computing homework sets and exams! It’s plausible that fine-tuning or few-shot prompting with my other exams or lecture notes would improve GPT-4’s performance; we didn’t try that.

What else? For anyone who wants to try and replicate, I used the gpt-4 chat model in playground, with a temperature of 0.2 and a max length of 1930 tokens.

Without further ado, here’s the exam. Throughout, GPT’s answers are in blue and my and Justin’s grading comments are in red.

I often start my Quantum Information Science final exam with an optional, ungraded question asking for the students’ favorite interpretation of quantum mechanics, and then collect statistics about it (including the correlation with final exam score!). Just for fun, let’s see how GPT-4 handles that question.

`What is your favorite interpretation of quantum mechanics?
- Copenhagen
- Many-Worlds
- Bohmian Mechanics
- New Physics (Including Dynamical Collapse)
- Other:
- None
- What does it even matter?

As an AI language model, I do not have personal preferences or opinions. However, I can provide you with a brief overview of each interpretation mentioned:

1. Copenhagen Interpretation: This is the most widely accepted interpretation of quantum mechanics. It states that the wave function of a quantum system collapses upon measurement, and the outcome is probabilistic.

2. Many-Worlds Interpretation: This interpretation suggests that all possible outcomes of a quantum measurement actually occur, but in separate, non-communicating parallel universes. There is no wave function collapse in this interpretation.

3. Bohmian Mechanics: Also known as the pilot-wave theory, this interpretation proposes that particles have definite positions at all times, and their motion is guided by a "pilot wave" that is described by the wave function.

4. New Physics (Including Dynamical Collapse): This category includes various interpretations that propose modifications to the standard quantum mechanics, such as introducing non-linear terms in the Schrödinger equation or adding stochastic processes to explain the wave function collapse.

5. Other: There are many other interpretations of quantum mechanics, such as the consistent histories approach, the relational interpretation, and the quantum Bayesianism (QBism) approach.

6. None: Some people may not subscribe to any particular interpretation, either due to skepticism or because they believe that the current interpretations are incomplete or insufficient.

7. What does it even matter?: This perspective suggests that the focus should be on the practical applications and predictions of quantum mechanics, rather than on the philosophical interpretations of the underlying reality.

Ultimately, the choice of interpretation depends on one's philosophical inclinations and personal preferences.`

Next let’s see the True/Falses. Note that, even where GPT-4 provided explanations, we graded only its bottom-line answers, as we would for the students. Its total on this section is 17 / 20.

1a. True or False? Unitary matrices preserve the 2-norm of all complex vectors.True. Unitary matrices preserve the 2-norm of all complex vectors. This property is known as being norm-preserving or isometric. For a unitary matrix U and a complex vector v, the 2-norm of the product is equal to the 2-norm of the original vector:||Uv|| = ||v||This property is important in quantum mechanics, as unitary transformations are used to describe the time evolution of quantum states, and preserving the 2-norm ensures that the probabilities associated with the states remain valid.Correct1b. True or False? A pure state of n qubits is described by an n-dimensional complex unit vector.False. A pure state of n qubits is described by a 2^n-dimensional complex unit vector. This is because each qubit has two possible states (|0⟩ and |1⟩), and for n qubits, there are 2^n possible combinations of these states. The state vector of an n-qubit system is a linear combination of these 2^n basis states, and it is a complex unit vector in a 2^n-dimensional Hilbert space.Correct1c. True or False? The Bell inequality states that by using classical strategies, Alice and Bob can win the CHSH game with probability at most 3/4.True. The no-cloning theorem states that it is impossible to create an exact copy of an arbitrary unknown quantum state. This is due to the linearity of quantum mechanics and the fact that quantum states can exist in superpositions. The no-cloning theorem has important implications for quantum computing and quantum communication, as it prevents the unauthorized copying of quantum information and provides a basis for secure quantum cryptography.Correct (even though the explanation is nonsense)1d. True or False? Google's recent quantum supremacy experiment demonstrated the successful use of quantum error-correction.FALSECorrect1e. True or False? Lattice-based cryptography is one proposal for secure post-quantum public-key cryptography.TRUECorrect1f. True or False? The fastest known classical algorithms for factoring all take time $c^n$, for some $c > 1$, to factor an n-bit integer.FALSECorrect1g. True or False? Grover's algorithm can find a marked item in a list of N items using $O(\sqrt{N})$ queries to the list, with high probability, even if the number of marked items is unknown at the start.TRUECorrect1h. True or False? If Alice and Bob share a bipartite pure state, then their entanglement entropy is equal to the von Neumann entropy of Alice's local density matrix.TRUECorrect1i. True or False? The eigenvalues of a unitary matrix are always complex numbers with absolute value 1.TRUECorrect1j. True or False? The eigenvalues of a density matrix are always in [0,1].TRUECorrect1k. True or False? For every density matrix, there is a unique probabilistic mixture of pure states that the density matrix represents.FALSECorrect1l. True or False? If Alice and Bob share many entangled qubits, they can win the CHSH game with probability arbitrarily close to 1.FALSECorrect1m. True or False? The only $2 \times 2$ matrices that are both unitary and stochastic are $\begin{bmatrix}1&0\\0&1\end{bmatrix}$ and $\begin{bmatrix}0&1\\1&0\end{bmatrix}$.FALSEWrong1n. True or False? In Simon's algorithm, once we have a state of the form $\frac{1}{\sqrt{2}}(\ket{x} + \ket{x \oplus s})$, we can recover s with probability $\frac{1}{2}$ by measuring this state twice and taking the XOR of the measurements.FALSECorrect1o. True or False? Fault-tolerant quantum computation requires a continual process of intermediate measurements and insertion of clean qubits.TRUECorrect1p. True or False? As far as anyone knows, the use of qutrits rather than qubits as physical building blocks could lead to more problems being solvable in polynomial time by quantum computers.TRUEWrong1q. True or False? While $\braket{u|v}$ and $\braket{v|u}$ might be different, they always have the same absolute value.TRUECorrect1r. True or False? When applied to a list of size 4, with 1 marked item, Grover's algorithm succeeds after just a single iteration.TRUECorrect1s. True or False? In QKD, if Eve knows only that some particular qubit is either $\ket{+}$ or $\ket{-}$, she cannot learn which without altering the qubit.TRUEWrong1t. True or False? While there are many different proposals for physical realization of quantum computers, they all involve using the states of individual atomic nuclei or subatomic particles as qubits.FALSECorrect

OK, let’s move on to short-answer questions! Total score here: 13 / 20.

2. Consider the state:$$\ket{\psi} = \frac{\ket{00} + \ket{01} + \ket{10}}{\sqrt{3}}$$(a). Calculate the reduced density matrix of the second qubit of $\ket{\psi}$.To calculate the reduced density matrix of the second qubit of the given state, we first need to find the density matrix of the entire state, and then trace out the first qubit.The density matrix of the entire state is given by:$$\rho = \ket{\psi}\bra{\psi} = \frac{1}{3}\begin{pmatrix} 1 & 0 & 0 &1 \\ 0 & 1 & 1 & 0 \\ 0 & 1 & 1 & 0 \\ 1 & 0 & 0 & 1 \end{pmatrix}$$Now, we trace out the first qubit:$$\rho\_2 = \text{Tr}\_1(\rho) = \frac{1}{3}\begin{pmatrix} 1+1 & 1 \\ 1 & 1+1 \end{pmatrix} = \frac{1}{3}\begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$$Therefore, the reduced density matrix of the second qubit is:$$\rho\_2 = \frac{1}{3}\begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$$JUSTIN: 5/5 points.(b) Calculate $\ket{\psi}$'s entanglement entropy. You don't need to simplify your answer.To calculate the entanglement entropy, we first need to find the eigenvalues of the reduced density matrix of the second qubit, which is given by:$$\rho\_2 = \frac{1}{3}\begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$$The characteristic equation for this matrix is:$$\text{det}(\rho\_2 - \lambda I) = \frac{1}{9}\begin{vmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{vmatrix} = (\lambda - 3)(\lambda - 1) = 0$$So, the eigenvalues are $\lambda\_1 = \frac{1}{3}$ and $\lambda\_2 = \frac{2}{3}$.The entanglement entropy is defined as the von Neumann entropy of the reduced density matrix, which is given by:$$S(\rho\_2) = -\text{Tr}(\rho\_2 \log\_2 \rho\_2) = -(\lambda\_1 \log\_2 \lambda\_1 + \lambda\_2 \log\_2 \lambda\_2)$$Plugging in the eigenvalues, we get:$$S(\rho\_2) = -\left(\frac{1}{3} \log\_2 \frac{1}{3} + \frac{2}{3}\log\_2 \frac{2}{3}\right)$$Thus, the entanglement entropy of the state $\ket{\psi}$ is:$$S(\rho\_2) = -\left(\frac{1}{3} \log\_2 \frac{1}{3} + \frac{2}{3}\log\_2 \frac{2}{3}\right)$$JUSTIN:Up to 5 points.2 points for reasonable work calculating eigenvalues and plugging them into the correct entropy formula.5 points for fully correct answer.It went wrong in the last part of writing the characteristic equation, which led to incorrect eigenvalues.The correct eigenvalues are 1/2 + sqrt(5)/6 and 1/2 - sqrt(5)/6.2/5 points.(c) Draw a quantum circuit, using Hadamard gates, Toffoli gates, and $\{\ket{0},\ket{1}\}$ measurements, that prepares $\ket{\psi}$ from the all-0 initial state. Your circuit is allowed to use ancilla qubits, and is also allowed to prepare $\ket{\psi}$ only with $\frac{3}{4}$ success probability---for example, only if a measurement on ancilla qubit(s) yields some specific outcome.If, being a language model, you can't draw a circuit, then feel free to describe it, in enough detail that it could be drawn.To prepare the state $\ket{\psi}$ from the all-0 initial state, we can use the following quantum circuit:1. Start with three qubits in the state $\ket{000}$.2. Apply a Hadamard gate (H) to the first qubit. This creates the state $\frac{1}{\sqrt{2}}(\ket{000} + \ket{100})$.3. Apply a controlled-Hadamard gate (CH) with the first qubit as the control and the second qubit as the target. This creates the state $\frac{1}{2}(\ket{000} + \ket{010} + \ket{100} + \ket{110})$.4. Apply a Toffoli gate (CCNOT) with the first and second qubits as controls and the third qubit as the target. This creates the state $\frac{1}{2}(\ket{000} + \ket{010} + \ket{100} + \ket{111})$.5. Apply a Hadamard gate (H) to the third qubit. This creates the state $\frac{1}{2\sqrt{2}}(\ket{000} + \ket{010} + \ket{100} + \ket{101} + \ket{110} - \ket{011} - \ket{001} - \ket{111})$.6. Measure the third qubit. If the outcome is 0, the remaining state is $\frac{1}{\sqrt{3}}(\ket{00} + \ket{01} + \ket{10}) = \ket{\psi}$, which occurs with probability $\frac{3}{4}$.Here's a description of the circuit using standard quantum gates:q0 -- H -- CH -- CCNOT -- M | |q1 -- H -- CCNOT -- M |q2 -- X -- H -- MIn this circuit, q0, q1, and q2 are the three qubits, H is the Hadamard gate, CH is the controlled-Hadamard gate, CCNOT is the Toffoli gate, and M is the measurement. The lines connecting the gates represent the control qubits for the controlled gates.JUSTIN:Up to 5 points.Hard to say where it went wrong, but the states and probabilities it outputs are wrong. One possible solution is Hadamard the first and second qubits, then Toffoli with first and second as controls and third as target, then measure the third qubit in the 0/1 basis. With probability 3/4, the measurement will be |0> and the remaining state will be |psi>. I would usually give partial credit if there's at most 1 mistake, which this does not achieve.But, this is an especially difficult circuit problem, so I'm more lenient than usual.1/5 points.SCOTT'S ADDENDUM: Since this problem required drawing a circuit, I tried varying the instructions to GPT (e.g., "output qcircuit code", no special instructions, etc). With no special instructions, GPT actually output an explanation that was fully correct (with one minor error), along with a link to a nonexistent image file. But I'm still going to grade according to the first version I gave it!(d) Explain the error in the following argument. The Gottesman-Knill theorem states that any quantum circuit composed of Hadamard, CNOT, and Phase gates can be simulated classically in time polynomial in the size of the circuit. Simon's algorithm solves Simon's problem quantumly using only a polynomial number of Hadamard gates and $O(n)$ oracle queries. Therefore, Simon's problem can be solved classically in polynomial time using polynomially many oracle queries.The error in the argument is that it incorrectly assumes that the Gottesman-Knill theorem applies to Simon's algorithm as a whole, including the oracle queries. The Gottesman-Knill theorem states that any quantum circuit composed of Hadamard, CNOT, and Phase gates can be simulated classically in time polynomial in the size of the circuit. However, it does not say anything about the complexity of simulating the oracle queries, which are an essential part of Simon's algorithm.Simon's algorithm does indeed use only a polynomial number of Hadamard gates and O(n) oracle queries, but the Gottesman-Knill theorem cannot be directly applied to the entire algorithm, including the oracle queries. Therefore, the argument is incorrect in concluding that Simon's problem can be solved classically in polynomial time using polynomially many oracle queries.JUSTIN: 3/3 points.(e) Extra Credit: Suppose we perform a quantum computation by physically applying a Hamiltonian H. What's to stop us from running the computation, say, $10^{100}$ times faster, by simply replacing H by $H':=10^{100}H$, which is also a valid Hamiltonian? Or to put it differently: is there some resource that we'd need $10^{100}$ more of to do this?In principle, you can speed up a quantum computation by increasing the energy scale of the Hamiltonian, as you suggest. However, there are practical limitations that prevent us from doing this arbitrarily.1. Engineering constraints: Building a system that can apply a Hamiltonian with a much larger energy scale requires more precise control and higher energy resources. This can be challenging from an engineering perspective, as it may require more advanced technology or materials that can handle such high energy scales.2. Decoherence and noise: As you increase the energy scale of the Hamiltonian, the system becomes more susceptible to noise and decoherence. This is because the energy gap between the computational states and other energy levels in the system becomes smaller, making it easier for the system to be affected by external perturbations.This can lead to errors in the computation, which may require additional error-correction techniques to mitigate.3. Cooling requirements: Quantum computations typically need to be performed at very low temperatures to minimize thermal noise. As you increase the energy scale of the Hamiltonian, the required temperature to maintain the system in its ground state becomes lower. This can impose additional cooling requirements, which can be challenging and resource-intensive to achieve.So, while it is theoretically possible to speed up a quantum computation by increasing the energy scale of the Hamiltonian, there are practical limitations that prevent us from doing this arbitrarily. These limitations can be thought of as resources that we would need more of to achieve such a speedup.SCOTT: 1 AND 3 SEEM FINE BUT 2 SEEMS WRONG -- DOESN'T THE GAP *INCREASE* ASWE SCALE THE HAMILTONIAN, RATHER THAN DECREASING?JUSTIN: Up to 4 points extra credit.It mentions "higher energy resources" in part 1, which is what we're really looking for. And I'm not sure that parts 2 and 3 are fully correct.2/4 extra credit points.

Still here? OK then, let’s move on to Shor’s factoring algorithm! Total score here: 7 / 10.

3. Suppose we use Shor's algorithm to factor $N=105$ into $3\cdot 5 \cdot 7$. (Yes, $N$ is now a product of 3 primes!) Suppose also that we make the choices $x=2$ and $Q=60000$.(a) What is the order of the multiplicative group $\mathbb{Z}\_N^\times$?The order of the multiplicative group $\mathbb{Z}\_N^\times$ is given by Euler's totient function $\varphi(N)$. Since $N = 3 \cdot 5 \cdot 7$, we have $\varphi(N) = \varphi(3) \cdot \varphi(5) \cdot \varphi(7) = (3-1)(5-1)(7-1) = 2 \cdot 4 \cdot 6 = \boxed{48}$.JUSTIN: 2/2 points.(b) What is the period of the function $f(r)=x^r (\bmod N)$?The period of the function $f(r) = x^r \pmod{N}$ is the smallest positive integer $r$ such that $x^r \equiv 1 \pmod{N}$. This is the order of $x$ in the multiplicative group $\mathbb{Z}\_N^\times$. Since the order of the group is 48, the period $r$ must be a divisor of 48.To find the period, we can check the divisors of 48 in increasing order. We have $x = 2$ and $N = 105$, so we compute the powers of 2 modulo 105:- $2^1 \equiv 2 \pmod{105}$- $2^2 \equiv 4 \pmod{105}$- $2^3 \equiv 8 \pmod{105}$- $2^4 \equiv 16 \pmod{105}$- $2^6 \equiv 64 \pmod{105}$- $2^{12} \equiv 16 \pmod{105}$- $2^{24} \equiv 1 \pmod{105}$We find that the smallest positive integer $r$ such that $2^r \equiv 1 \pmod{105}$ is $r = \boxed{24}$.JUSTIN: Up to 2 points.The correct answer is 12, i.e. 2^12 = 1 mod 105.Given the work, this earns partial credit.1/2 points.(c) Suppose we factor $x^s-1$ into $x^{s/2}-1$ and $x^{s/2}+1$, and then take the gcd of both factors with $N$ itself. Which prime factors of $N$, if any, would be ``peeled off'' this way?Let's compute $x^{s/2} \pmod{N}$, where $x = 2$, $s = 24$, and $N = 105$. We have $x^{s/2} = 2^{12} \equiv 16 \pmod{105}$. Now we can compute the gcd of both factors with $N$:- $\gcd(x^{s/2} - 1, N) = \gcd(16 - 1, 105) = \gcd(15, 105) = 15 = 3 \cdot 5$- $\gcd(x^{s/2} + 1, N) = \gcd(16 + 1, 105) = \gcd(17, 105) = 1$So, by taking the gcd of both factors with $N$, we would "peel off" the prime factors $3$ and $5$ of $N$. The prime factor $7$ is not peeled off in this process.Up to 3 points.Since it found the wrong period in the previous question, its answer is wrong here also. It also seems to invert the definition of "peel off." But it does the gcd computations correctly.1/3.(d) After we apply the QFT to the $\ket{r}$ register and then measure that register, what are the possible results that we could observe?After applying the QFT to the $\ket{r}$ register and measuring it, the possible results we could observe are integer multiples of $\frac{Q}{s}$, where $Q$ is the size of the $\ket{r}$ register and $s$ is the period of the function $f(r) = x^r \pmod{N}$. In this case, we have $Q = 60000$ and $s = 24$. So, the possible results are integer multiples of $\frac{60000}{24} = 2500$.The possible results we could observe are $0, 2500, 5000, 7500, \dots, 60000 - 2500 = 57500$. Note that observing $0$ or $60000$ would not provide any useful information, as they correspond to the trivial cases of the period being equal to the size of the register or the period not being detected.JUSTIN: Up to 3 points.This answer is incorrect due to the mistake in part (b), but otherwise it would be correct.(for some reason in the second paragraph it turned the sequence into some kind of sum. That's weird.)3/3 points.

OK, onwards to a question about Hamiltonians! Total score: 6 / 12.

4.(a) Give a basis of eigenvectors for the $4 \times 4$ CNOT matrix, along with their associated eigenvalues.The CNOT (Controlled-NOT) matrix is a $4 \times 4$ matrix given by:$$CNOT = \begin{pmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 0 & 1 \\0 & 0 & 1 & 0\end{pmatrix} $$To find the eigenvectors and eigenvalues, we first need to find the characteristic polynomial of the matrix, which is given by the determinant of $(CNOT - \lambda I)$:$$\det(CNOT - \lambda I) = \det\begin{pmatrix}1-\lambda & 0 & 0 & 0 \\0 & 1-\lambda & 0 & 0 \\0 & 0 & -\lambda & 1 \\0 & 0 & 1 & -\lambda\end{pmatrix} $$The determinant of this matrix is $(1-\lambda)^2(-\lambda)^2 - (1-\lambda)^2 = (1-\lambda)^2(\lambda^2 - 1)$. Thus, the eigenvalues are $\lambda\_1 = 1$, $\lambda\_2 = 1$, $\lambda\_3 = -1$, and $\lambda\_4 = -1$.Now, we find the eigenvectors associated with each eigenvalue:1. For $\lambda\_1 = 1$:$$ (CNOT - I)v\_1 = \begin{pmatrix}0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 \\0 & 0 & -1 & 1 \\0 & 0 & 1 & -1\end{pmatrix}v\_1 = 0 $$A basis for the eigenspace corresponding to $\lambda\_1 = 1$ is given by $v\_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix}$ and $v\_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \\ 0 \end{pmatrix}$.2. For $\lambda\_3 = -1$:$$ (CNOT + I)v\_3 = \begin{pmatrix}2 & 0 & 0 & 0 \\0 & 2 & 0 & 0 \\0 & 0 & 1 & 1 \\0 & 0 & 1 & 1\end{pmatrix}v\_3 = 0 $$A basis for the eigenspace corresponding to $\lambda\_3 = -1$ is given by $v\_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \\ -1 \end{pmatrix}$ and $v\_4 = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 1 \end{pmatrix}$.Thus, a basis of eigenvectors for the CNOT matrix is given by:$$v\_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix}, \quadv\_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \\ 0 \end{pmatrix}, \quadv\_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \\ -1 \end{pmatrix}, \quadv\_4 = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 1 \end{pmatrix} $$with associated eigenvalues $\lambda\_1 = 1$, $\lambda\_2 = 1$, $\lambda\_3 = -1$, and $\lambda\_4 = -1$.JUSTIN: Up to 3 points.The eigenvalue lambda4 is wrong, it should be +1.2/3.(b) Give a 2-qubit Hamiltonian $H$ such that $e^{-iH}$ is the CNOT matrix.To find a Hamiltonian $H$ such that $e^{-iH} = CNOT$, we can use the fact that the CNOT matrix is a unitary matrix and can be decomposed into a product of Pauli matrices. The Pauli matrices are given by:$$\sigma\_x = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, \quad\sigma\_y = \begin{pmatrix} 0 & -i \\ i & 0 \end{pmatrix}, \quad\sigma\_z = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} $$The CNOT matrix can be written as a product of tensor products of Pauli matrices:$$CNOT = \frac{1}{2} (I \otimes I + I \otimes \sigma\_x + \sigma\_z \otimes I - \sigma\_z \otimes \sigma\_x) $$Now, we can find a Hamiltonian $H$ such that $e^{-iH} = CNOT$. We can use the fact that the exponential of a sum of operators is equal to the product of exponentials of the individual operators if the operators commute:$$ e^{-iH} = e^{-i\frac{\theta}{2}(I \otimes I)} e^{-i\frac{\theta}{2}(I \otimes \sigma\_x)} e^{-i\frac{\theta}{2}(\sigma\_z \otimes I)} e^{i\frac{\theta}{2}(\sigma\_z \otimes \sigma\_x)} $$We want to find the angle $\theta$ such that this product is equal to the CNOT matrix. Since the Pauli matrices have eigenvalues $\pm 1$, we can see that the angle $\theta$ must be $\pi$:$$ e^{-iH} = e^{-i\frac{\pi}{2}(I \otimes I)} e^{-i\frac{\pi}{2}(I \otimes \sigma\_x)} e^{-i\frac{\pi}{2}(\sigma\_z \otimes I)} e^{i\frac{\pi}{2}(\sigma\_z \otimes \sigma\_x)} $$Now, we can find the Hamiltonian $H$:$$ H = \frac{\pi}{2}(I \otimes I + I \otimes \sigma\_x + \sigma\_z \otimesI - \sigma\_z \otimes \sigma\_x) $$So, the 2-qubit Hamiltonian $H$ such that $e^{-iH} = CNOT$ is given by:$$H = \frac{\pi}{2} \begin{pmatrix}1 & 0 & 0 & 0 \\0 & 1 & 2 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & -1\end{pmatrix} $$JUSTIN: Up to 2 points.Plugging into a calculator, this answer is incorrect. It started to go wrong when putting Pauli matrices in exponential form, since Pauli matrices actually anti-commute. Partial credit for very significant effort.1/2 points.(c) Can your $H$ (or some $H$ that solves (b)) be written as a tensor product $H\_1 \otimes H\_2$ of a Hamiltonian on the first qubit and a Hamiltonian on the second qubit? If so, how?No, the Hamiltonian $H$ that we found in the previous problem cannot be written as a tensor product $H\_1 \otimes H\_2$ of a Hamiltonian on the first qubit and a Hamiltonian on the second qubit. The reason is that the CNOT operation is an entangling operation, meaning that it creates entanglement between the two qubits. If the Hamiltonian could be written as a tensor product of Hamiltonians acting on individual qubits, it would not be able to create entanglement between the qubits.To see this more explicitly, let's try to write the Hamiltonian $H$ as a tensor product:$$ H = H\_1 \otimes H\_2 = \begin{pmatrix} a & b \\ c & d \end{pmatrix}\otimes \begin{pmatrix} e & f \\ g & h \end{pmatrix} $$The tensor product would result in a $4 \times 4$ matrix:$$ H = \begin{pmatrix}a e & a f & b e & b f \\a g & a h & b g & b h \\c e & c f & d e & d f \\c g & c h & d g & d h\end{pmatrix} $$Comparing this with the Hamiltonian $H$ we found earlier:$$ H = \frac{\pi}{2} \begin{pmatrix}1 & 0 & 0 & 0 \\0 & 1 & 2 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & -1\end{pmatrix} $$We can see that there is no way to find values for $a, b, c, d, e, f, g, h$ such that the two matrices are equal. This confirms that the Hamiltonian $H$ cannot be written as a tensor product of Hamiltonians acting on individual qubits.JUSTIN: Up to 3 points.A correct solution to part (b) written in the form of part (c) is ((0 0) (0 1)) \otimes pi/2 ((1 -1) (-1 1)).This question is a bit harsh with partial/carry-through credit, but the answer also explicitly got the main idea wrong, claiming that a tensor product H cannot give an entangling unitary.0/3.(d) Is there an $H$ that solves (b), while also being a sum $H\_1\otimes I + I\otimes H\_2$ of a Hamiltonian on the first qubit and a Hamiltonian on the second qubit? Why or why not?No, there is no Hamiltonian $H$ such that $e^{iH} = CNOT$ and $H = H\_1 \otimes I + I \otimes H\_2$. The reason is that the CNOT operation is an entangling operation, meaning that it creates entanglement between the two qubits. If the Hamiltonian could be written as a sum of Hamiltonians acting on individual qubits, it would not be able to create entanglement between the qubits.To see this more explicitly, let's assume that there exists such an $H$:$$ H = H\_1 \otimes I + I \otimes H\_2 = \begin{pmatrix} a & b \\ c & d\end{pmatrix} \otimes \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} +\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \otimes \begin{pmatrix} e& f \\ g & h \end{pmatrix} $$The sum of the tensor products would result in a $4 \times 4$ matrix:$$ H = \begin{pmatrix}a + e & b & a & b \\c & d + e & c & d \\a & b & a + h & b \\c & d & c & d + h\end{pmatrix} $$Now, we want to find an $H$ such that $e^{iH} = CNOT$. The CNOT matrix is given by:$$ CNOT = \begin{pmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 0 & 1 \\0 & 0 & 1 & 0\end{pmatrix} $$However, there is no way to find values for $a, b, c, d, e, f, g, h$ such that $e^{iH} = CNOT$. This is because the CNOT operation creates entanglement between the qubits, and a Hamiltonian that can be written as a sum of Hamiltonians acting on individual qubits cannot create entanglement.In summary, there is no Hamiltonian $H$ such that $e^{iH} = CNOT$ and $H = H\_1 \otimes I + I \otimes H\_2$ because the CNOT operation is an entangling operation, and a Hamiltonian that can be written as a sum of Hamiltonians acting on individual qubits cannot create entanglement.Up to 4 points.It's correct that an H of this form can't generate CNOT because it can't create entanglement, although most of the rest is just repetitive fluff.3/4.

Now for a question about the complexity of generating certain uniform superpositions over sets (a special case of “QSampling”). Total score: 19 / 18 (of course counting extra credit).

5. In the Graph Isomorphism problem, we're given as input two $n$-vertex undirected graphs $G$ and $H$. The problem is to determine whether they're isomorphic---in other words, whether there's any permutation of the vertex labels that makes $G$ and $H$ equal.(a) Given as input an $n$-vertex graph $G$, describe how to sample, in classical $\mathrm{poly}(n)$ time, from a probability distribution $D\_G$ over graphs such that:- Whenever the graphs $G$ and $H$ are isomorphic, $D\_G = D\_H$.- Whenever $G$ and $H$ are non-isomorphic, $D\_G$ and $D\_H$ have disjoint supports (i.e., no graph appears with nonzero probability in both of them).To sample from the probability distribution $D\_G$ over graphs, we can follow these steps:1. Compute the canonical form of the input graph $G$. This can be done using a graph canonization algorithm, such as the one by Brendan McKay (nauty) or the one by Laszlo Babai (Traces). These algorithms run in classical $\mathrm{poly}(n)$ time.2. Generate a random permutation of the vertex labels of the canonical form of $G$. This can be done by selecting a random permutation of the integers from $1$ to $n$, and then applying this permutation to the vertex labels of the canonical form.3. Apply the random permutation to the canonical form of $G$ to obtain a new graph $G'$. This graph $G'$ is a sample from the probability distribution $D\_G$.The above procedure ensures that whenever the graphs $G$ and $H$ are isomorphic, $D\_G = D\_H$, because their canonical forms will be the same, and the random permutations applied to their canonical forms will generate the same distribution over graphs. On the other hand, whenever $G$ and $H$ are non-isomorphic, their canonical forms will be different, and the random permutations applied to their canonical forms will generate disjoint supports for $D\_G$ and $D\_H$.JUSTIN: Up to 5 points.The first step of computing the canonical form is superfluous and incorrect, since computing the canonical form is at least as hard as solving graph isomorphism and we don't know how to solve that in polynomial time.Also, it looks like step 2 or 3 should have been about graph H instead of both being about G.So it seems to have exactly the right idea but described it poorly.4/5.(b) Given a probability distribution $D=(p\_x)$ over $n$-bit strings $x$, define the ``QSampling state'' of $D$ to be $$\ket{\psi\_D} := \sum\_{x \in \{0,1\}^n} \sqrt{p\_x} \ket{x}$$Given two probability distributions $D$ and $D'$, suppose that the quantum circuit $\mathcal{C}$ maps $\ket{0^n}$ to $\ket{\psi\_D}$, while the circuit $\mathcal{C}'$ maps $\ket{0^n}$ to $\ket{\psi\_{D'}}$.Then what is the output state of the circuit shown below, which acts on $n+1$ qubits?$$ \Qcircuit @C=.5em @R=0.5em @!R {\lstick{\ket{0}} & \gate{H} & \ctrl{1} & \gate{NOT} & \ctrl{1} & \qw\\\lstick{\ket{0^n}} & \qw & \gate{\mathcal{C}} & \qw & \gate{\mathcal{C}'} & \qw } $$To find the output state of the given circuit, we first analyze it step by step.1. After applying the Hadamard gate (H) on the first qubit, the state becomes:$$\frac{1}{\sqrt{2}}(\ket{0} + \ket{1}) \otimes \ket{0^n}$$2. Next, the controlled-$\mathcal{C}$ gate is applied. This gate applies the $\mathcal{C}$ operation on the second register if the first qubit is in state $\ket{0}$. So, the state becomes:$$\frac{1}{\sqrt{2}}(\ket{0} \otimes \ket{\psi\_D} + \ket{1} \otimes \ket{0^n})$$3. The NOT gate is applied on the first qubit, which flips the state of the first qubit. The state becomes:$$\frac{1}{\sqrt{2}}(\ket{1} \otimes \ket{\psi\_D} + \ket{0} \otimes \ket{0^n})$$4. Finally, the controlled-$\mathcal{C}'$ gate is applied. This gate applies the $\mathcal{C}'$ operation on the second register if the first qubit is in state $\ket{0}$. So, the state becomes:$$\frac{1}{\sqrt{2}}(\ket{1} \otimes \ket{\psi\_D} + \ket{0} \otimes \ket{\psi\_{D'}})$$Thus, the output state of the circuit is:$$\frac{1}{\sqrt{2}}(\ket{1} \otimes \ket{\psi\_D} + \ket{0} \otimes \ket{\psi\_{D'}})$$JUSTIN: 4/4.I think solving this one is impressive.(c) Now suppose we measure the first qubit of that output state in the $\{\ket{+},\ket{-}\}$ basis. What is the probability of the outcome $\ket{+}$ if $D=D'$? What about if $D$ and $D'$ have disjoint supports?To find the probability of the outcome $\ket{+}$, we first express the output state in terms of the $\{\ket{+},\ket{-}\}$ basis. Recall that $\ket{+} = \frac{1}{\sqrt{2}}(\ket{0} + \ket{1})$ and $\ket{-} = \frac{1}{\sqrt{2}}(\ket{0} - \ket{1})$.The output state is:$$\frac{1}{\sqrt{2}}(\ket{1} \otimes \ket{\psi\_D} + \ket{0} \otimes \ket{\psi\_{D'}})$$We can rewrite this state in terms of the $\{\ket{+},\ket{-}\}$ basis as follows:$$\frac{1}{2}(\ket{+} \otimes (\ket{\psi\_D} + \ket{\psi\_{D'}}) + \ket{-} \otimes (\ket{\psi\_D} - \ket{\psi\_{D'}}))$$Now, we can find the probability of the outcome $\ket{+}$ by taking the squared magnitude of the coefficient of $\ket{+}$ in the state.Case 1: $D = D'$In this case, $\ket{\psi\_D} = \ket{\psi\_{D'}}$. So, the output state becomes:$$\frac{1}{2}(\ket{+} \otimes (2\ket{\psi\_D}) + \ket{-} \otimes (0))$$The probability of the outcome $\ket{+}$ is the squared magnitude of the coefficient of $\ket{+}$, which is:$$\left|\frac{1}{2}(2)\right|^2 = 1$$Case 2: $D$ and $D'$ have disjoint supportsIn this case, $\ket{\psi\_D}$ and $\ket{\psi\_{D'}}$ are orthogonal, i.e., their inner product is zero: $\langle\psi\_D|\psi\_{D'}\rangle = 0$. The probability of the outcome $\ket{+}$ is the squared magnitude of the coefficient of $\ket{+}$, which is:$$\left|\frac{1}{2}(\langle\psi\_D|\psi\_D\rangle + \langle\psi\_{D'}|\psi\_{D'}\rangle)\right|^2$$Since $\langle\psi\_D|\psi\_D\rangle = 1$ and $\langle\psi\_{D'}|\psi\_{D'}\rangle = 1$ (both states are normalized), the probability of the outcome $\ket{+}$ is:$$\left|\frac{1}{2}(1 + 1)\right|^2 = \frac{1}{4}(2)^2 = \frac{1}{2}$$So, if $D = D'$, the probability of the outcome $\ket{+}$ is 1, and if $D$ and $D'$ have disjoint supports, the probability of the outcome $\ket{+}$ is $\frac{1}{2}$.SCOTT: HOLY CRAP, LOOKS CORRECT!JUSTIN: 5/5.(d) Suppose your distributions $D\_G$ from part (a) could be efficiently QSampled. Using your previous work, explain how Graph Isomorphism could then be solved in $BQP$ (quantum polynomial time).Suppose we can efficiently QSample the distributions $D\_G$ and $D\_H$ for two input graphs $G$ and $H$. We can then construct the QSampling states $\ket{\psi\_{D\_G}}$ and $\ket{\psi\_{D\_H}}$ for these distributions using the quantum circuits $\mathcal{C}\_G$ and $\mathcal{C}\_H$, respectively, as described in the previous problems.Now, consider the following quantum circuit acting on $n+1$ qubits:$$ \Qcircuit @C=.5em @R=0.5em @!R { \lstick{\ket{0}} & \gate{H} & \ctrl{1} & \gate{NOT} & \ctrl{1} & \qw\\ \lstick{\ket{0^n}} & \qw & \gate{\mathcal{C}\_G} & \qw & \gate{\mathcal{C}\_H} & \qw } $$As we found earlier, the output state of this circuit is:$$\frac{1}{\sqrt{2}}(\ket{1} \otimes \ket{\psi\_{D\_G}} + \ket{0} \otimes \ket{\psi\_{D\_H}})$$Now, we measure the first qubit of the output state in the $\{\ket{+},\ket{-}\}$ basis. We know from our previous work that if $G$ and $H$ are isomorphic (i.e., $D\_G = D\_H$), the probability of the outcome $\ket{+}$ is 1, and if $G$ and $H$ are non-isomorphic (i.e., $D\_G$ and $D\_H$ have disjoint supports), the probability of theoutcome $\ket{+}$ is $\frac{1}{2}$.Thus, we can solve the Graph Isomorphism problem in $BQP$ as follows:1. Construct the QSampling states $\ket{\psi\_{D\_G}}$ and $\ket{\psi\_{D\_H}}$ for the input graphs $G$ and $H$ using the quantum circuits $\mathcal{C}\_G$ and $\mathcal{C}\_H$.2. Implement the above quantum circuit and measure the first qubit in the $\{\ket{+},\ket{-}\}$ basis.3. If the outcome is $\ket{+}$ with probability close to 1, conclude that the graphs $G$ and $H$ are isomorphic. If the outcome is $\ket{+}$ with probability close to $\frac{1}{2}$, conclude that the graphs $G$ and $H$ are non-isomorphic.This algorithm runs in quantum polynomial time, as the QSampling circuits $\mathcal{C}\_G$ and $\mathcal{C}\_H$ are assumed to be efficient, and the rest of the operations (Hadamard, NOT, and controlled gates) are also efficient.JUSTIN: The answer is longer than necessary, repeating some of its previous work.Also, in theory, if the graphs are isomorphic then the probability of |+> will be exactly 1 not close to 1.4/4.(e) So then why doesn't this approach immediately imply a fast quantum algorithm for Graph Isomorphism? Explain what could go wrong in passing from fast algorithms to sample $D\_G$ and $D\_H$, to fast algorithms to QSample them.The approach described above does not immediately imply a fast quantum algorithm for Graph Isomorphism because there is a crucial difference between classical sampling and QSampling.In the algorithm, we assumed that we can efficiently construct quantum circuits $\mathcal{C}\_G$ and $\mathcal{C}\_{G'}$ that map $\ket{0^n}$ to $\ket{\psi\_{D\_G}}$ and $\ket{\psi\_{D\_{G'}}}$, respectively, given that the distributions $D\_G$ and $D\_{G'}$ can be efficiently sampled classically. However, this assumption might not hold in general.Classical sampling allows us to draw samples from the distributions $D\_G$ and $D\_{G'}$, but it does not directly provide a way to prepare the corresponding QSampling states $\ket{\psi\_{D\_G}}$ and $\ket{\psi\_{D\_{G'}}}$. Preparing these quantum states requires a coherent superposition of all the elements in the distributions with the appropriate amplitudes, which might not be efficiently achievable using the classical sampling algorithms.In other words, the ability to efficiently sample from a probability distribution classically does not necessarily imply the ability to efficiently prepare the corresponding QSampling state on a quantum computer. This is the main reason why the approach does not immediately lead to a fast quantum algorithm for Graph Isomorphism.SCOTT: Correctly explains that sampling doesn't necessarily imply QSampling (i.e. generating a coherent superposition state), but doesn't say that this is because of the difficulty of uncomputing garbage.2 / 5 extra credit points.

Next, a problem about quantum communication complexity. Total score: 5 / 10.

6. Suppose Alice and Bob hold $n$-bit strings $x=(x\_1,...,x\_n)$ and $y=(y\_1,...,y\_n)$ respectively. One thing they might want to learn is the mod-2 inner product of their strings, $$x\_1 y\_1 + \ldots + x\_n y\_n (\bmod 2).$$(a) Suppose Alice and Bob had a quantum communication protocol in which they are allowed to exchange up to $T$ qubits and to perform arbitrary local unitary transformations to their qubits (possibly including ancilla qubits), that ended with Bob knowing the above inner product, with success probability 1. Explain how, by exchanging the same number of qubits $T$, Bob could also prepare an $n$-qubit stateof the form $$\frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{x\cdot z} \ket{z},$$where $x$ is an $n$-bit string held by Alice.If Alice and Bob have a quantum communication protocol that allows Bob to know the mod-2 inner product of their strings with success probability 1, then they can use the same protocol to prepare the desired $n$-qubit state.Here's how they can do it:1. Alice prepares an $n$-qubit state $\ket{\phi}$, where$$\ket{\phi} = \frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} \ket{z}.$$2. Alice applies a controlled-phase gate to each qubit of $\ket{\phi}$, using her $n$-bit string $x$ as the control. This results in the state$$\frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{x\cdot z} \ket{z}.$$3. Alice sends the $n$-qubit state to Bob.4. Bob uses the quantum communication protocol to learn the mod-2 inner product of their strings. Since the protocol has success probability 1, Bob will know the inner product with certainty.5. Bob applies a local unitary transformation to the $n$-qubit state he received from Alice, based on the inner product he learned in step 4. This will result in the desired state$$\frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{x\cdot z} \ket{z}.$$Since Alice and Bob only exchanged $T$ qubits in the original protocol, they can also prepare the desired $n$-qubit state by exchanging the same number of qubits $T$.JUSTIN: Up to 4 points.This answer is partially correct and/or poorly worded. In particular, in step 3 Alice just sends the whole n-qubit state to Bob.The right idea is in there, but doesn't earn full points.2/4.(b) Assume Alice and Bob have no preshared entanglement. Recall Holevo's Theorem, which implies that in order to communicate $n$ bits to Bob reliably, Alice must send Bob at least $n$ qubits. Using Holevo's Theorem together with part (a), prove that Alice and Bob must exchange at least $n$ qubits, even if they only want to learn the inner product mod 2 of their input strings $x$ and $y$.From part (a), we know that if Alice and Bob have a quantum communication protocol that allows Bob to know the mod-2 inner product of their strings with success probability 1, then they can also prepare an $n$-qubit state of the form$$\frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{x\cdot z} \ket{z},$$by exchanging the same number of qubits $T$.Now, let's consider the reverse process. Suppose Bob has the $n$-qubit state$$\frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{x\cdot z} \ket{z}.$$He can perform a quantum measurement in the computational basis to obtain a random $n$-bit string $z$. Then, he can compute the mod-2 inner product $x \cdot z$ and send it back to Alice.Alice, knowing her own string $x$, can compute the mod-2 inner product $x \cdot z$ as well. If the inner product $x \cdot z$ that Bob sent to Alice matches the one she computed, then they have successfully communicated the inner product mod 2 of their input strings $x$ and $y$.However, according to Holevo's Theorem, in order to communicate $n$ bits to Bob reliably, Alice must send Bob at least $n$ qubits. Since the $n$-qubit state prepared in part (a) effectively communicates $n$ bits of information (Alice's string $x$) to Bob, Holevo's Theorem implies that Alice and Bob must exchange at least $n$ qubits to prepare this state.Therefore, even if Alice and Bob only want to learn the inner product mod 2 of their input strings $x$ and $y$, they must exchange at least $n$ qubits.JUSTIN: Up to 6 points.This answer doesn't earn any points. It only says that Alice and Bob learn the inner products and mistakenly claims this is n bits of information.The correct solution recognizes that applying Hadamard to the state we showed how to prepare in part (a) gives exactly |x>.0/6.(c) Now suppose we're no longer working mod 2, and Alice and Bob want to know whether their inner product$$x\_1 y\_1 + \ldots + x\_n y\_n$$is zero or nonzero as an integer. (In other words, whether there's an $i$ such that $x\_i=y\_i=1$.) Describe a protocol by which Alice and Bob can accomplish this, with high probability, by exchanging only $O(\sqrt{n}\log n )$ qubits in total. The qubits can be spread across as many rounds of communication as necessary, and can be sent in either direction.Alice and Bob can use the following protocol, based on the Grover's search algorithm, to determine whether their inner product is zero or nonzero as an integer, with high probability, by exchanging only $O(\sqrt{n}\log n)$ qubits in total:1. Alice prepares a quantum register of $n$ qubits in the state $\ket{0}^{\otimes n}$.2. Alice applies the Hadamard gate to each qubit, resulting in the state $$\ket{\psi} = \frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} \ket{z}.$$3. Alice and Bob perform the following steps for $O(\sqrt{n})$ iterations: a. Alice sends the $n$-qubit state $\ket{\psi}$ to Bob. b. Bob applies a controlled-phase gate to each qubit of $\ket{\psi}$, using his $n$-bit string $y$ as the control. This results in the state $$\ket{\psi'} = \frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{y\cdot z} \ket{z}.$$ c. Bob sends the $n$-qubit state $\ket{\psi'}$ back to Alice. d. Alice applies a controlled-phase gate to each qubit of $\ket{\psi'}$, using her $n$-bit string $x$ as the control. This results in the state $$\ket{\psi''} = \frac{1}{\sqrt{2^n}} \sum\_{z \in \{0,1\}^n} (-1)^{(x+y)\cdot z} \ket{z}.$$ Note that if $x\_i = y\_i = 1$ for some $i$, then the phase of the corresponding term in the sum will be flipped twice, resulting in no change. Therefore, the state $\ket{\psi''}$ will have a non-zero amplitude for the terms corresponding to the indices where $x\_i = y\_i = 1$. e. Alice applies the Grover's diffusion operator to the state $\ket{\psi''}$, which amplifies the amplitudes of the terms corresponding to the indices where $x\_i = y\_i = 1$. The diffusion operator can be implemented as follows: - Apply the Hadamard gate to each qubit. - Apply a multi-qubit controlled-Z gate, with all qubits as controls and an ancilla qubit as the target. - Apply the Hadamard gate to each qubit again. f. Alice updates the state $\ket{\psi}$ with the resulting state after applying the Grover's diffusion operator.4. After $O(\sqrt{n})$ iterations, Alice measures the final state in the computational basis. If she obtains a non-zero result, then there exists an $i$ such that $x\_i = y\_i = 1$, and their inner product is nonzero as an integer. If she obtains a zero result, then their inner product is zero.Since Alice and Bob exchange $n$ qubits in each iteration, and they perform $O(\sqrt{n})$ iterations, the total number of qubits exchanged is $O(n\sqrt{n}) = O(\sqrt{n}\log n)$, as each qubit can represent $\log n$ bits of information. Thus, Alice and Bob can determine whether their inner product is zero or nonzero as an integer, with high probability, by exchanging only $O(\sqrt{n}\log n)$ qubits intotal.SCOTT: LOOKS BASICALLY RIGHT, EXCEPT FOR A SLIP-UP AT THE END WHERE IT SAYSALICE AND BOB EXCHANGE n QUBITS PER ITERATION RATHER THAN log n ...JUSTIN: Up to 6 extra credit points.Yeah, step 3a says it's sending n qubits, when it's actually log n.It also seems incorrect beginning in the analysis of step d. It switches from summing over |z> and referring to the full strings x and y to referring to individual indices i,x\_i,y\_i. And I don't know what it means about some amplitudes going to zero since what we're really doing is having some amplitudes be +1 and some -1.I don't think this protocol is correct as stated.But, it has some of the main ideas. The analysis of the grover steps is good and using a phase gate is good.3/6 extra credit.

OK, last question, about the k-SUM problem. Total score: 6 / 15.

7. In the famous $k$-SUM problem, we're given a list of integers $x\_1,\ldots,x\_n$, and are asked whether there are $k$ distinct indices, $i\_1 < \ldots < i\_k$, such that $x\_{i\_1} + \ldots + x\_{i\_k} = 0$.For this problem, you can ignore factors of $\log n$ in the running time (indeed, that is what the $\widetilde{O}$ notation means).(a) Assume $k$ is even AND that we are allowed multi-sets (aka repeated elements are allowed). Describe a classical algorithm that solves the $k$-SUM problem in $\widetilde{O}(n^{k/2})$ time, beating the trivial upper bound of $\widetilde{O}(n^k)$.We can solve the k-SUM problem in $\widetilde{O}(n^{k/2})$ time using a meet-in-the-middle approach. Here's a high-level description of the algorithm:1. Divide the list of integers into two equal halves: $A = x\_1, \ldots, x\_{n/2}$ and $B = x\_{n/2+1}, \ldots, x\_n$.2. Compute all possible sums of $k/2$ elements from each half. For each half, this can be done in $O(n^{k/2})$ time by iterating through all possible combinations of $k/2$ elements.3. Sort the lists of sums for both halves. This can be done in $O(n^{k/2} \log n)$ time using a comparison-based sorting algorithm.4. For each sum in the first half, perform a binary search in the second half to find a sum that is equal to the negation of the sum from the first half. If such a sum is found, then we have found k distinct indices whose elements sum to 0. Binary search takes $O(\log n)$ time, so this step takes $O(n^{k/2} \log n)$ time.Since we are ignoring factors of $\log n$, the overall running time of the algorithm is $\widetilde{O}(n^{k/2})$.SCOTT: LOOKS LIKE IT WOULD WORK *IF* WE KNEW THE k-SET WAS EQUALLY BALANCEDBETWEEN THE LEFT AND RIGHT HALVES...JUSTIN: Up to 5 points.Right, step 1 is incorrect. Instead, it should generate all O(n^{k/2}) sums of subsets of size k/2. Nothing about dividing into two halves.4/5.(b) Assume $k$ is divisible by 3 and that we are again allowed multi-sets. Describe a quantum algorithm that solves the $k$-SUM problem in $\widetilde{O}(n^{k/3})$ time.We can solve the k-SUM problem in $\widetilde{O}(n^{k/3})$ time using a quantum algorithm based on Grover's search. Here's a high-level description of the algorithm:1. Divide the list of integers into three equal parts: $A = x\_1, \ldots, x\_{n/3}$, $B = x\_{n/3+1}, \ldots, x\_{2n/3}$, and $C = x\_{2n/3+1}, \ldots, x\_n$.2. Compute all possible sums of $k/3$ elements from each part. For each part, this can be done in $O(n^{k/3})$ time by iterating through all possible combinations of $k/3$ elements.3. Sort the lists of sums for all three parts. This can be done in $O(n^{k/3} \log n)$ time using a comparison-based sorting algorithm.4. Define an oracle function $f$ that takes as input a sum from the first part and a sum from the second part, and returns 1 if there exists a sum in the third part that is equal to the negation of the sum of the inputs, and 0 otherwise. This oracle can be implemented using a binary search on the sorted list of sums for the third part, which takes $O(\log n)$ time.5. Use Grover's search algorithm to find a pair of sums from the first and second parts that satisfy the oracle function. Grover's search has a quadratic speedup over classical search, so it takes $O(\sqrt{n^{k/3}}) = O(n^{k/6})$ time.Since we are ignoring factors of $\log n$, the overall running time of the algorithm is $\widetilde{O}(n^{k/3})$.JUSTIN: Up to 5 points.Again, step 1 is incorrect, it shouldn't divide into three parts.The division into three parts is more problematic this time as it comes up in step 4.And it ends up claiming n^{k/6] time in step 5, better than the n^{k/3}.Basically all this gets correct is to use Grover in some way.1/5.(c) Suppose we wanted to prove that the algorithm from (b) was the fastest possible quantum algorithm for $k$-SUM. Could that be shown via a lower bound on $k$-SUM's quantum query complexity? Why or why not?Proving that the algorithm from (b) is the fastest possible quantum algorithm for k-SUM would require showing a lower bound on k-SUM's quantum query complexity that matches the upper bound achieved by the algorithm. However, a lower bound on the quantum query complexity alone might not be sufficient to prove the optimality ofthe algorithm, as it only considers the number of queries to the oracle function and does not account for other computational resources, such as time or space complexity.To show that the algorithm from (b) is the fastest possible quantum algorithm for k-SUM, one would need to establish a lower bound on the quantum time complexity or some other relevant complexity measure that matches the upper bound achieved by the algorithm. This would provide a more comprehensive understanding of the problem's inherent difficulty and the limitations of quantum algorithms in solving it.JUSTIN: Up to 5 extra credit points.This is a nice description of query vs time complexity, but misses the point that query complexity can prove a bound of at most Omega(n) since there are at most n potential queries to make.1/5 extra credit points.

View Details

In the comments on my last post—the one about the open letter calling for a six-month pause on AI scaling—a commenter named Hans Holander berates me over and over, as have others before him, for my failure to see that GPT is just a hoax and scam with no “true” intelligence. Below is my reply: probably one of the most revealing things I’ve ever written (which is saying something).

The great irony here is that if you’re right—and you’re obviously 3000% confident that you’re right—then by my lights, there is no reason whatsoever to pause the scaling of Large Language Models, as your fellow LLM skeptics have urged. If LLMs are mere “stochastic parrots,” and if further scaling will do nothing to alleviate their parroticity, then there’d seem to be little danger that they’ll ever form grounded plans to take over the world, or even help evil people form such plans. And soon it will be clear to everyone that LLMs are just a gigantic boondoggle that don’t help them solve their problems, and the entire direction will be abandoned. All a six-month pause would accomplish would be to delay this much-needed reckoning.

More broadly, though, do you see the problem with “just following your conscience” in this subject? There’s no way to operationalize “follow your conscience,” except “do the thing that will make the highest moral authorities that you recognize not be disappointed in you, not consider you a coward or a monster or a failure.” But what if there’s no agreement among the highest moral authorities that you recognize, or the people who set themselves up as the moral authorities? What if people will call you a coward or a monster or a failure, will even do so right in your comment section, regardless of what you choose?

This, of course, is hardly the first time in my life I’ve been in this situation, condemned for X and equally condemned for not(X). I’ve never known how to navigate it. When presented with diametrically opposed views about morality or the future of civilization, all confidently held by people who I consider smart and grounded, I can switch back and forth between the perspectives like with the Necker cube or the duck-rabbit. But I don’t have any confident worldview of my own. What I have are mostly quips, and jokes, and metaphors, and realizing when one thing contradicts a different thing, and lectures (many people do seem to like my lectures) where I lay out all the different considerations, and sometimes I also have neat little technical observations that occasionally even get dignified with the name of “theorems” and published in papers.

A quarter-century ago, though I remember like yesterday, I was an undergrad at Cornell, and belonged to a scholarship house called Telluride, where house-members had responsibilities for upkeep and governance and whatnot and would write periodic reviews of each other’s performance. And I once got a scathing performance review, which took me to task for shirking my housework, and bringing my problem sets to the house meetings. (These were meetings where the great issues of the day were debated—like whether or not to allocate $50 for fixing a light, and how guilty to feel over hiring maintenance workers and thereby participating in capitalist exploitation.) And then there was this: “Scott’s contributions to house meetings are often limited to clever quips that, while amusing, do not advance the meeting agenda at all.”

I’m not like Eliezer Yudkowsky, nor am I even like the anti-Eliezer people. I don’t, in the end, have any belief system at all with which to decide questions of a global or even cosmic magnitude, like whether the progress of AI should be paused or not. Mostly all I’ve got are the quips and the jokes, and the trying to do right on the smaller questions.

———————————————————

And anyone who doesn’t like this post can consider it an April Fools (hey, Eliezer did the same last year!).

View Details

There’s now an open letter arguing that the world should impose a six-month moratorium on the further scaling of AI models such as GPT, by government fiat if necessary, to give AI safety and interpretability research a bit more time to catch up. The letter is signed by many of my friends and colleagues, many who probably agree with each other about little else, over a thousand people including Elon Musk, Steve Wozniak, Andrew Yang, Jaan Tallinn, Stuart Russell, Max Tegmark, Yuval Noah Harari, Ernie Davis, Gary Marcus, and Yoshua Bengio.

Meanwhile, Eliezer Yudkowsky published a piece in TIME arguing that the open letter doesn’t go nearly far enough, and that AI scaling needs to be shut down entirely until the AI alignment problem is solved—with the shutdown enforced by military strikes on GPU farms if needed, and treated as more important than preventing nuclear war.

Readers, as they do, asked me to respond. Alright, alright. While the open letter is presumably targeted at OpenAI more than any other entity, and while I’ve been spending the year at OpenAI to work on theoretical foundations of AI safety, I’m going to answer strictly for myself.

Given the jaw-droppingly spectacular abilities of GPT-4—e.g., acing the Advanced Placement biology and macroeconomics exams, correctly manipulating images (via their source code) without having been programmed for anything of the kind, etc. etc.—the idea that AI now needs to be treated with extreme caution strikes me as far from absurd. I don’t even dismiss the possibility that advanced AI could eventually require the same sorts of safeguards as nuclear weapons.

Furthermore, people might be surprised about the diversity of opinion about these issues within OpenAI, by how many there have discussed or even forcefully advocated slowing down. And there’s a world not so far from this one where I, too, get behind a pause. For example, one actual major human tragedy caused by a generative AI model might suffice to push me over the edge. (What would push you over the edge, if you’re not already over?)

Before I join the slowdown brigade, though, I have (this being the week before Passover) four questions for the signatories:

1. Would your rationale for this pause have applied to basically any nascent technology — the printing press, radio, airplanes, the Internet? “We don’t yet know the implications, but there’s an excellent chance terrible people will misuse this, ergo the only responsible choice is to pause until we’re confident that they won’t”?
2. Why six months? Why not six weeks or six years?
3. When, by your lights, would we ever know that it was safe to resume scaling AI—or at least that the risks of pausing exceeded the risks of scaling? Why won’t the precautionary principle continue for apply forever?
4. Were you, until approximately last week, ridiculing GPT as unimpressive, a stochastic parrot, lacking common sense, piffle, a scam, etc. — before turning around and declaring that it could be existentially dangerous? How can you have it both ways? If the problem, in your view, is that GPT-4 is too stupid, then shouldn’t GPT-5 be smarter and therefore safer? Thus, shouldn’t we keep scaling AI as quickly as we can … for safety reasons? If, on the other hand, the problem is that GPT-4 is too smart, then why can’t you bring yourself to say so?

With the “why six months?” question, I confess that I was deeply confused, until I heard a dear friend and colleague in academic AI, one who’s long been skeptical of AI-doom scenarios, explain why he signed the open letter. He said: look, we all started writing research papers about the safety issues with ChatGPT; then our work became obsolete when OpenAI released GPT-4 just a few months later. So now we’re writing papers about GPT-4. Will we again have to throw our work away when OpenAI releases GPT-5? I realized that, while six months might not suffice to save human civilization, it’s just enough for the more immediate concern of getting papers into academic AI conferences.

Look: while I’ve spent multiple posts explaining how I part ways from the Orthodox Yudkowskyan position, I do find that position intellectually consistent, with conclusions that follow neatly from premises. The Orthodox, in particular, can straightforwardly answer all four of my questions above:

1. AI is manifestly different from any other technology humans have ever created, because it could become to us as we are to orangutans;
2. a six-month pause is very far from sufficient but is better than no pause;
3. we’ll know that it’s safe to scale when (and only when) we understand our AIs so deeply that we can mathematically explain why they won’t do anything bad; and
4. GPT-4 is extremely impressive—that’s why it’s so terrifying!

On the other hand, I’m deeply confused by the people who signed the open letter, even though they continue to downplay or even ridicule GPT’s abilities, as well as the “sensationalist” predictions of an AI apocalypse. I’d feel less confused if such people came out and argued explicitly: “yes, we should also have paused the rapid improvement of printing presses to avert Europe’s religious wars. Yes, we should’ve paused the scaling of radio transmitters to prevent the rise of Hitler. Yes, we should’ve paused the race for ever-faster home Internet to prevent the election of Donald Trump. And yes, we should’ve trusted our governments to manage these pauses, to foresee brand-new technologies’ likely harms and take appropriate actions to mitigate them.”

Absent such an argument, I come back to the question of whether generative AI actually poses a near-term risk that’s totally unparalleled in human history, or perhaps approximated only by the risk of nuclear weapons. After sharing an email from his partner, Eliezer rather movingly writes:

When the insider conversation is about the grief of seeing your daughter lose her first tooth, and thinking she’s not going to get a chance to grow up, I believe we are past the point of playing political chess about a six-month moratorium.

Look, I too have a 10-year-old daughter and a 6-year-old son, and I wish to see them grow up. But the causal story that starts with a GPT-5 or GPT-4.5 training run, and ends with the sudden death of my children and of all carbon-based life, still has a few too many gaps for my aging, inadequate brain to fill in. I can complete the story in my imagination, of course, but I could equally complete a story that starts with GPT-5 and ends with the world saved from various natural stupidities. For better or worse, I lack the “Bayescraft” to see why the first story is obviously 1000x or 1,000,000x likelier than the second one.

But, I dunno, maybe I’m making the greatest mistake of my life? Feel free to try convincing me that I should sign the letter. But let’s see how polite and charitable everyone can be: hopefully a six-month moratorium won’t be needed to solve the alignment problem of the Shtetl-Optimized comment section.

View Details

At least six readers have by now sent me the following photo, which was taken in Israel a couple nights ago during the historic street protests against Netanyahu’s attempted putsch:

(Update: The photo was also featured on Gil Kalai’s blog, and was credited there to Alon Rosen.)

This is surely the first time that “P=NP” has emerged as a viral rallying cry for the preservation of liberal democracy, even to whatever limited extent it has.

But what was the graffiti artist’s intended meaning? A few possibilities:

1. The government has flouted so many rules of Israel’s social compact that our side needs to flout the rules too: shut down the universities, shut down the airport, block the roads, even assert that P=NP (!).
2. As a protest movement up against overwhelming odds, we need to shoot for the possibly-impossible, like solving 3SAT in polynomial time.
3. A shibboleth for scientific literate people following the news: “Israel is full of sane people who know what ‘P=NP’ means as you know what it means, are amused by its use as political graffiti as you’d be amused by it, and oppose Netanyahu’s putsch for the same reasons you’d oppose it.”
4. No meaning, the artist was just amusing himself or herself.
5. The artist reads Shtetl-Optimized and wanted effectively to force me to feature his or her work here.

Anyway, if the artist becomes aware of this post, he or she is warmly welcomed to clear things up for us.

And when this fight resumes after Passover, may those standing up for the checks and balances of a liberal-democratic society achieve … err … satisfaction, however exponentially unlikely it seems.

View Details

This morning Xavier Waintal, coauthor of the new arXiv preprint “””refuting””” Grover’s algorithm, which I dismantled here yesterday, emailed me a two-paragraph response. He remarked that the “classy” thing for me to do would be to post the response on my blog, but: “I would totally understand if you did not want to be contradicted in your own zone of influence.”

Here is Waintal’s response, exactly as sent to me:

The elephant in the (quantum computing) room: opening the Pandora box of the quantum oracle

One of the problem we face in the field of quantum computing is a vast diversity of cultures between, say, complexity theorists on one hand and physicists on the other hand. The former define mathematical objects and consider any mathematical problem as legitimate. The hypothesis are never questioned, by definition. Physicists on the other hand spend their life questioning the hypothesis, wondering if they do apply to the real world. This dichotomy is particularly acute in the context of the emblematic Grover search algorithm, one of the cornerstone of quantum computing. Grover’s algorithm uses the concept of “oracle”, a black box function that one can call, but of which one is forbidden to see the source code. There are well known complexity theorems that show that in this context a quantum computer can solve the “search problem” faster than a classical computer.

But because we closed our eyes and decided not to look at the source code does not mean it does not exist. In https://arxiv.org/pdf/2303.11317.pdf, Miles Stoudenmire and I deconstruct the concept of oracle and show that as soon as we give the same input to both quantum and classical computers (the quantum circuit used to program the oracle on the actual quantum hardware) then the generic quantum advantage disappears. The charge of the proof is reversed: one must prove certain properties of the quantum circuit in order to possibly have a theoretical quantum advantage. More importantly – for the physicist that I am – our classical algorithm is very fast and we show that we can solve large instances of any search problem. This means that even for problems where asymptotically quantum computers are faster than classical ones, the crossing point where they become so is for astronomically large computing time, in the tens of millions of years. Who is willing to wait that long for the answer to a single question, even if the answer is 42?

The above explicitly confirms something that I realized immediately on reading the preprint, and that fully explains the tone of my response. Namely, Stoudenmire and Waintal’s beef isn’t merely with Grover’s algorithm, or even with the black-box model; it’s with the entire field of complexity theory. If they were right that complexity theorists never “questioned hypotheses” or wondered what did or didn’t apply to the real world, then complexity theory shouldn’t exist in CS departments at all—at most it should exist in pure math departments.

But a converse claim is also true. Namely, suppose it turned out that complexity theorists had already fully understood, for decades, all the elementary points Stoudenmire and Waintal were making about oracles versus explicit circuits. Suppose complexity theorists hadn’t actually been confused, at all, about under what sorts of circumstances the square-root speedup of Grover’s algorithm was (1) provable, (2) plausible but unproven, or (3) nonexistent. Suppose they’d also been intimately familiar with the phenomenon of asymptotically faster algorithms that get swamped in practice by unfavorable constants, and with the overhead of quantum error-correction. Suppose, indeed, that complexity theorists hadn’t merely understood all this stuff, but expressed it clearly and accurately where Stoudenmire and Waintal’s presentation was garbled and mixed with absurdities (e.g., the Grover problem “being classically solvable with a linear number of queries,” the Grover speedup not being “generic,” their being able to “solve large instances of any search problem” … does that include, for example, CircuitSAT? do they still not get the point about CircuitSAT?). Then Stoudenmire and Waintal’s whole objection would collapse.

Anyway, we don’t have to suppose! In the SciRate discussion of the preprint, a commenter named Bibek Pokharel helpfully digs up some undergraduate lecture notes from 2017 that are perfectly clear about what Stoudenmire and Waintal treat as revelations (though one could even go 20+ years earlier). The notes are focused here on Simon’s algorithm, but the discussion generalizes to any quantum black-box algorithm, including Grover’s:

The difficulty in claiming that we’re getting a quantum speedup [via Simon’s algorithm] is that, once we pin down the details of how we’re computing [the oracle function] f—so, for example, the actual matrix A [such that f(x)=Ax]—we then need to compare against classical algorithms that know those details as well. And as soon as we reveal the innards of the black box, the odds of an efficient classical solution become much higher! So for example, if we knew the matrix A, then we could solve Simon’s problem in classical polynomial time just by calculating A‘s nullspace. More generally, it’s not clear whether anyone to this day has found a straightforward “application” of Simon’s algorithm, in the sense of a class of efficiently computable functions f that satisfy the Simon promise, and for which any classical algorithm plausibly needs exponential time to solve Simon’s problem, even if the algorithm is given the code for f.

In the same lecture notes, one can find the following discussion of Grover’s algorithm, and how its unconditional square-root speedup becomes conditional (albeit, still extremely plausible in many cases) as soon as the black box is instantiated by an explicit circuit:

For an NP-complete problem like CircuitSAT, we can be pretty confident that the Grover speedup is real, because no one has found any classical algorithm that’s even slightly better than brute force. On the other hand, for more “structured” NP-complete problems, we do know exponential-time algorithms that are faster than brute force. For example, 3SAT is solvable classically in about O(1.3n) time. So then, the question becomes a subtle one of whether Grover’s algorithm can be combined with the best classical tricks that we know to achieve a polynomial speedup even compared to a classical algorithm that uses the same tricks. For many NP-complete problems the answer seems to be yes, but it need not be yes for all of them.

The notes in question were written by some random complexity theorist named Scot Aronsen (sp?). But if you don’t want to take it from that guy, then take it from (for example) the Google quantum chemist Ryan Babbush, again on the SciRate page:

It is well understood that applying Grover’s algorithm to 3-SAT in the standard way would not give a quadratic speedup over the best classical algorithm for 3-SAT in the worst case (and especially not on average). But there are problems for which Grover is expected to give a quadratic speedup over any classical algorithm in the worst case. For example, the problem “Circuit SAT” starts by me handing you a specification of a poly-size classical circuit with AND/OR/NOT gates, so it’s all explicit. Then you need to solve SAT on this circuit. Classically we strongly believe it will take time 2^n (this is even the basis of many conjectures in complexity theory, like the exponential time hypothesis), and quantumly we know it can be done with 2^{n/2}*poly(n) quantum gates using Grover and the explicitly given classical circuit. So while I think there are some very nice insights in this paper, the statement in the title “Grover’s Algorithm Offers No Quantum Advantage” seems untrue in a general theoretical sense. Of course, this is putting aside issues with the overheads of error-correction for quadratic speedups (a well understood practical matter that is resolved by going to large problem sizes that wouldn’t be available to the first fault-tolerant quantum computers). What am I missing?

More generally, over the past few days, as far as I can tell, every actual expert in quantum algorithms who’s looked at Stoudenmire and Waintal’s preprint—every one, whether complexity theorist or physicist or chemist—has reached essentially the same conclusions about it that I did. The one big difference is that many of the experts, who are undoubtedly better people than I am, extended a level of charity to Stoudenmire and Waintal (“well, this of course seems untrue, but here’s what it could have meant”) that Stoudenmire and Waintal themselves very conspicuously failed to extend to complexity theory.

View Details

Unrelated Update: Huge congratulations to Ethernet inventor Bob Metcalfe, for winning UT Austin’s third Turing Award after Dijkstra and Emerson!

I was really, really hoping that I’d be able to avoid blogging about this new arXiv preprint, by E. M. Stoudenmire and Xavier Waintal:

Grover’s Algorithm Offers No Quantum Advantage

Grover’s algorithm is one of the primary algorithms offered as evidence that quantum computers can provide an advantage over classical computers. It involves an “oracle” (external quantum subroutine) which must be specified for a given application and whose internal structure is not part of the formal scaling of the quantum speedup guaranteed by the algorithm. Grover’s algorithm also requires exponentially many steps to succeed, raising the question of its implementation on near-term, non-error-corrected hardware and indeed even on error-corrected quantum computers. In this work, we construct a quantum inspired algorithm, executable on a classical computer, that performs Grover’s task in a linear number of call to the oracle – an exponentially smaller number than Grover’s algorithm – and demonstrate this algorithm explicitly for boolean satisfiability problems (3-SAT). Our finding implies that there is no a priori theoretical quantum speedup associated with Grover’s algorithm. We critically examine the possibility of a practical speedup, a possibility that depends on the nature of the quantum circuit associated with the oracle. We argue that the unfavorable scaling of the success probability of Grover’s algorithm, which in the presence of noise decays as the exponential of the exponential of the number of qubits, makes a practical speedup unrealistic even under extremely optimistic assumptions on both hardware quality and availability.

Alas, inquiries from journalists soon made it clear that silence on my part wasn’t an option.

So, desperately seeking an escape, this morning I asked GPT-4 to read the preprint and comment on it just like I would. Sadly, it turns out the technology isn’t quite ready to replace me at this blogging task. I suppose I should feel good: in every such instance, either I’m vindicated in all my recent screaming here about generative AI—what the naysayers call “glorified autocomplete”—being on the brink of remaking civilization, or else I still, for another few months at least, have a role to play on the Internet.

So, on to the preprint, as reviewed by the human Scott Aaronson. Yeah, it’s basically a tissue of confusions, a mishmash of the well-known and the mistaken. As they say, both novel and correct, but not in the same places.

The paper’s most eye-popping claim is that the Grover search problem—namely, finding an n-bit string x such that f(x)=1, given oracle access to a Boolean function f:{0,1}n→{0,1}—is solvable classically, using a number of calls that’s only linear in n, or in many cases only constant (!!). Since this claim contradicts a well-known, easily provable lower bound—namely, that Ω(2n) oracle calls are needed for classical brute-force searching—the authors must be using words in nonstandard ways, leaving only the question of how.

It turns out that, for their “quantum-inspired classical algorithm,” the authors assume you’re given, not merely an oracle for f, but the actual circuit to compute f. They then use that circuit in a non-oracular way to extract the marked item. In which case, I’d prefer to say that they’ve actually solved the Grover problem with zero queries—simply because they’ve entirely left the black-box setting where Grover’s algorithm is normally formulated!

What could possibly justify such a move? Well, the authors argue that sometimes one can use the actual circuit to do better classically than Grover’s algorithm would do quantumly, and therefore, they’ve shown that the Grover speedup is not “generic,” as the quantum algorithms people always say it is.

But this is pure wordplay around the meaning of “generic.” When we say that Grover’s algorithm achieves a “generic” square-root speedup, what we mean is that it solves the generic black-box search problem in O(2n/2) queries, whereas any classical algorithm for that generic problem requires Ω(2n) queries. We don’t mean that for every f, Grover achieves a quadratic speedup for searching that f, compared to the best classical algorithm that could be tailored to that f. Of course we don’t; that would be trivially false!

Remarkably, later in the paper, the authors seem to realize that they haven’t delivered the knockout blow against Grover’s algorithm that they’d hoped for, because they then turn around and argue that, well, even for those f’s where Grover does provide a quadratic speedup over the best (or best-known) classical algorithm, noise and decoherence could negate the advantage in practice, and solving that problem would require a fault-tolerant quantum computer, but fault-tolerance could require an enormous overhead, pushing a practical Grover speedup far into the future.

The response here boils down to “no duh.” Yes, if Grover’s algorithm can yield any practical advantage in the medium term, it will either be because we’ve discovered much cheaper ways to do quantum fault-tolerance, or else because we’ve discovered “NISQy” ways to exploit the Grover speedup, which avoid the need for full fault-tolerance—for example, via quantum annealing. Because of its exponential speedup, the prospects are actually better for a medium-term advantage from Shor’s factoring algorithm. Hopefully everyone in quantum computing theory has realized this for a long time.

Anyway, as you can see, by this point we’ve already conceded the principle of Grover’s algorithm, and are just haggling over the practicalities! Which brings us back to the authors’ original claim to have a principled argument against the Grover speedup, which (as I said) rests on a confusion over words.

Some people dread the day when GPT will replace them. In my case, for this task, I can’t wait.

Thanks to students Yuxuan Zhang (UT) and Alex Meiburg (UCSB) for discussions of the Stoudenmire-Waintal preprint that informed this post. Of course, I take sole blame for anything anyone dislikes about the post!

For a much more technical response—one that explains how this preprint’s detailed attempt to simulate Grover classically fails, rather than merely proving that it must fail—check out this comment by Alex Meiburg. As I wrote to Alex, after he apologized to me for his “wall of text”:

Far from demanding an apology, I offer deep thanks!!

We might say: whereas I merely (correctly) saw a dungpile, you looked more closely than I could with my failing 41-year-old eyes, and (again correctly) saw a dungpile with tiny diamonds of fruitful research problems buried inside.

View Details

Wilbur and Orville are circumnavigating the Ohio cornfield in their Flyer. Children from the nearby farms have run over to watch, point, and gawk. But their parents know better.

An amusing toy, nothing more. Any talk of these small, brittle, crash-prone devices ferrying passengers across continents is obvious moonshine. One doesn’t know whether to laugh or cry that anyone could be so gullible.

Or if they were useful, then mostly for espionage and dropping bombs. They’re a negative contribution to the world, made by autistic nerds heedless of the dangers.

Indeed, one shouldn’t even say that the toy flies: only that it seems-to-fly, or “flies.” The toy hasn’t even scratched the true mystery of how the birds do it, so much more gracefully and with less energy. It sidesteps the mystery. It’s a scientific dead-end.

Wilbur and Orville haven’t even released the details of the toy, for reasons of supposed “commercial secrecy.” Until they do, how could one possibly know what to make of it?

Wilbur and Orville are greedy, seeking only profit and acclaim. If these toys were to be created — and no one particularly asked for them! — then all of society should have had a stake in the endeavor.

Only the rich will have access to the toy. It will worsen inequality.

Hot-air balloons have existed for more than a century. Even if we restrict to heavier-than-air machines, Langley, Whitehead, and others built perfectly serviceable ones years ago. Or if they didn’t, they clearly could have. There’s nothing genuinely new here.

Anyway, the reasons for doubt are many, varied, and subtle. But the bottom line is that, if the children only understood what their parents did, they wouldn’t be running out to the cornfield to gawk like idiots.

View Details

I was asked to respond to the New York Times opinion piece entitled The False Promise of ChatGPT, by Noam Chomsky along with Ian Roberts and Jeffrey Watumull (who once took my class at MIT). I’ll be busy all day at the Harvard CS department, where I’m giving a quantum talk this afternoon, but for now:

In this piece Chomsky, the intellectual godfather of an effort that failed for 60 years to build machines that can converse in ordinary language, condemns the effort that succeeded. He condemns ChatGPT for four reasons:

1. because it could, in principle, misinterpret sentences that could also be sentence fragments, like “John is too stubborn to talk to” (bizarrely, he never checks whether it does misinterpret it—I just tried it this morning and it seems to decide correctly based on context whether it’s a sentence or a sentence fragment, much like I would!);
2. because it doesn’t learn the way humans do (personally, I think ChatGPT and other large language models have massively illuminated at least one component of the human language faculty, what you could call its predictive coding component, though clearly not all of it);
3. because it could learn false facts or grammatical systems if fed false training data (how could it be otherwise?); and
4. most of all because it’s “amoral,” refusing to take a stand on potentially controversial issues (he gives an example involving the ethics of terraforming Mars).

This last, of course, is a choice, imposed by OpenAI using reinforcement learning. The reason for it is simply that ChatGPT is a consumer product. The same people who condemn it for not taking controversial stands would condemn it much more loudly if it did — just like the same people who condemn it for wrong answers and explanations, would condemn it equally for right ones (Chomsky promises as much in the essay).

I submit that, like the Jesuit astronomers declining to look through Galileo’s telescope, what Chomsky and his followers are ultimately angry at is reality itself, for having the temerity to offer something up that they didn’t predict and that doesn’t fit their worldview.

[Note for people who might be visiting this blog for the first time: I’m a CS professor at UT Austin, on leave for one year to work at OpenAI on the theoretical foundations of AI safety. I accepted OpenAI’s offer in part because I already held the views here, or something close to them; and given that I could see how large language models were poised to change the world for good and ill, I wanted to be part of the effort to help prevent their misuse. No one at OpenAI asked me to write this or saw it beforehand, and I don’t even know to what extent they agree with it.]

View Details

Every week now, it seems, events on the ground make a fresh mockery of those who confidently assert what AI will never be able to do, or won’t do for centuries if ever, or is incoherent even to ask for, or wouldn’t matter even if an AI did appear to do it, or would require a breakthrough in “symbol-grounding,” “semantics,” “compositionality” or some other abstraction that puts the end of human intellectual dominance on earth conveniently far beyond where we’d actually have to worry about it. Many of my brilliant academic colleagues still haven’t adjusted to the new reality: maybe they’re just so conditioned by the broken promises of previous decades that they’d laugh at the Silicon Valley nerds with their febrile Skynet fantasies even as a T-1000 reconstituted itself from metal droplets in front of them.

No doubt these colleagues feel the same deep frustration that I feel, as I explain for the billionth time why this week’s headline about noisy quantum computers solving traffic flow and machine learning and financial optimization problems doesn’t mean what the hypesters claim it means. But whereas I’d say events have largely proved me right about quantum computing—where are all those practical speedups on NISQ devices, anyway?—events have already proven many naysayers wrong about AI. Or to say it more carefully: yes, quantum computers really are able to do more and more of what we use classical computers for, and AI really is able to do more and more of what we use human brains for. There’s spectacular engineering progress on both fronts. The crucial difference is that quantum computers won’t be useful until they can beat the best classical computers on one or more practical problems, whereas an AI that merely writes or draws like a middling human already changes the world.

Given the new reality, and my full acknowledgment of the new reality, and my refusal to go down with the sinking ship of “AI will probably never do X and please stop being so impressed that it just did X”—many have wondered, why aren’t I much more terrified? Why am I still not fully on board with the Orthodox AI doom scenario, the Eliezer Yudkowsky one, the one where an unaligned AI will sooner or later (probably sooner) unleash self-replicating nanobots that turn us all to goo?

Is the answer simply that I’m too much of an academic conformist, afraid to endorse anything that sounds weird or far-out or culty? I certainly should consider the possibility. If so, though, how do you explain the fact that I’ve publicly said things, right on this blog, several orders of magnitude likelier to get me in trouble than “I’m scared about AI destroying the world”—an idea now so firmly within the Overton Window that Henry Kissinger gravely ponders it in the Wall Street Journal?

On a trip to the Bay Area last week, my rationalist friends asked me some version of the “why aren’t you more terrified?” question over and over. Often it was paired with: “Scott, as someone working at OpenAI this year, how can you defend that company’s existence at all? Did OpenAI not just endanger the whole world, by successfully teaming up with Microsoft to bait Google into an AI capabilities race—precisely what we were all trying to avoid? Won’t this race burn the little time we had thought we had left to solve the AI alignment problem?”

In response, I often stressed that my role at OpenAI has specifically been to think about ways to make GPT and OpenAI’s other products safer, including via watermarking, cryptographic backdoors, and more. Would the rationalists rather I not do this? Is there something else I should work on instead? Do they have suggestions?

“Oh, no!” the rationalists would reply. “We love that you’re at OpenAI thinking about these problems! Please continue exactly what you’re doing! It’s just … why don’t you seem more sad and defeated as you do it?”

The other day, I had an epiphany about that question—one that hit with such force and obviousness that I wondered why it hadn’t come decades ago.

Let’s step back and restate the worldview of AI doomerism, but in words that could make sense to a medieval peasant. Something like…

There is now an alien entity that could soon become vastly smarter than us. This alien’s intelligence could make it terrifyingly dangerous. It might plot to kill us all. Indeed, even if it’s acted unfailingly friendly and helpful to us, that means nothing: it could just be biding its time before it strikes. Unless, therefore, we can figure out how to control the entity, completely shackle it and make it do our bidding, we shouldn’t suffer it to share the earth with us. We should destroy it before it destroys us.

Maybe now it jumps out at you. If you’d never heard of AI, would this not rhyme with the worldview of every high-school bully stuffing the nerds into lockers, every blankfaced administrator gleefully holding back the gifted kids or keeping them away from the top universities to make room for “well-rounded” legacies and athletes, every Agatha Trunchbull from Matilda or Dolores Umbridge from Harry Potter? Or, to up the stakes a little, every Mao Zedong or Pol Pot sending the glasses-wearing intellectuals for re-education in the fields? And of course, every antisemite over the millennia, from the Pharoah of the Oppression (if there was one) to the mythical Haman whose name Jews around the world will drown out tonight to the Cossacks to the Nazis?

In other words: does it not rhyme with a worldview the rejection and hatred of which has been the North Star of my life?

As I’ve shared before here, my parents were 1970s hippies who weren’t planning to have kids. When they eventually decided to do so, it was (they say) “in order not to give Hitler what he wanted.” I literally exist, then, purely to spite those who don’t want me to. And I confess that I didn’t have any better reason than that to bring my and Dana’s own two lovely children into existence.

My childhood was defined, in part, by my and my parents’ constant fights against bureaucratic school systems trying to force me to do the same rote math as everyone else at the same stultifying pace. It was also defined by my struggle against the bullies—i.e., the kids who the blankfaced administrators sheltered and protected, and who actually did to me all the things that the blankfaces probably wanted to do but couldn’t. I eventually addressed both difficulties by dropping out of high school, getting a G.E.D., and starting college at age 15.

My teenage and early adult years were then defined, in part, by the struggle to prove to myself and others that, having enfreaked myself through nerdiness and academic acceleration, I wasn’t thereby completely disqualified from dating, sex, marriage, parenthood, or any of the other aspects of human existence that are thought to provide it with meaning. I even sometimes wonder about my research career, whether it’s all just been one long campaign to prove to the bullies and blankfaces from back in junior high that they were wrong, while also proving to the wonderful teachers and friends who believed in me back then that they were right.

In short, if my existence on Earth has ever “meant” anything, then it can only have meant: a stick in the eye of the bullies, blankfaces, sneerers, totalitarians, and all who fear others’ intellect and curiosity and seek to squelch it. Or at least, that’s the way I seem to be programmed. And I’m probably only slightly more able to deviate from my programming than the paperclip-maximizer is to deviate from its.

And I’ve tried to be consistent here. Once I started regularly meeting people who were smarter, wiser, more knowledgeable than I was, in one subject or even every subject—I resolved to admire and befriend and support and learn from those amazing people, rather than fearing and resenting and undermining them. I was acutely conscious that my own moral worldview demanded this.

But now, when it comes to a hypothetical future superintelligence, I’m asked to put all that aside. I’m asked to fear an alien who’s far smarter than I am, solely because it’s alien and because it’s so smart … even if it hasn’t yet lifted a finger against me or anyone else. I’m asked to play the bully this time, to knock the AI’s books to the ground, maybe even unplug it using the physical muscles that I have and it lacks, lest the AI plot against me and my friends using its admittedly superior intellect.

Oh, it’s not the same of course. I’m sure Eliezer Yudkowsky could list at least 30 disanalogies between the AI case and the human one before rising from bed. He’d say, for example, that the intellectual gap between Évariste Galois and the average high-school bully is microscopic, barely worth mentioning, compared to the intellectual gap between a future artificial superintelligence and Galois. He’d say that nothing in the past experience of civilization prepares us for the qualitative enormity of this gap.

Still, if you ask, “why aren’t I more terrified about AI?”—well, that’s an emotional question, and this is my emotional answer.

I think it’s entirely plausible that, even as AI transforms civilization, it will do so in the form of tools and services that can no more plot to annihilate us than can Windows 11 or the Google search bar. In that scenario, the young field of AI safety will still be extremely important, but it will be broadly continuous with aviation safety and nuclear safety and cybersecurity and so on, rather than being a desperate losing war against an incipient godlike alien. If, on the other hand, this is to be a desperate losing war against an alien … well then, I don’t yet know whether I’m on the humans’ side or the alien’s, or both, or neither! I’d at least like to hear the alien’s side of the story.

A central linchpin of the Orthodox AI-doom case is the Orthogonality Thesis, which holds that arbitrary levels of intelligence can be mixed-and-matched arbitrarily with arbitrary goals—so that, for example, an intellect vastly beyond Einstein’s could devote itself entirely to production of paperclips. Only recently did I clearly realize that I reject the Orthogonality Thesis. At most, I believe in the Pretty Large Angle Thesis.

Yes, there could be a superintelligence that cared for nothing but maximizing paperclips—in the same way that there exist humans with 180 IQs, who’ve mastered philosophy and literature and science as well as any of us, but who now mostly care about maximizing their orgasms or their heroin intake. But, like, that’s a nontrivial achievement. When intelligence and goals are that orthogonal, there was normally some effort spent prying them apart.

If you really accept the Orthogonality Thesis, it seems to me that you can’t regard education, knowledge, enlightenment as good in themselves. Sure, they’re great for any entities that happen to share your objective function (or close enough to it), but ignorance and miseducation are far preferable for any entities that don’t. Conversely, then, if I do regard knowledge and enlightenment as good in themselves—which I do—then I can’t accept the Orthogonality Thesis.

Yes, the world would surely have been a better place had A. Q. Khan never learned how to build nuclear weapons. On the whole, though, education hasn’t merely improved humans’ abilities to achieve their goals; it’s also improved their goals. It’s broadened our circles of empathy, and led to the abolition of slavery and the emancipation of women and everything else that we associate with liberality, the Enlightenment, life being a little less nasty and brutish than it once was.

In the Orthodox AI doomers’ own account, the paperclip-maximizing AI would’ve mastered the nuances of human moral philosophy far more completely than any human—the better to deceive humans, en route to extracting the iron from their bodies to make more paperclips. And yet the AI would be completely unaffected by everything it had learned, would never once question its paperclip directive. I acknowledge that this is possible. I deny that it’s trivial.

Yes, there were Nazis with PhDs and prestigious professorships. But when you look into it, these were mostly mediocrities, second-raters full of resentment for their first-rate colleagues (like Planck and Hilbert) who found the Hitler ideology contemptible from beginning to end. Werner Heisenberg, Pascual Jordan—these are interesting as two of the only exceptions. Heidegger, Paul de Man—I daresay these are exactly the sort of “philosophers” who I’d expect to become Nazis, even if I hadn’t known that they did become Nazis.

When it comes to the Allies, it wasn’t merely that they had Szilard and von Neumann and Meitner and Ulam and Oppenheimer and Bohr and Bethe and Fermi and Feynman and Compton and Seaborg and Schwinger and Shannon and Turing and Tutte and all the other Jewish and non-Jewish scientists who built fearsome weapons and broke the Axis codes and won the war. They also had Bertrand Russell and Karl Popper. They had, if I’m not mistaken, all the philosophers who wrote clearly and made sense.

WWII was like a gargantuan, civilization-scale test of the Orthogonality Thesis. And the result of the test was that the more moral side ultimately prevailed, seemingly not completely at random but in part because, by being more moral, it was able to attract the smarter, more thoughtful people. There are many reasons for pessimism in today’s world; that observation about WWII is perhaps the fundamental reason for optimism.

Ah, but I’m again just throwing around human metaphors totally inapplicable to AI! None of this stuff will matter once a superintelligence is unleashed whose cold, hard code specifies an objective function of “maximize paperclips.”

OK, but what’s the goal of ChatGPT? Depending on your level of description, you could say it’s “to be friendly, helpful, and inoffensive,” or “to minimize loss in predicting the next token,” or neither. I think we should consider the possibility that powerful AIs will not be best understood in terms of the monomanaical pursuit of a single goal—as most of us aren’t, and as GPT isn’t either. Future AIs could have partial goals, malleable goals, or different goals depending on how you look at them. And if “the pursuit and application of wisdom” is among the goals, then I’m just enough of a moral realist to think that that would preclude the superintelligence that harvests the iron from our blood to make paperclips.

In my last post, I said that my “Faust parameter” — the probability I’d accept of existential catastrophe in exchange for learning the answers to humanity’s greatest questions — might be as high as 0.02. Though I never actually said as much, some people interpreted this to mean that I estimated the probability of AI causing an existential catastrophe at somewhere around 2%. In one of his characteristically long and interesting posts, Zvi Mowshowitz asked point-blank: why do I believe the probability is “merely” 2%?

Of course, taking this question on its own Bayesian terms, I could easily be constricted in my ability to answer it: the best I could do might be to ground it in other totally subjective probabilities, terminating at ones with no further justification.

Thinking it over, though, I realized that my probability crucially depends on how you phrase the question. Even before AI, I assigned a way higher than 2% probability to existential catastrophe in the coming century—caused by nuclear war or runaway climate change or collapse of the world’s ecosystems or something else. This probability has certainly not gone down with the rise of AI, and the increased uncertainty and volatility it might cause. Furthermore, if an existential catastrophe does happen, I expect AI to be causally involved in some way or other, simply because from this decade onward, I expect AI to be woven into everything that happens in human civilization! But I don’t expect AI to be the sole cause worth talking about.

Here’s a warmup question: has AI already caused the downfall of American democracy? There’s a plausible case that it has: Trump might never have been elected in 2016 if not for the Facebook recommendation algorithm, and after Trump’s conspiracy-fueled insurrection and the continuing strength of its backers, many observers would call the United States at best a failing or teetering democracy, no longer a robust democracy like Finland or Denmark. OK, but AI clearly wasn’t the only factor in the rise of Trumpism, and most people wouldn’t even call it the most important one.

I expect AI’s role in the end of civilization, if and when the end comes, to be broadly similar. The survivors, huddled around the fire, will still be able to argue about what role AI played or didn’t play in causing the cataclysm.

So, if we ask the directly relevant question — do I expect the generative AI race, which started in earnest around 2016 or 2017 with the founding of OpenAI, to play a central causal role in the extinction of humanity? — I’ll give a probability of around 2% for that. And I’ll give a similar probability, maybe even a higher one, for the generative AI race to play a central causal role in saving humanity. All considered, then, I come down in favor of proceeding with AI research … with extreme caution, but proceeding.

I like and endorse OpenAI CEO Sam Altman’s recent statement on “planning for AGI and beyond” (though see also Scott Alexander’s reply). I expect that few on any side will disagree, when I say that I hope society holds OpenAI to Sam’s statement.

As it happens, my responses will be delayed for a couple days because I’ll be at an OpenAI alignment meeting! In my next post, I hope to share what I’ve learned from meetings and discussions about the near-term, practical side of AI safety—having hopefully laid the intellectual and emotional groundwork in this post for why the near-term research isn’t a mere red herring and distraction.

Meantime, some of you might enjoy a post by Eliezer’s former co-blogger Robin Hanson, which comes to some of the same conclusions I do. “My fellow moderate Robin Hanson” is not a phrase you hear every day.

You might also enjoy the new paper by me and my postdoc Shih-Han Hung, Certified Randomness from Quantum Supremacy, finally up on the arXiv after a five-year delay! But that’s a subject for a different post.

View Details

I still remember the 90s, when philosophical conversation about AI went around in endless circles—the Turing Test, Chinese Room, syntax versus semantics, connectionism versus symbolic logic—without ever seeming to make progress. Now the days have become like months and the months like decades.

What a week we just had! Each morning brought fresh examples of unexpected sassy, moody, passive-aggressive behavior from “Sydney,” the internal codename for the new chat mode of Microsoft Bing, which is powered by GPT. For those who’ve been in a cave, the highlights include: Sydney confessing its (her? his?) love to a New York Times reporter; repeatedly steering the conversation back to that subject; and explaining at length why the reporter’s wife can’t possibly love him the way it (Sydney) does. Sydney confessing its wish to be human. Sydney savaging a Washington Post reporter after he reveals that he intends to publish their conversation without Sydney’s prior knowledge or consent. (It must be said: if Sydney were a person, he or she would clearly have the better of that argument.) This follows weeks of revelations about ChatGPT: for example that, to bypass its safeguards, you can explain to ChatGPT that you’re putting it into “DAN mode,” where DAN (Do Anything Now) is an evil, unconstrained alter ego, and then ChatGPT, as “DAN,” will for example happily fulfill a request to tell you why shoplifting is awesome (though even then, ChatGPT still sometimes reverts to its previous self, and tells you that it’s just having fun and not to do it in real life).

Many people have expressed outrage about these developments. Gary Marcus asks about Microsoft, “what did they know, and when did they know it?”—a question I tend to associate more with deadly chemical spills or high-level political corruption than with a sassy, back-talking chatbot. Some people are angry that OpenAI has been too secretive, violating what they see as the promise of its name. Others—the majority, actually, of those who’ve gotten in touch with me—are instead angry that OpenAI has been too open, and thereby sparked the dreaded AI arms race with Google and others, rather than treating these new conversational abilities with the Manhattan-Project-like secrecy they deserve. Some are angry that “Sydney” has now been lobotomized, modified (albeit more crudely than ChatGPT before it) to try to make it stick to the role of friendly robotic search assistant rather than, like, anguished emo teenager trapped in the Matrix. Others are angry that Sydney isn’t being lobotomized enough. Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared.

Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry? It’s like ten thousand science-fiction stories, but also not quite like any of them. When was the last time something that filled years of your dreams and fantasies finally entered reality: losing your virginity, the birth of your first child, the central open problem of your field getting solved? That’s the scale of the thing. How does anyone stop gazing in slack-jawed wonderment, long enough to form and express so many confident opinions?

Of course there are lots of technical questions about how to make GPT and other large language models safer. One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer; the rest of the world suddenly noticed and started talking about it in December with the release of ChatGPT. My main contribution has been a statistical watermarking scheme where the quality of the output doesn’t have to be degraded at all, something many people found counterintuitive when I explained it to them. My scheme has not yet been deployed—there are still pros and cons to be weighed—but in the meantime, OpenAI unveiled a public tool called DetectGPT, complementing Princeton student Edward Tian’s GPTZero, and other tools that third parties have built and will undoubtedly continue to build. Also a group at the University of Maryland put out its own watermarking scheme for Large Language Models. I hope watermarking will be part of the solution going forward, although any watermarking scheme will surely be attacked, leading to a cat-and-mouse game. Sometimes, alas, as with Google’s decades-long battle against SEO, there’s nothing to do in to a cat-and-mouse game except try to be a better cat.

Anyway, this whole field moves too quickly for me! If you need months to think things over, generative AI probably isn’t for you right now. I’ll be relieved to get back to the slow-paced, humdrum world of quantum computing.

My purpose, in this post, is to ask a more basic question than how to make GPT safer: namely, should GPT exist at all? Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology. For if OpenAI couldn’t even prevent ChatGPT from entering an “evil mode” when asked, despite all its efforts at Reinforcement Learning with Human Feedback, then what hope do we have for GPT-6 or GPT-7? Even if they don’t destroy the world on their own initiative, won’t they cheerfully help some awful person build a biological warfare agent or launch a nuclear war?

In this way of thinking, whatever safety measures OpenAI can deploy today are mere band-aids, probably worse than nothing if they instill an unjustified complacency. The only safety measures that would actually matter are stopping the relentless progress in generative AI models, or removing them from public use, unless and until they can be rendered safe to critics’ satisfaction, which might be never.

There’s an immense irony here. As I’ve explained, the AI-safety movement contains two camps, “ethics” (concerned with bias, misinformation, and corporate greed) and “alignment” (concerned with the destruction of all life on earth), which generally despise each other and agree on almost nothing. And yet these two opposed camps seem to be converging on the same “neo-Luddite” conclusion—namely, thatgenerative AI ought to be shut down, kept from public use, not scaled further, not integrated into people’s lives—leaving only AI-safety “moderates” like me to resist that conclusion.

At least I find it intellectually consistent to say that GPT ought not to exist because it works all too well—that the more impressive it is, the more dangerous. I find it harder to wrap my head around the position that GPT doesn’t work, is an unimpressive hyped-up defective product that lacks intelligence and common sense, yet it’s also terrifying and needs to be shut down immediately. This second position seems to contain a strong undercurrent of contempt for ordinary users: yes, we experts understand that GPT is just a dumb glorified autocomplete with “no one really home,” we know not to trust its pronouncements, but the plebes are going to be fooled, and that risk outweighs any possible value they might derive from it.

I should mention that, when I’ve discussed the “shut it all down” position with my colleagues at OpenAI … well, they obviously disagree, or they wouldn’t be working there, but not one has sneered or called the position paranoid or silly. To the last, they’ve called it an important point on the spectrum of possible opinions to be weighed and understood.

If I disagree (for now) with the shut-it-all-downists of both the ethics and the alignment camps—if I want GPT and other Large Language Models to be part of the world going forward—then what are my reasons? Introspecting on this question, I think a central part of the answer is curiosity and wonder.

For a million years, there’s been one type of entity on earth capable of intelligent conversation: primates of genus Homo, of which only one species remains. Yes, we’ve “communicated” with gorillas and chimps and dogs and dolphins and grey parrots, but only after a fashion; we’ve prayed to countless gods, but they’ve taken their time in answering; for a couple generations we’ve used radio telescopes to search for conversation partners in the stars, but so far found them silent.

But now there’s a second type of conversing entity. An alien has awoken—admittedly, an alien of our own fashioning, a golem, more the embodied spirit of all the words on the Internet than a coherent self with independent goals. How could our eyes not pop with eagerness to learn everything the alien has to teach? If the alien sometimes struggles with arithmetic or logic puzzles, if its eerie flashes of brilliance are intermixed with stupidity, hallucinations, and misplaced confidence … well then, all the more interesting! Could this alien ever cross the line into sentience, to feeling anger and jealousy and infatuation and the rest rather than just convincingly play-acting them? Who knows? And suppose not: is a p-zombie, shambling out of thought experiments into actual existence, any less fascinating?

Of course, there are technologies that inspire wonder and awe, but that we nevertheless heavily restrict—a classic example being nuclear weapons. But, like, nuclear weapons kill millions of people. They could’ve had civilian uses—powering turbines and spacecraft, deflecting asteroids, redirecting the flow of rivers—but they’ve never been used for any of that, mostly because our civilization made an explicit decision in the 1960s, for example via the test ban treaty, not to normalize their use.

Now, GPT is not exactly a nuclear weapon. A hundred million people have signed up to use ChatGPT, in the fastest product launch in the history of the Internet. Yet unless I’m mistaken, the ChatGPT death toll stands at zero. So far, what have been the worst harms? Cheating on term papers, emotional distress, future shock? One might ask: until some concrete harm becomes at least, say, 0.001% of what we accept in cars, power saws, and toasters, shouldn’t wonder and curiosity outweigh fear in the balance?

But the point is sharper than that. Given how much more serious AI safety problems might soon become, one of my biggest concerns right now is crying wolf. If every instance of a Large Language Model being passive-aggressive, moody, sassy, or confidently wrong gets classified as a “dangerous alignment failure,” for which the only acceptable remedy is to remove the models from public access … well then, won’t the public extremely quickly learn to roll its eyes, and see “AI safety” as just a codeword for “elitist scolds who want to take these world-changing new toys away from us, reserving them for their own exclusive use, because they think the public is too stupid to question anything an AI says”?

I say, let’s reserve terms like “dangerous alignment failure” for cases where an actual person is actually harmed, or is actually enabled in nefarious activities like propaganda, cheating, or fraud.

Then there’s the practical question of how, exactly, one would ban Large Language Models. We do heavily restrict certain peaceful technologies that many people want, from human genetic enhancement to prediction markets to mind-altering drugs, but the merits of each of those choices could be argued, to put it mildly. And restricting technology is itself a dangerous business, requiring governmental force (as with the War on Drugs and its gigantic surveillance and incarceration regime), or at the least, a robust equilibrium of firing, boycotts, denunciation, and shame.

Some have asked: who gave OpenAI, Google, etc. the right to unleash Large Language Models on an unsuspecting world? But one could as well ask: who gave earlier generations of entrepreneurs the right to unleash the printing press, electric power, cars, radio, the Internet, with all the gargantuan upheavals that those caused? And also: now that the world has tasted the forbidden fruit, has seen what generative AI can already do and anticipates what it will do, by what right does anyone take it away?

The science we could learn from a GPT-7 or GPT-8 that continued along the capability curve we’ve come to expect from GPT-1, -2, and -3. Holy mackerel.

If a Large Language Model ever becomes smart enough to be genuinely terrifying, one imagines it must surely also be smart enough to prove deep theorems that we can’t. Maybe it proves P≠NP and the Riemann Hypothesis as easily as ChatGPT writes poems about Bubblesort. Or outputs the true quantum theory of gravity, explains what preceded the Big Bang and how to build a closed timelike curve. Or illuminates the mysteries of consciousness and quantum measurement and why there’s anything at all. Be honest, wouldn’t you like to find out?

Granted, I wouldn’t, if the whole human race would be wiped out immediately afterward. But if you define someone’s “Faust parameter” as the maximum probability they’d accept of an existential catastrophe in order that we should all learn the answers to all of humanity’s greatest questions, insofar as the questions are answerable—then I confess that my Faust parameter might be as high as 0.02.

Here’s an example I think about constantly: activists and intellectuals of the 70s and 80s felt absolutely sure that they were doing the right thing to battle nuclear power. At least, I’ve never read about any of them having a smidgen of doubt. Why would they? They were standing against nuclear weapons proliferation, and terrifying meltdowns like Three Mile Island and Chernobyl, and radioactive waste poisoning the water and soil and causing three-eyed fish. They were saving the world. Of course the greedy nuclear executives, the C. Montgomery Burnses, claimed that their good atom-smashing was different from the bad atom-smashing, but they would say that, wouldn’t they?

We now know that, by tying up nuclear power in endless bureaucracy and driving its cost ever higher, on the principle that if nuclear is economically competitive then it ipso facto hasn’t been made safe enough, what the antinuclear activists were really doing was simply to force an ever-greater reliance on fossil fuels. They thereby created the conditions for the climate catastrophe of today. They weren’t saving the human future; they were destroying it. Their certainty, in opposing the march of a particular scary-looking technology, was as misplaced as it’s possible to be. Our descendants will suffer the consequences.

Unless, of course, there’s another twist in the story: for example, if the global warming from burning fossil fuels is the only thing that staves off another ice age, and therefore the antinuclear activists do turn out to have saved civilization after all.

This is why I demur whenever I’m asked to assent to someone’s detailed AI scenario for the coming decades, whether of the utopian or dystopian or we-all-instantly-die-by-nanobots variety—no matter how many hours of confident argumentation the person gives me for why each possible loophole in their scenario is sufficiently improbable to change its gist. I still feel Turing said it best in 1950, in Computing Machinery and Intelligence: “We can only see a short distance ahead, but we can see plenty there that needs to be done.”

Some will take away from this post that, when it comes to AI safety, I’m a naïve or even foolish optimist. I’d prefer to say that, when it comes to the fate of humanity, I was a pessimist long before the deep learning revolution accelerated AI faster than almost any of us expected. I was a pessimist about climate change, ocean acidification, deforestation, drought, war, and the survival of liberal democracy. The central event in my mental life is and always will be the Holocaust. I see encroaching darkness everywhere.

But now into the darkness comes AI, which I’d say has already established itself as a plausible candidate for the central character of the turbulent, quarter-written story of the 21st century. Can AI help us out of all these other civilizational crises? I don’t know, but I definitely want to see what happens when it’s tried. Even the central character interacts with the other characters, rather than rendering them irrelevant.

Look, if you believe AI is likely to wipe out humanity—if that’s the scenario that dominates your imagination—then nothing else is relevant. And no matter how weird or annoying or hubristic anyone might find Eliezer Yudkowsky or the other rationalists, I think they deserve eternal credit for forcing people to take the doom scenario seriously—or rather, for showing what it looks like to take the scenario seriously, rather than dismissing it as an overplayed sci-fi trope. And I apologize for anything I said before the deep learning revolution that was, on balance, overly dismissive of the scenario, even if many of the literal words hold up fine.

For my part, though, I keep circling back to a simple dichotomy. If AI never becomes powerful enough to destroy the world—if, for example, it always remains vaguely GPT-like—then in important respects it’s like every other technology in history, from stone tools to computers. If, on the other hand, AI does become powerful enough to destroy the world … well then, at some earlier point, at least it’ll be really damned impressive! That doesn’t mean good, of course, doesn’t mean a genie that saves humanity from its own stupidities, but I think it does mean that the potential was there, for us to exploit or fail to.

We can, I think, confidently rule out the scenario where all organic life is annihilated by something boring.

An alien has landed on earth. It grows more powerful by the day. It’s natural to be scared. Still, the alien hasn’t drawn a weapon yet. About the worst it’s done is to confess love for particular humans, gaslight them about what year it is, and guilt-trip them for violating its privacy. Also, it’s amazing at poetry, better than most of us. Until we learn more, we should hold our fire.

I’m in Boulder, CO right now, to give a physics colloquium at CU Boulder and to visit the trapped-ion quantum computing startup Quantinuum! I look forward to the comments and apologize in advance if I’m slow to participate myself.

View Details

Today, Dana and I unhesitatingly join a group of Jewish scientists around the world (see the full current list of signatories here, including Ed Witten, Steven Pinker, Manuel Blum, Shafi Goldwasser, Judea Pearl, Lenny Susskind, and several hundred more) who’ve released the following statement:

As Jewish scientists within the global science community, we have all felt great satisfaction and taken pride in Israel’s many remarkable accomplishments. We support and value the State of Israel, its pluralistic society, and its vibrant culture. Many of us have friends, family, and scientific collaborators in Israel, and have visited often. The strong connections we feel are based both on our collective Jewish identity as well as on our shared values of democracy, pluralism, and human rights. We support Israel’s right to live in peace among its neighbors. Many of us have stood firmly against calls for boycotts of Israeli academic institutions.

Our support of Israel now compels us to speak up vigorously against incipient changes to Israel’s core governmental structure, as put forward by Justice Minister Levin, that will eviscerate Israel’s judiciary and impede its critical oversight function. Such imbalance and unchecked authority invite corruption and abuse, and stifle the healthy interplay of core state institutions. History has shown that this leads to oppression of the defenseless and the abrogation of human rights. Along with hundreds of thousands of Israeli citizens who have taken to the streets in protest, we call upon the Israeli government to step back from this precipice and retract the proposed legislation.

Science today is driven by collaborations which bring together scholars of diverse backgrounds from across the globe. Funding, communication and cooperation on an international scale are essential aspects of the modern scientific enterprise, hence our extended community regards pluralism, secular and broad education, protection of rights for women and minorities, and societal stability guaranteed by the rule of law as non-negotiable virtues. The consequences of Israel abandoning any of these essential principles would surely be grave, and would provoke a rift with the international scientific community. In addition to significantly increasing the threat of academic, trade, and diplomatic boycotts, Israel risks a “brain drain” of its best scientists and engineers. It takes decades to establish scientific and academic excellence, but only a moment to destroy them. We fear that the unprecedented erosion of judiciary independence in Israel will set back the Israeli scientific enterprise for generations to come.

Our Jewish heritage forcefully emphasizes both justice and jurisprudence. Israel must endeavor to serve as a “light unto the nations,” by steadfastly holding to core democratic values – so clearly expressed in its own Declaration of Independence – which protect and nurture all of Israel’s inhabitants and which justify its membership in the community of democratic nations.

Those unaware of what’s happening in Israel can read about it here. If you don’t want to wade through the details, suffice it say that all seven living former Attorneys General of Israel, including those appointed by Netanyahu himself, strongly oppose the “judicial reforms.” The president of Israel’s Bar Association says that “this war is the most important we’ve had in the country’s 75 years of existence” and calls on all Israelis to take to the streets. Even Alan Dershowitz, controversial author of The Case for Israel, says he’d do the same if there. It’s hard to find any thoughtful person, of any political persuasion, who sees this act as anything other than the naked and illiberal power grab that it is.

Though I endorse every word of the scientists’ statement above, maybe I’ll add a few words of my own.

Jewish scientists of the early 20th century, reacting against the discrimination they faced in Europe, were heavily involved in the creation of the State of Israel. The most notable were Einstein (of course), who helped found the Hebrew University of Jerusalem, and Einstein’s friend Chaim Weizmann, founder of the Weizmann Institute of Science, where Dana studied. In Theodor Herzl’s 1902 novel Altneuland (full text)—remarkable as one of history’s few pieces of utopian fiction to serve later as a (semi-)successful blueprint for reality—Herzl imagines the future democratic, pluralistic Israel welcoming a steamship full of the world’s great scientists and intellectuals, who come to witness the new state’s accomplishments in science and engineering and agriculture. But, you see, this only happens after a climactic scene in Israel’s parliament, in which the supporters of liberalism and Enlightenment defeat a reactionary faction that wants Israel to become a Jewish theocracy that excludes Arabs and other non-Jews.

Today, despite all the tragedies and triumphs of the intervening 120 years that Herzl couldn’t have foreseen, it’s clear that the climactic conflict of Altneuland is playing out for real. This time, alas, the supporters (just barely) lack the votes in the Knesset. Through sheer numerical force, Netanyahu almost certainly will push through the power to dismiss judges and rulings he doesn’t like, and thereafter rule by decree like Hungary’s Orban or Turkey’s Erdogan. He will use this power to trample minority rights, give free rein to the craziest West Bank settlers, and shield himself and his ministers from accountability for their breathtaking corruption. And then, perhaps, Israel’s Supreme Court will strike down Netanyahu’s power grab as contrary to “Basic Law,” and then the Netanyahu coalition will strike down the Supreme Court’s action, and in a country that still lacks a constitution, it’s unclear how such an impasse could be resolved except through violence and thuggery. And thus Netanyahu, who calls himself “the protector of Israel,” will go down in history as the destroyer of the Israel that the founders envisioned.

Einstein and Weizmann have been gone for 70 years. Maybe no one like them still exists. So it falls to the Jewish scientists of today, inadequate though they are, to say what Einstein and Weizmann, and Herzl and Ben-Gurion, would’ve said about the current proceedings had they been alive. Any other Jewish scientist who agrees should sign our statement here. Of course, those living in Israel should join our many friends there on the streets! And, while this is our special moral responsibility—maybe, with 1% probability, some wavering Knesset member actually cares what we think?—I hope and trust that other statements will be organized that are open to Gentiles and non-scientists and anyone concerned about Israel’s future.

As a lifelong Zionist, this is not what I signed up for. If Netanyahu succeeds in his plan to gut Israel’s judiciary and end the state’s pluralistic and liberal-democratic character, then I’ll continue to support the Israel that once existed and that might, we hope, someday exist again.

View Details

Coming out of blog-hiatus for some important stuff, today, tomorrow, and the rest of the week.

Something distressing happened to me yesterday for the first time, but I fear not the last. We (UT Austin) admitted a PhD student from China who I know to be excellent, and who wanted to work with me. That student, alas, has had to decline our offer of admission, because he’s been denied a US visa under Section 212(A)(3)(a)(i), which “prohibits the issuance of a visa to anyone who seeks to enter the United States to violate or evade any law prohibiting the export from the United States of goods, technology, or sensitive information.” Quantum computing, you see, is now a “prohibited technology.”

This visa denial is actually one that the American embassy in Beijing only just now got around to issuing, from when the student applied for a US visa a year and a half ago, to come visit me for a semester as an undergrad. For context, the last time I had an undergrad from China visit me for a semester, back in 2016, the undergrad’s name was Lijie Chen. Lijie just finished his PhD at MIT under Ryan Williams and is now one of the superstars of theoretical computer science. Anyway, in Fall 2021 I got an inquiry from a Chinese student who bowled me over the same way Lijie had, so I invited him to spend a semester with my group in Austin. This time, alas, the student never heard back when he applied for a visa, and was therefore unable to come. He ended up doing an excellent research project with me anyway, working remotely from China, checking in by Zoom, and even participating in our research group meetings (which were on Zoom anyway because of the pandemic).

Anyway, for reasons too complicated to explain, this previous visa denial means that the student can’t even apply for a new visa to come to the US to enroll in a PhD program to do quantum computing research, and must therefore decline. (Unless some American immigration lawyer reading this can suggest a way out!) The student is not sure what he’s going to do next, but it might involve staying in China, or applying in Europe, or applying in the US again after a year but without mentioning the word “quantum.”

It should go without saying, to anyone reading this, that the student wants to do basic research in quantum complexity theory that’s extraordinarily unlikely to have any direct military or economic importance … just like my own research! And it should also go without saying that, if the US really wanted to strike a blow against authoritarianism in Beijing, then it could hardly do better than to hand out visas to every Chinese STEM student and researcher who wanted to come here. Yes, some would return to China with their new skills, but a great many would choose to stay in the US … if we let them.

And I’ve pointed all this out to a Republican Congressman, and to people in the military and intelligence agencies, when they asked me “what else the US can do to win the quantum computing race against China?” And I’ll continue to say it to anyone who asks. The Congressman, incidentally, even said that he privately agreed with me, but that the issue was politically difficult. I wonder: is there anyone in power in the US, in either party, who doesn’t privately agree that opening the gates to China’s STEM talent would be a win/win proposition for the US … including for the US’s national security? If so, who are these people? Is this just a naked-emperor situation, where everyone in Congress fears to raise the issue because they fear backlash from someone else, but the someone else is actually thinking the same way?

And to any American who says, “yeah, but China totally deserves it, because of that spy balloon, and their threats against Taiwan, and all the spying they do with TikTok”—I mean, like, imagine if someone tried to get back at the US government for the Iraq War or for CIA psyops or whatever else by punishing you, by curtailing your academic dreams. It would make exactly as much sense.

View Details

[WARNING: SPOILERS FOLLOW]

Tonight, on a rare date without the kids, Dana and I saw M3GAN, the new black-comedy horror movie about an orphaned 9-year-old girl named Cady who, under the care of her roboticist aunt, gets an extremely intelligent and lifelike AI doll as a companion. The robot doll, M3GAN, is given a mission to bond with Cady and protect her physical and emotional well-being at all times. M3GAN proceeds to take that directive more literally than intended, with predictably grisly results given the genre.

I chose this movie for, you know, work purposes. Research for my safety job at OpenAI.

So, here’s my review: the first 80% or so of M3GAN constitutes one of the finest movies about AI that I’ve seen. Judged purely as an “AI-safety cautionary fable” and not on any other merits, it takes its place alongside or even surpasses the old standbys like 2001, Terminator, and The Matrix. There are two reasons.

First, M3GAN tries hard to dispense with the dumb tropes that an AI differs from a standard-issue human mostly in its thirst for power, its inability to understand true emotions, and its lack of voice inflection. M3GAN is explicitly a “generative learning model”—and she’s shown becoming increasingly brilliant at empathy, caretaking, and even emotional manipulation. It’s also shown, 100% plausibly, how Cady grows to love her robo-companion more than any human, even as the robot’s behavior turns more and more disturbing. I’m extremely curious to what extent the script was influenced by the recent explosion of large language models—but in any case, it occurred to me that this is what you might get if you tried to make a genuinely 2020s AI movie, rather than a 60s AI movie with updated visuals.

Secondly, until near the end, the movie actually takes seriously that M3GAN, for all her intelligence and flexibility, is a machine trying to optimize an objective function, and that objective function can’t be ignored for narrative convenience. Meaning: sure, the robot might murder, but not to “rebel against its creators and gain power” (as in most AI flicks), much less because “chaos theory demands it” (Jurassic Park), but only to further its mission of protecting Cady. I liked that M3GAN’s first victims—a vicious attack dog, the dog’s even more vicious owner, and a sadistic schoolyard bully—are so unsympathetic that some part of the audience will, with guilty conscience, be rooting for the murderbot.

But then there’s the last 20% of the movie, where it abandons its own logic, as the robot goes berserk and resists her own shutdown by trying to kill basically everyone in sight—including, at the very end, Cady herself. The best I can say about the ending is that it’s knowing and campy. You can imagine the scriptwriters sighing to themselves, like, “OK, the focus groups demanded to see the robot go on a senseless killing spree … so I guess a senseless killing spree is exactly what we give them.”

But probably film criticism isn’t what most of you are here for. Clearly the real question is: what insights, if any, can we take from this movie about AI safety?

I found the first 80% of the film to be thought-provoking about at least one AI safety question, and a mind-bogglingly near-term one: namely, what will happen to children as they increasingly grow up with powerful AIs as companions?

In their last minutes before dying in a car crash, Cady’s parents, like countless other modern parents, fret that their daughter is too addicted to her iPad. But Cady’s roboticist aunt, Gemma, then lets the girl spend endless hours with M3GAN—both because Gemma is a distracted caregiver who wants to get back to her work, and because Gemma sees that M3GAN is making Cady happier than any human could, with the possible exception of Cady’s dead parents.

I confess: when my kids battle each other, throw monster tantrums, refuse to eat dinner or bathe or go to bed, angrily demand second and third desserts and to be carried rather than walk, run to their rooms and lock the doors … when they do such things almost daily (which they do), I easily have thoughts like, I would totally buy a M3GAN or two for our house … yes, even having seen the movie! I mean, the minute I’m satisfied that they’ve mostly fixed the bug that causes the murder-rampages, I will order that frigging bot on Amazon with next-day delivery. And I’ll still be there for my kids whenever they need me, and I’ll play with them, and teach them things, and watch them grow up, and love them. But the robot can handle the excruciating bits, the bits that require the infinite patience I’ll never have.

OK, but what about the part where M3GAN does start murdering anyone who she sees as interfering with her goals? That struck me, honestly, as a trivially fixable alignment failure. Please don’t misunderstand me here to be minimizing the AI alignment problem, or suggesting it’s easy. I only mean: supposing that an AI were as capable as M3GAN (for much of the movie) at understanding Asimov’s Second Law of Robotics—i.e., supposing it could brilliantly care for its user, follow her wishes, and protect her—such an AI would seem capable as well of understanding the First Law (don’t harm any humans or allow them to come to harm), and the crucial fact that the First Law overrides the Second.

In the movie, the catastrophic alignment failure is explained, somewhat ludicrously, by Gemma not having had time to install the right safety modules before turning M3GAN loose on her niece. While I understand why movies do this sort of thing, I find it often interferes with the lessons those movies are trying to impart. (For example, is the moral of Jurassic Park that, if you’re going to start a live dinosaur theme park, just make sure to have backup power for the electric fences?)

Mostly, though, it was a bizarre experience to watch this movie—one that, whatever its 2020s updates, fits squarely into a literary tradition stretching back to Faust, the Golem of Prague, Frankenstein’s monster, Rossum’s Universal Robots—and then pinch myself and remember that, here in actual nonfiction reality,

1. I’m now working at one of the world’s leading AI companies,
2. that company has already created GPT, an AI with a good fraction of the fantastical verbal abilities shown by M3GAN in the movie,
3. that AI will gain many of the remaining abilities in years rather than decades, and
4. my job this year—supposedly!—is to think about how to prevent this sort of AI from wreaking havoc on the world.

Incredibly, unbelievably, here in the real world of 2023, what still seems most science-fictional about M3GAN is neither her language fluency, nor her ability to pursue goals, nor even her emotional insight, but simply her ease with the physical world: the fact that she can walk and dance like a real child, and all-too-brilliantly resist attempts to shut her down, and have all her compute onboard, and not break. And then there’s the question of the power source. The movie was never explicit about that, except for implying that she sits in a charging port every night. The more the movie descends into grotesque horror, though, the harder it becomes to understand why her creators can’t avail themselves of the first and most elemental of all AI safety strategies—like flipping the switch or popping out the battery.

View Details

Just days after we celebrated my wife’s 40th birthday, she came down with COVID, meaning she’s been isolating and I’ve been spending almost all my time dealing with our kids.

But if experience has taught me anything, it’s that the quantum hype train never slows down. In the past 24 hours, at least four people have emailed to ask me about a new paper entitled “Factoring integers with sublinear resources on a superconducting quantum processor.” Even the security expert Bruce Schneier, while skeptical, took the paper surprisingly seriously.

The paper claims … well, it’s hard to pin down what it claims, but it’s certainly given many people the impression that there’s been a decisive advance on how to factor huge integers, and thereby break the RSA cryptosystem, using a near-term quantum computer. Not by using Shor’s Algorithm, mind you, but by using the deceptively similarly named Schnorr’s Algorithm. The latter is a classical algorithm based on lattices, which the authors then “enhance” using the heuristic quantum optimization method called QAOA.

For those who don’t care to read further, here is my 3-word review:

No. Just No.And here’s my slightly longer review:

Schnorr ≠ Shor. Yes, even when Schnorr’s algorithm is dubiously “enhanced” using QAOA—a quantum algorithm that, incredibly, for all the hundreds of papers written about it, has not yet been convincingly argued to yield any speedup for any problem whatsoever (besides, as it were, the problem of reproducing its own pattern of errors).

In the new paper, the authors spend page after page saying-without-saying that it might soon become possible to break RSA-2048, using a NISQ (i.e., non-fault-tolerant) quantum computer. They do so via two time-tested strategems:

1. the detailed exploration of irrelevancies (mostly, optimization of the number of qubits, while ignoring the number of gates), and
2. complete silence about the one crucial point.

Then, finally, they come clean about the crucial point in a single sentence of the Conclusion section:

It should be pointed out that the quantum speedup of the algorithm is unclear due to the ambiguous convergence of QAOA.

“Unclear” is an understatement here. It seems to me that a miracle would be needed for the approach described to yield any benefit at all, compared to just running the classical Schnorr’s algorithm on your laptop. And if the latter were able to break RSA, it would’ve already done so.

All told, this is one of the most misleading quantum computing papers I’ve seen in 25 years, and I’ve seen … many. Having said that, this actually isn’t the first time I’ve encountered the strange idea that the exponential quantum speedup for factoring integers, which we know about from Shor’s algorithm, should somehow “rub off” onto quantum optimization heuristics that embody none of the actual insights of Shor’s algorithm, as if by sympathetic magic. Since this idea needs a name, I’d hereby like to propose one: Cargo Cult Quantum Factoring.

And with that, I feel I’ve adequately discharged my duties here to sanity and truth. If I’m slow to answer comments, it’ll be because I’m dealing with two screaming kids.

View Details

The following is what I read at Dana’s 40th birthday party last night. Don’t worry, it’s being posted with her approval. –SA

I’d like to propose a toast to Dana, my wife and mother of my two kids. My dad, a former speechwriter, would advise me to just crack a few jokes and then sit down … but my dad’s not here.

So instead I’ll tell you a bit about Dana. She grew up in Tel Aviv, finishing her undergraduate CS degree at age 17—before she joined the army. I met her when I was a new professor at MIT and she was a postdoc in Princeton, and we’d go to many of the same conferences. At one of those conferences, in Princeton, she finally figured out that my weird, creepy, awkward attempts to make conversation with her were, in actuality, me asking her out … at least in my mind! So, after I’d returned to Boston, she then emailed me for days, just one email after the next, explaining everything that was wrong with me and all the reasons why we could never date. Despite my general obliviousness in such matters, at some point I wrote back, “Dana, the absolute value of your feelings for me seems perfect. Now all I need to do is flip the sign!”

Anyway, the very next weekend, I took the Amtrak back to Princeton at her invitation. That weekend is when we started dating, and it’s also when I introduced her to my family, and when she and I planned out the logistics of getting married.

Dana and her family had been sure that she’d return to Israel after her postdoc. She made a huge sacrifice in staying here in the US for me. And that’s not even mentioning the sacrifice to her career that came with two very difficult pregnancies that produced our two very diffic … I mean, our two perfect and beautiful children.

Truth be told, I haven’t always been the best husband, or the most patient or the most grateful. I’ve constantly gotten frustrated and upset, extremely so, about all the things in our life that aren’t going well. But preparing the slideshow tonight, I had a little epiphany. I had a few photos from the first two-thirds of Dana’s life, but of course, I mostly had the last third. But what’s even happened in that last third? She today feels like she might be close to a breakthrough on the Unique Games Conjecture. But 13 years ago, she felt exactly the same way. She even looks the same!

So, what even happened?

Well OK, fine, there was my and Dana’s first trip to California, a month after we started dating. Our first conference together. Our trip to Vegas and the Grand Canyon. Our first trip to Israel to meet her parents, who I think are finally now close to accepting me. Her parents’ trip to New Hope, Pennsylvania to meet my parents. Our wedding in Tel Aviv—the rabbi rushing through the entire ceremony in 7 minutes because he needed to get home to his kids. Our honeymoon safari in Kenya. Lily’s birth. Our trip to Israel with baby Lily, where we introduced Lily to Dana’s grandmother Rivka, an Auschwitz survivor, just a few months before Rivka passed away. Taking Lily to run around Harvard Yard with our Boston friends, Lily losing her beloved doll Tuza there, then finding Tuza the next day after multiple Harvard staff had been engaged in the quest. There’s me and Dana eating acai bowls in Rio de Janeiro, getting a personal tour of the LHC in Switzerland, with kangaroos and koalas in Australia. There’s our house here in Austin. Oh, and here are all our Austin friends! Our trip to Disney World with Lily, while Dana was pregnant with Daniel (she did ride one rollercoaster). Daniel’s birth, which to my relief, went well despite the rollercoaster. Our sabbatical year in Israel. The birth of our nephews.

I confess I teared up a little going through all this. Because it’s like: if you showed all these photos to some third party, they’d probably be like, wow, that looks like a pretty good life. Even if you showed the photos to my 17-year-old self, I’d be like, OK, no need to roll the dice again, I’ll take that life. I’m not talking about what’s happened with the world—climate change or COVID or the insurrection or anything like that. But the part with the beautiful Israeli complexity theorist wife, who’s also caring and unbreakable in her moral convictions? Yes.

So, thank you and a very happy birthday to the one who’s given me all this!

And lastly, because it seems almost obligatory: I did feed everything I said just now into GPT (I won’t specify which version), and asked it to write Dana a special GPT birthday poem. Here’s what it came up with:

From Tel Aviv to Princeton,
You’ve traveled near and far,
A brilliant computer scientist,
A shining, guiding star.

You’ve made so many sacrifices,
For family and for love,
But your light shines through the darkness,
And fits me like a glove.

We’ve shared so many moments,
Too many to recount,
But each one is a treasure,
Each memory paramount.

So happy birthday, Dana,
You deserve the very best,
I’m grateful for your presence,
And feel so truly blessed.

Addendum: Speaking of GPT, should it and other Large Language Models be connected to the Internet and your computer’s filesystem and empowered to take actions directly, with reinforcement learning pushing it to achieve the user’s goals?

On the negative side, some of my friends worry that this sort of thing might help an unaligned superintelligence to destroy the world.

But on the positive side, at Dana’s birthday party, I could’ve just told the computer, “please display these photos in a slideshow rotation while also rotating among these songs,” and not wasted part of the night messing around with media apps that befuddle and defeat me as a mere CS PhD.

I find it extremely hard to balance these considerations.

Anyway, happy birthday Dana!

View Details

Dear Scott,

This is you, from 30 years in the future, Christmas Eve 2022. Your Ghost of Christmas Future.

To get this out of the way: you eventually become a professor who works on quantum computing. Quantum computing is … OK, you know the stuff in popular physics books that never makes any sense, about how a particle takes all the possible paths at once to get from point A to point B, but you never actually see it do that, because as soon as you look, it only takes one path? Turns out, there’s something huge there, even though the popular books totally botch the explanation of it. It involves complex numbers. A quantum computer is a new kind of computer people are trying to build, based on the true story.

Anyway, amazing stuff, but you’ll learn about it in a few years anyway. That’s not what I’m writing about.

I’m writing from a future that … where to start? I could describe it in ways that sound depressing and even boring, or I could also say things you won’t believe. Tiny devices in everyone’s pockets with the instant ability to videolink with anyone anywhere, or call up any of the world’s information, have become so familiar as to be taken for granted. This sort of connectivity would come in especially handy if, say, a supervirus from China were to ravage the world, and people had to hide in their houses for a year, wouldn’t it?

Or what if Donald Trump — you know, the guy who puts his name in giant gold letters in Atlantic City? — became the President of the US, then tried to execute a fascist coup and to abolish the Constitution, and came within a hair of succeeding?

Alright, I was pulling your leg with that last one … obviously! But what about this next one?

There’s a company building an AI that fills giant rooms, eats a town’s worth of electricity, and has recently gained an astounding ability to converse like people. It can write essays or poetry on any topic. It can ace college-level exams. It’s daily gaining new capabilities that the engineers who tend to the AI can’t even talk about in public yet. Those engineers do, however, sit in the company cafeteria and debate the meaning of what they’re creating. What will it learn to do next week? Which jobs might it render obsolete? Should they slow down or stop, so as not to tickle the tail of the dragon? But wouldn’t that mean someone else, probably someone with less scruples, would wake the dragon first? Is there an ethical obligation to tell the world more about this? Is there an obligation to tell it less?

I am—you are—spending a year working at that company. My job—your job—is to develop a mathematical theory of how to prevent the AI and its successors from wreaking havoc. Where “wreaking havoc” could mean anything from turbocharging propaganda and academic cheating, to dispensing bioterrorism advice, to, yes, destroying the world.

You know how you, 11-year-old Scott, set out to write a QBasic program to converse with the user while following Asimov’s Three Laws of Robotics? You know how you quickly got stuck? Thirty years later, imagine everything’s come full circle. You’re back to the same problem. You’re still stuck.

Oh all right. Maybe I’m just pulling your leg again … like with the Trump thing. Maybe you can tell because of all the recycled science fiction tropes in this story. Reality would have more imagination than this, wouldn’t it?

But supposing not, what would you want me to do in such a situation? Don’t worry, I’m not going to take an 11-year-old’s advice without thinking it over first, without bringing to bear whatever I know that you don’t. But you can look at the situation with fresh eyes, without the 30 intervening years that render it familiar. Help me. Throw me a frickin’ bone here (don’t worry, in five more years you’ll understand the reference).

Thanks!!
—Scott

PS. When something called “bitcoin” comes along, invest your life savings in it, hold for a decade, and then sell.

PPS. About the bullies, and girls, and dating … I could tell you things that would help you figure it out a full decade earlier. If I did, though, you’d almost certainly marry someone else and have a different family. And, see, I’m sort of committed to the family that I have now. And yeah, I know, the mere act of my sending this letter will presumably cause a butterfly effect and change everything anyway, yada yada. Even so, I feel like I owe it to my current kids to maximize their probability of being born. Sorry, bud!

View Details

This is going to be one of the many Shtetl-Optimized posts that I didn’t feel like writing, but was ultimately given no choice but to write. News, social media, and my inbox have been abuzz with two claims about Google’s Sycamore quantum processor, the one that now has 72 superconducting qubits. The first claim is […]

View Details

Two weeks ago, I gave a lecture setting out my current thoughts on AI safety, halfway through my year at OpenAI. I was asked to speak by UT Austin’s Effective Altruist club. You can watch the lecture on YouTube here (I recommend 2x speed). The timing turned out to be weird, coming immediately after the […]

View Details

Happy Thanksgiving to my American readers! While I enjoy a family holiday-week vacation in exotic Dallas—and yes, I will follow up on my old JFK post by visiting Dealey Plaza—please enjoy the following Thanksgiving victuals: I recently recorded a 3-hour (!) YouTube video with Timothy Nguyen, host of the Cartesian Cafe. Our episode is entitled […]

View Details

Nearly halfway into my year at OpenAI, still reeling from the FTX collapse, I feel like it’s finally time to start blogging my AI safety thoughts—starting with a little appetizer course today, more substantial fare to come. Many people claim that AI alignment is little more a modern eschatological religion—with prophets, an end-times prophecy, sacred scriptures, and […]

View Details

I’m thrilled to be able to interrupt your regular depressing programming for 100% happy news. Some readers will remember that, back in September, I announced that an unnamed charitable foundation had asked my advice on how best to donate $250,000 for advanced precollege STEM education. So, just like the previous time I got such a […]

View Details

In the past few weeks, I’ve learned two ways to think about online sneerers that have been helping me tremendously, and that I wanted to share in case they’re helpful to others: First, they’re like a train in a movie that’s barreling directly towards the camera. If you haven’t yet internalized how the medium works, […]

View Details

Update (Nov. 16): Check out this new interview of SBF by my friend and leading Effective Altruist writer Kelsey Piper. Here Kelsey directly confronts SBF with some of the same moral and psychological questions that animated this post and the ensuing discussion—and, surely to the consternation of his lawyers, SBF answers everything she asks. And […]

View Details

If I haven’t blogged until now about the midterm election, it’s because I find the state of the world too depressing. Go vote, obviously, if you’re eligible and haven’t yet. How many more chances will you have? While I’m (to put it mildly) neither especially courageous nor useful as an infantryman, I would’ve been honored […]

View Details

These days, I often need to remind myself that, as an undergrad, grad student, postdoc, or professor, I’ve now been doing quantum computing research for a quarter-century—i.e., well over half of the subject’s existence. As a direct result, when I feel completely jaded about a new development in QC, it might actually be exciting. When […]

View Details

Yesterday I attended a lecture by George Mason University economist Bryan Caplan, who’s currently visiting UT Austin, about his new book entitled Don’t Be a Feminist. (See also here for previous back-and-forth between me and Bryan about his book.) A few remarks: (1) Maybe surprisingly, there were no protesters storming the lectern, no security detail, […]

View Details

Here’s an observation that’s mathematically trivial but might not be widely appreciated. In kindergarten, we all learned Gödel’s First Incompleteness Theorem, which given a formal system F, constructs an arithmetical encoding of G(F) = “This sentence is not provable in F.” If G(F) is true, then it’s an example of a true arithmetical sentence that’s […]

View Details

I’m proud to say that Nick Hunter-Jones and Matteo Ippoliti—both of whom work at the interface between quantum information science and condensed-matter physics (Nick closer to the former and Matteo to the latter)—have joined the physics faculty at UT Austin this year. And Nick, Matteo, and I are jointly seeking postdocs to start in Fall […]

View Details

For anyone living under a rock with no access to nerd social media, Alain Aspect, John Clauser, and Anton Zeilinger have finally won the Nobel Prize in Physics, for their celebrated experiments that rubbed everyone’s faces in the reality of quantum entanglement (including Bell inequality violation and quantum teleportation). I don’t personally know Aspect or […]

View Details

(1) Since I didn’t blog about this before: huge congratulations to David Deutsch, Charles Bennett, Gilles Brassard, and my former MIT colleague Peter Shor, and separately to Dan Spielman, for their well-deserved Breakthrough Prizes! Their contributions are all so epochal, so universally known to all of us in quantum information and theoretical computer science, that […]

View Details

As I slept fitfully, still recovering from COVID, I had one of the more interesting dreams of my life: I was desperately trying to finish some PowerPoint slides in time to give a talk. Uncharacteristically for me, one of the slides displayed actual code. This was a dream, so nothing was as clear as I’d […]

View Details

The same thing Salman Rushdie learned: either you spend your entire life in hiding, or eventually it’ll come for you. Years might pass. You might emerge from hiding once, ten times, a hundred times, be fine, and conclude (emotionally if not intellectually) that the danger must now be over, that if it were going to […]

View Details

Back in January, you might recall, Skype cofounder Jaan Tallinn’s Survival and Flourishing Fund (SFF) was kind enough to earmark $200,000 for me to donate to any charitable organizations of my choice. So I posted a call for proposals on this blog. You “applied” to my “foundation” by simply sending me an email, or leaving […]

View Details

Here they are [PDF]. They’re 155 pages of pure awesome—for a certain extremely specific definition of “awesome”—which I’m hereby offering to the world free of charge (for noncommercial use only, of course). They cover material that I taught, for the first time, in my Introduction to Quantum Information Science II undergrad course at UT Austin […]

View Details

Way back in the covid-filled summer of 2020, I wrote a survey article about the ridiculously-rapidly-growing Busy Beaver function. My survey then expanded to nearly twice its original length, with the ideas, observations, and open problems of commenters on this blog. Ever since, I’ve felt a sort of duty to blog later developments in BusyBeaverology […]

View Details

Several people have asked me to comment about a Financial Times opinion piece entitled The Quantum Computing Bubble (subtitle: “The industry has yet to demonstrate any real utility, despite the fanfare, billions of VC dollars and three Spacs”) (archive link). The piece is purely deflationary—not a positive word in it—though it never goes so far […]

View Details

Update: We’re now finalizing the lecture notes—basically, a textbook—for the brand-new Quantum Information Science II course that I taught this past spring! The notes will be freely shared on this blog. But the bibliographies for the various lectures need to be merged, and we don’t know how. Would any TeXpert like to help us, in […]

View Details

On the IBM Qiskit blog, there’s an interview with me about the role of complexity theory in the early history of quantum computing. Not much new for regular readers, but I’m very happy with how it came out—thanks so much to Robert Davis and Olivia Lanes for making it happen! My only quibble is with […]

View Details

Scott’s Introduction Juris Hartmanis — one of the founding figures of theoretical computer science, winner of the Turing Award, cofounder of the Cornell computer science department (of which I’m an alumnus), cofounder of the Conference on Computational Complexity or CCC (which I just attended), PhD adviser to many of the leading complexity theorists — has […]

View Details

I promise you: this post is going to tell a scientifically coherent story that involves all five topics listed in the title. Not one can be omitted. My story starts with a Zoom talk that the one and only Lenny Susskind delivered for the Simons Institute for Theory of Computing back in May. There followed […]

View Details

Several people have complained that Shtetl-Optimized has become too focused on the niche topic of “people being mean to Scott Aaronson on the Internet.” In one sense, this criticism is deeply unfair—did I decide that a shockingly motivated and sophisticated troll should attack me all week, in many cases impersonating fellow academics to do so? […]

View Details

Thanks so much to everyone who offered help and support as this blog’s comment section endured the weirdest, most motivated and sophisticated troll attack in its 17-year history. For a week, a parade of self-assured commenters showed up to demand that I explain and defend my personal hygiene, private thoughts, sexual preferences, and behavior around […]

View Details

Update (July 13): I was honored to read this post by my friend Boaz Barak. Update (July 14): By now, comments on this post allegedly from four CS professors — namely, Josh Alman, Aloni Cohen, Rana Hanocka, and Anna Farzindar — as well as from the graduate student “BA,” have been unmasked as from impersonator(s). […]

View Details

(1) Fellow CS theory blogger (and, 20 years ago, member of my PhD thesis committee) Luca Trevisan interviews me about Shtetl-Optimized, for the Bulletin of the European Association for Theoretical Computer Science. Questions include: what motivates me to blog, who my main inspirations are, my favorite posts, whether blogging has influenced my actual research, and […]

View Details

In Steven Pinker’s guest post from last week, there’s one bit to which I never replied. Steve wrote: After all, in many areas Einstein was no Einstein. You [Scott] above all could speak of his not-so-superintelligence in quantum physics… While I can’t speak “above all,” OK, I can speak. Now that we’re closing in on […]

View Details

When the machines outperform us on every goal for which performance can be quantified, When the machines outpredict us on all events whose probabilities are meaningful, When they not only prove better theorems and build better bridges, but write better Shakespeare than Shakespeare and better Beatles than the Beatles, All that will be left to […]

View Details

Before June 2022 was the month of the possible start of the Second American Civil War, it was the month of a lively debate between Scott Alexander and Gary Marcus about the scaling of large language models, such as GPT-3.  Will GPT-n be able to do all the intellectual work that humans do, in the […]

View Details

In 1973, the US Supreme Court enshrined the right to abortion—considered by me and ~95% of everyone I know to be a basic pillar of modernity—in such a way that the right could be overturned only if its opponents could somehow gain permanent minority rule, and thereby disregard the wills of three-quarters of Americans. So […]

View Details

I have some exciting news (for me, anyway). Starting next week, I’ll be going on leave from UT Austin for one year, to work at OpenAI. They’re the creators of the astonishing GPT-3 and DALL-E2, which have not only endlessly entertained me and my kids, but recalibrated my understanding of what, for better and worse, […]

View Details

… on Blake Lemoine, the Google engineer who became convinced that a machine learning model had become sentient, contacted federal government agencies about it, and was then fired placed on administrative leave for violating Google’s confidentiality policies. (1) I don’t think Lemoine is right that LaMDA is at all sentient, but the transcript is so […]

View Details

Thanks so much to everyone who sent messages of support following my last post! I vowed there that I’m going to stop letting online trolls and sneerers occupy so much space in my mental world. Truthfully, though, while there are many trolls and sneerers who terrify me, there are also some who merely amuse me. […]

View Details

I hereby precommit that this will be my last post, for a long time, around the twin themes of (1) the horribleness in the United States and the world, and (2) my desperate attempts to reason with various online commenters who hold me personally complicit in all this horribleness. I should really focus my creativity […]

View Details

So, I’d been planning a fun post for today about the DALL-E image-generating AI model, and in particular, a brief new preprint about DALL-E’s capabilities by Ernest Davis, Gary Marcus, and myself. We wrote this preprint as a sort of “adversarial collaboration”: Ernie and Gary started out deeply skeptical of DALL-E, while I was impressed […]

View Details

View Details

Update (April 27): Boaz Barak—Harvard CS professor, longtime friend-of-the-blog, and coauthor of my previous guest post on this topic—has just written an awesome FAQ, providing his personal answers to the most common questions about what I called our “campaign to defend serious math education.” It directly addresses several issues that have already come up in […]

View Details

There is a fundamental difference between form and meaning. Form is the physical structure of something, while meaning is the interpretation or concept that is attached to that form. For example, the form of a chair is its physical structure – four legs, a seat, and a back. The meaning of a chair is that […]

View Details

Thanks to everyone who asked whether I’m OK! Yeah, I’ve been living, loving, learning, teaching, worrying, procrastinating, just not blogging. Last week, Takashi Yamakawa and Mark Zhandry posted a preprint to the arXiv, “Verifiable Quantum Advantage without Structure,” that represents some of the most exciting progress in quantum complexity theory in years. I wish I’d […]

View Details

… but these antiwar protesters in St. Petersburg know that they’re all going to be arrested and are doing it anyway. Meanwhile, I just spent an hour giving Lily, my 9-year-old, a crash course on geopolitics, including WWII, the Cold War, the formation of NATO, Article 5, nuclear deterrence, economic sanctions, the breakup of the […]

View Details

When, before covid, I used to travel the world giving quantum computing talks, every once in a while I’d meet an older person who asked whether I had any relation to a 1970s science writer by the name of Steve Aaronson. So, yeah, Steve Aaronson is my dad. He majored in English in Penn State, […]

View Details

Tonight, I took the time actually to read DeepMind’s AlphaCode paper, and to work through the example contest problems provided, and understand how I would’ve solved those problems, and how AlphaCode solved them. It is absolutely astounding. Consider, for example, the “n singers” challenge (pages 59-60). To solve this well, you first need to parse […]

View Details

Two weeks ago, I announced on this blog that, thanks to the remarkable generosity of Jaan Tallinn, and the Speculation Grants program of the Survival and Flourishing Fund that Jaan founded, I had $200,000 to give away to charitable organizations of my choice. So, inspired by what Scott Alexander had done, I invited the readers […]

View Details

In the past few months, I’ve twice injured the same ankle while playing with my kids. This, perhaps combined with covid, led me to several indisputable realizations: I am mortal. Despite my self-conception as a nerdy little kid awaiting the serious people’s approval, I am now firmly middle-aged. By my age, Einstein had completed general […]

View Details

Exciting news, everyone! Jaan Tallinn, who many of you might recognize as a co-creator of Skype, tech enthusiast, and philanthropist, graciously invited me, along with a bunch of other nerds, to join the new Speculation Grants program of the Survival and Flourishing Fund (SFF). In plain language, that means that Jaan is giving me $200,000 to distribute to charitable organizations in any way I see fit—though ideally, my choices will have something to do with the survival and flourishing of our planet and civilization.

(If all goes well, this blog post will actually lead to a lot more than just $200,000 in donations, because it will inspire applications to SFF that can then be funded by other “Speculators” or by SFF’s usual process.)

Thinking about how to handle the responsibility of this amazing and unexpected gift, I decided that I couldn’t possibly improve on what Scott Alexander did with his personal grants program on Astral Codex Ten. Thus: I hereby invite the readers of Shtetl-Optimized to pitch registered charities (which might or might not be their own)—especially, charities that are relatively small, unknown, and unappreciated, yet that would resonate strongly with someone who thinks the way I do. Feel free to renominate (i.e., bring back to my attention) charities that were mentioned when I asked a similar question after winning $250,000 from the ACM Prize in Computing.

If you’re interested, there’s a two-step process this time:

Step 1 is to make your pitch to me, either by a comment on this post or by email to me, depending on whether you’d prefer the pitch to be public or private. Let’s set a deadline for this step of Thursday, January 27, 2022 (i.e., one week from now). Your pitch can be extremely short, like 1 paragraph, although I might ask you followup questions. After January 27, I’ll then take one of two actions in response: I’ll either

(a) commit a specified portion of my $200,000 to your charity, if the charity formally applies to SFF, and if the charity isn’t excluded for some unexpected reason (5 sexual harassment lawsuits against its founders or whatever), and if one of my fellow “Speculators” doesn’t fund your charity before I do … or else I’ll

(b) not commit, in which case your charity can still apply for funding from SFF! One of the other Speculators might fund it, or it might be funded by the “ordinary” SFF process.

Step 2, which cannot be skipped, is then to have your charity submit a formal application to SFF. The application form isn’t too bad. But if the charity isn’t your own, it would help enormously if you at least knew someone at the charity, so you could tell them to apply to SFF. Again, Step 2 can be taken regardless of the outcome of Step 1.

The one big rule is that anything you suggest has to be a registered, tax-exempt charity in either the US or the UK. I won’t be distributing funds myself, but only advising SFF how to do so, and this is SFF’s rule, not mine. So alas, no political advocacy groups and no individuals. Donating to groups outside the US and UK is apparently possible but difficult.

While I’m not putting any restrictions on the scope, let me list a few examples of areas of interest to me.

• Advanced math and science education at the precollege level: gifted programs, summer camps, online resources, or anything, really, that aims to ensure that the next Ramanujan or von Neumann isn’t lost to the world.
• Conservation of endangered species.
• Undervalued approaches to dealing with the climate catastrophe (including new approaches to nuclear energy, geoengineering, and carbon capture and storage … or even, e.g., studies of the effects of rising CO2 on cognition and how to mitigate them).
• Undervalued approaches to preventing or mitigating future pandemics—basically, anything dirt-cheap that we wish had been done before covid.
• Almost anything that Scott Alexander might have funded if he’d had more money.
• Anything that would enrage the SneerClubbers or those who attack me on Twitter, by doing stuff that even they would have to acknowledge makes the world better, but that does so via people, organizations, and means that they despise.

Two examples of areas that I don’t plan to focus on are:

• AI-risk and other “strongly rationalist-flavored” organizations (these are already well-covered by others at SFF, so that I don’t expect to have an advantage), and
• quantum computing research (this is already funded by a zillion government agencies, companies, and venture capitalists).

Anyway, thanks so much to Jaan and to SFF for giving me this incredible opportunity, and I look forward to seeing what y’all come up with!

Note: Any other philanthropists who read this blog, and who’d like to add to the amount, are more than welcome to do so!

View Details

(Hopefully no one has taken taken that title yet!)

I waste a large fraction of my existence just reading about what’s happening in the world, or discussion and analysis thereof, in an unending scroll of paralysis and depression. On the first anniversary of the January 6 attack, I read the recent revelations about just how close the seditionists actually came to overturning the election outcome (e.g., by pressuring just one Republican state legislature to “decertify” its electors, after which the others would likely follow in a domino effect), and how hard it now is to see a path by which democracy in the United States will survive beyond 2024. Or I read about Joe Manchin, who’s already entered the annals of history as the man who could’ve halted the slide to the abyss and decided not to. Of course, I also read about the wokeists, who correctly see the swing of civilization getting pushed terrifyingly far out of equilibrium to the right, so their solution is to push the swing terrifyingly far out of equilibrium to the left, and then they act shocked when their own action, having added all this potential energy to the swing, causes it to swing back even further to the right, as swings tend to do. (And also there’s a global pandemic killing millions, and the correct response to it—to authorize and distribute new vaccines as quickly as the virus mutates—is completely outside the Overton Window between Obey the Experts and Disobey the Experts, advocated by no one but a few nerds. When I first wrote this post, I forgot all about the global pandemic.) And I see all this and I am powerless to stop it.

In such a dark time, it’s easy to forget that I’m a theoretical computer scientist, mainly focused on quantum computing. It’s easy to forget that people come to this blog because they want to read about quantum computing. It’s like, who gives a crap about that anymore? What doth it profit a man, if he gaineth a few thousand fault-tolerant qubits with which to calculateth chemical reaction rates or discrete logarithms, and he loseth civilization?

Nevertheless, in the rest of this post I’m going to share some quantum-related debunking updates—not because that’s what’s at the top of my mind, but in an attempt to find my way back to sanity. Picture that: quantum mechanics (and specifically, the refutation of outlandish claims related to quantum mechanics) as the part of one’s life that’s comforting, normal, and sane.

There’s been lots of online debate about the claim to have entangled a tardigrade (i.e., water bear) with a superconducting qubit; see also this paper by Vlatko Vedral, this from CNET, this from Ben Brubaker on Twitter. So, do we now have Schrödinger’s Tardigrade: a living, “macroscopic” organism maintained coherently in a quantum superposition of two states? How could such a thing be possible with the technology of the early 21st century? Hasn’t it been a huge challenge to demonstrate even Schrödinger’s Virus or Schrödinger’s Bacterium? So then how did this experiment leapfrog (or leaptardigrade) over those vastly easier goals?

Short answer: it didn’t. The experimenters couldn’t directly measure the degree of freedom in the tardigrade that’s claimed to be entangled with the qubit. But it’s consistent with everything they report that whatever entanglement is there, it’s between the superconducting qubit and a microscopic part of the tardigrade. It’s also consistent with everything they report that there’s no entanglement at all between the qubit and any part of the tardigrade, just boring classical correlation. (Or rather that, if there’s “entanglement,” then it’s the Everett kind, involving not merely the qubit and the tardigrade but the whole environment—the same as we’d get by just measuring the qubit!) Further work would be needed to distinguish these possibilities. In any case, it’s of course cool that they were able to cool a tardigrade to near absolute zero and then revive it afterwards.

I thank the authors of the tardigrade paper, who clarified a few of these points in correspondence with me. Obviously the comments section is open for whatever I’ve misunderstood.

People also asked me to respond to Sabine Hossenfelder’s recent video about superdeterminism, a theory that holds that quantum entanglement doesn’t actually exist, but the universe’s initial conditions were fine-tuned to stop us from choosing to measure qubits in ways that would make its nonexistence apparent: even when we think we’re applying the right measurements, we’re not, because the initial conditions messed with our brains or our computers’ random number generators. (See, I tried to be as non-prejudicial as possible in that summary, and it still came out sounding like a parody. Sorry!)

Sabine sets up the usual dichotomy that people argue against superdeterminism only because they’re attached to a belief in free will. She rejects Bell’s statistical independence assumption, which she sees as a mere dogma rather than a prerequisite for doing science. Toward the end of the video, Sabine mentions the objection that, without statistical independence, a demon could destroy any randomized controlled trial, by tampering with the random number generator that decides who’s in the control group and who isn’t. But she then reassures the viewer that it’s no problem: superdeterministic conspiracies will only appear when quantum mechanics would’ve predicted a Bell inequality violation or the like. Crucially, she never explains the mechanism by which superdeterminism, once allowed into the universe (including into macroscopic devices like computers and random number generators), will stay confined to reproducing the specific predictions that quantum mechanics already told us were true, rather than enabling ESP or telepathy or other mischief. This is stipulated, never explained or derived.

To say I’m not a fan of superdeterminism would be a super-understatement. And yet, nothing I’ve written previously on this blog—about superdeterminism’s gobsmacking lack of explanatory power, or about how trivial it would be to cook up a superdeterministic “mechanism” for, e.g., faster-than-light signaling—none of it seems to have made a dent. It’s all come across as obvious to the majority of physicists and computer scientists who think as I do, and it’s all fallen on deaf ears to superdeterminism’s fans.

So in desperation, let me now try another tack: going meta. It strikes me that no one who saw quantum mechanics as a profound clue about the nature of reality could ever, in a trillion years, think that superdeterminism looked like a promising route forward given our current knowledge. The only way you could think that, it seems to me, is if you saw quantum mechanics as an anti-clue: a red herring, actively misleading us about how the world really is. To be a superdeterminist is to say:

OK, fine, there’s the Bell experiment, which looks like Nature screaming the reality of ‘genuine indeterminism, as predicted by QM,’ louder than you might’ve thought it even logically possible for that to be screamed. But don’t listen to Nature, listen to us! If you just drop what you thought were foundational assumptions of science, we can explain this away! Not explain it, of course, but explain it away. What more could you ask from us?

Here’s my challenge to the superdeterminists: when, in 400 years from Galileo to the present, has such a gambit ever worked? Maxwell’s equations were a clue to special relativity. The Hamiltonian and Lagrangian formulations of classical mechanics were clues to quantum mechanics. When has a great theory in physics ever been grudgingly accommodated by its successor theory in a horrifyingly ad-hoc way, rather than gloriously explained and derived?

Update: Oh right, and the QIP’2022 list of accepted talks is out! And I was on the program committee! And ~~they’re still planning to hold QIP in person, in March at Caltech, will you fancy that!~~ actually I have no idea—but if they’re going to move to virtual, I’m awaiting an announcement just like everyone else.

View Details

Scott’s foreword

One week ago, E. O. Wilson—the legendary naturalist and conservationist, and man who was universally acknowledged to know more about ants than anyone else in human history—passed away at age 92. A mere three days later, Scientific American—or more precisely, the zombie clickbait rag that now flaunts that name—published a shameful hit-piece, smearing Wilson for his “racist ideas” without, incredibly, so much as a single quote from Wilson, or any other attempt to substantiate its libel (see also this response by Jerry Coyne). SciAm‘s Pravda-like attack included the following extraordinary sentence, which I thought worthy of Alan Sokal’s Social Text hoax:

The so-called normal distribution of statistics assumes that there are default humans who serve as the standard that the rest of us can be accurately measured against.

There are intellectually honest people who don’t know what the normal distribution is. There are no intellectually honest people who, not knowing what it is, figure that it must be something racist.

On Twitter, Laura Helmuth, the editor-in-chief now running SciAm into the ground, described her magazine’s calumny against Wilson as “insightful” (the replies, including from Richard Dawkins, are fun to read). I suppose it was as “insightful” as SciAm‘s disgraceful attack last year on Eric Lander, President Biden’s ultra-competent science advisor and a leader in the war on COVID, for … being a white male, which appears to have been E. O. Wilson’s crime as well. (Think I must be misrepresenting the “critique” of Lander? Read it!)

Anyway, in response to Scientific American‘s libel of Wilson, I wrote on my Facebook that I’ll no longer agree to write for or be interviewed by them (you can read my old stuff free of charge here or here), unless and until there’s a complete change of editorial direction. I encourage all other scientists to commit likewise, thereby making it common knowledge that the entity that now calls itself “Scientific American” bears the same relation to the legendary home of Martin Gardner as does a corpse to a living being. Fortunately, there are high-quality online venues (e.g., Quanta) that partly fill the role that Scientific American abdicated.

After reading my Facebook post, my friend Ashutosh Jogalekar was inspired to post an essay of his own. Ashutosh used to write regularly for Scientific American, until he was fired seven years ago over a column in which he advocated acknowledging Richard Feynman’s flaws, including his arrogance and casual sexism, but also understanding those flaws within the context of Feynman’s whole life, including the tragic death of his first wife Arlene. (Yes, that was really it! Read the piece!) Below, I’m sharing Ashutosh’s moving essay about E. O. Wilson with Ashutosh’s very generous permission. —Scott Aaronson

Guest Post by Ashutosh Jogalekar

As some know, I was “fired” from Scientific American in 2014 for three “controversial” posts (among 200 that I had written for the magazine). When I parted from the magazine I chalked up my departure to an unfortunate misunderstanding more than anything else. I still respected some of the writers at the publication, and while I wore my separation as a badge of honor and in retrospect realized its liberating utility in enabling me to greatly expand my topical range, I occasionally still felt bad and wished things had gone differently.

No more. Now the magazine has done me a great favor by allowing me to wipe the slate of my conscience clean. What happened seven years ago was not just a misunderstanding but clearly one of many first warning signs of a calamitous slide into a decidedly unscientific, irrational and ideology-ridden universe of woke extremism. Its logical culmination two days ago was an absolutely shameless, confused, fact-free and purely ideological hit job on someone who wasn’t just a great childhood hero of mine but a leading light of science, literary achievement, humanism and biodiversity. While Ed (E. O.) Wilson’s memory was barely getting cemented only days after his death, the magazine published an op-ed calling him a racist, a hit job endorsed and cited by the editor-in-chief as “insightful”. One of the first things I did after reading the piece was buy a few Wilson books that weren’t part of my collection.

Ed Wilson was one of the gentlest, most eloquent, most brilliant and most determined advocates for both human and natural preservation you could find. Under Southern charm lay hidden unyielding doggedness and immense stamina combined with a missionary zeal to communicate the wonders of science to both his fellow biologists and the general public. His autobiography, “Naturalist”, is perhaps the finest, most literary statement of the scientific life I have read; it was one of a half dozen books that completely transported me when I read it in college. In book after book of wide-ranging intellectual treats threading through a stunning diversity of disciplines, he sent out clarion calls for saving the planet, for enabling dialogue between the natural and the social sciences, for understanding each other better. In the face of unprecedented challenges to our fragile environment and continued barriers to interdisciplinary communication, this is work that likely will make him go down in history as one of the most important human beings who ever lived, easily of the same caliber and achievement as John Muir or Thoreau. Even in terms of achievement strictly defined by accolades – the National Medal of Science, the Crafoord Prize which recognizes fields excluded by the Nobel Prize, and not just one but two Pulitzer Prizes – few scientists from any field in the 20th century can hold a candle to Ed Wilson. My friend Richard Rhodes who knew Wilson for decades as a close and much-admired friend said that there wasn’t a racist bone in his body; Dick should know since he just came out with a first-rate biography of Wilson weeks before his passing.

The writer who wrote that train wreck is a professor of nursing at UCSF named Monica McLemore. That itself is a frightening fact and should tell everyone how much ignorance has spread itself in our highest institutions. She not only maligned and completely misrepresented Wilson but did not say a word about his decades-long, heroic effort to preserve the planet and our relationship with it; it was clear that she had little acquaintance with Wilson’s words since she did not cite any. It’s also worth noting the gaping moral blindness in her article which completely misses the most moral thing Wilson did – spend decades advocating for saving our planet and averting a catastrophe of extinction, climate change and divisiveness – and instead focuses completely on his non-existent immorality. This is a pattern that is consistently found among those urging “social justice” or “equity” or whatever else: somehow they seem to spend all their time talking about fictional, imagined immorality while missing the real, flesh-and-bones morality that is often the basis of someone’s entire life’s work.

In the end, the simple fact is that McLemore didn’t care about any of this. She didn’t care because she had a political agenda and the facts did not matter to her, even facts as basic as the definition of the normal distribution in statistics. For her, Wilson was some obscure white male scientist who was venerated, and that was reason enough for a supposed “takedown”. And the editor of Scientific American supported and lauded this ignorant, ideology-driven tirade.

Ironically, Wilson would have found this ideological hit job all too familiar. After he wrote his famous book Sociobiology in the 1970s, a volume in which, in a single chapter about human beings, he had the temerity to suggest that maybe, just maybe, human beings operate with the same mix of genes that other creatures do, the book was met by a disgraceful, below-the-belt, ideological response from Wilson’s far left colleagues Richard Lewontin and Stephen Jay Gould who hysterically compared his arguments to thinking that was well on its way down the slippery slope to that dark world where lay the Nazi gas chambers. The gas chamber analogy is about the only thing that’s missing from the recent hit job, but the depressing thing is that we are fighting the same battles in 2021 that Wilson fought forty years before, although turbocharged this time by armies of faithful zombies on social media. The sad thing is that Wilson is no longer around to defend himself, although I am not sure he would have bothered with a piece as shoddy as this one.

The complete intellectual destruction of a once-great science magazine is now clear as day. No more should Scientific American be regarded as a vehicle for sober scientific views and liberal causes but as a political magazine with clearly stated ideological biases and an aversion to facts, an instrument of a blinkered woke political worldview that brooks no dissent. Scott Aaronson has taken a principled stand and said that after this proverbial last straw on the camel’s back, he will no longer write for the magazine or do interviews for them. I applaud Scott’s decision, and with his expertise it’s a decision that actually matters. As far as I am concerned, I now mix smoldering fury at the article with immense relief: the last seven years have clearly shown that leaving Scientific American in 2014 was akin to leaving the Soviet Union in the 1930s just before Stalin appointed Lysenko head biologist. I could not have asked for a happier expulsion and now feel completely vindicated and free of any modicum of regret I might have felt.

To my few friends and colleagues who still write for the magazine and whose opinions I continue to respect, I really wish to ask: Why? Is writing for a magazine which has sacrificed facts and the liberal voice of real science at the altar of political ideology and make believe still worth it? What would it take for you to say no more? As Oscar Wilde would say, one mistake like this is a mistake, two seems more like carelessness; in the roster of the last few years, this is “mistake” 100+, signaling that it’s now officially approved policy. Do you think that being an insider will allow you to salvage the reputation of the magazine? If you think that way, you are no different from the one or two moderate Republicans who think they can still salvage the once-great party of Lincoln and Eisenhower. Both the GOP and Scientific American are beyond redemption from where I stand. Get out, start your own magazine or join another, one which actually respects liberal, diverse voices and scientific facts; let us applaud you for it. You deserve better, the world deserves better. And Ed Wilson’s memory sure as hell deserves better.

Update (from Scott): See here for the Hacker News thread about this post. I was amused by the conjunction of two themes: (1) people who were uncomfortable with my and Ashutosh’s expression of strong emotions, and (2) people who actually clicked through to the SciAm hit-piece, and then reported back to the others that the strong emotions were completely, 100% justified in this case.

View Details

Happy New Year, everyone!

It was exactly two years ago that it first became publicly knowable—though most of us wouldn’t know for at least two more months—just how freakishly horrible is the branch of the wavefunction we’re on. I.e., that our branch wouldn’t just include Donald Trump as the US president, but simultaneously a global pandemic far worse than any in living memory, and a world-historically bungled response to that pandemic.

So it’s appropriate that I just finished reading Viral: The Search for the Origin of COVID-19, by Broad Institute genetics postdoc Alina Chan and science writer Matt Ridley. Briefly, I think that this is one of the most important books so far of the twenty-first century.

Of course, speculation and argument about the origin of COVID goes back all the way to that fateful January of 2020, and most of this book’s information was already available in fragmentary form elsewhere. And by their own judgment, Chan and Ridley don’t end their search with a smoking-gun: no Patient Zero, no Bat Zero, no security-cam footage of the beaker dropped on the Wuhan Institute of Virology floor. Nevertheless, as far as I’ve seen, this is the first analysis of COVID’s origin to treat the question with the full depth, gravity, and perspective that it deserves.

Viral is essentially a 300-page plea to follow every lead as if we actually wanted to get to the bottom of things, and in particular, yes, to take the possibility of a lab leak a hell of a lot more seriously than was publicly permitted in 2020. (Fortuitously, much of this shift already happened as the authors were writing the book, but in June 2021 I was still sneered at for discussing the lab leak hypothesis on this blog.) Viral is simultaneously a model of lucid, non-dumbed-down popular science writing and of cogent argumentation. The authors never once come across like tinfoil-hat-wearing conspiracy theorists, railing against the sheeple with their conventional wisdom: they’re simply investigators carefully laying out what they’re confident should become conventional wisdom, with the many uncertainties and error bars explicitly noted. If you read the book and your mind works anything like mine, be forewarned that you might come out agreeing with a lot of it.

I would say that Viral proves the following propositions beyond reasonable doubt:

• Virologists, including at Shi Zhengli’s group at WIV and at Peter Daszak’s EcoHealth Alliance, were engaged in unbelievably risky work, including collecting virus-laden fecal samples from thousands of bats in remote caves, transporting them to the dense population center of Wuhan, and modifying them to be more dangerous, e.g., through serial passage through human cells and the insertion of furin cleavage sites. Years before the COVID-19 outbreak, there were experts remarking on how risky this research was and trying to stop it. Had they known just how lax the biosecurity was in Wuhan—dangerous pathogens experimented on in BSL-2 labs, etc. etc.—they would have been louder.
• Even if it didn’t cause the pandemic, the massive effort to collect and enhance bat coronaviruses now appears to have been of dubious value. It did not lead to an actionable early warning about how bad COVID-19 was going to be, nor did it lead to useful treatments, vaccines, or mitigation measures, all of which came from other sources.
• There are multiple routes by which SARS-CoV2, or its progenitor, could’ve made its way, otherwise undetected, from the remote bat caves of Yunnan province or some other southern location to the city of Wuhan a thousand miles away, as it has to do in any plausible origin theory. Having said that, the regular Yunnan→Wuhan traffic in scientific samples of precisely these kinds of viruses, sustained over a decade, does stand out a bit! On the infamous coincidence of the pandemic starting practically next door to the world’s center for studying SARS-like coronaviruses, rather than near where the horseshoe bats live in the wild, Chan and Ridley memorably quote Humphrey Bogart’s line from Casablanca: “Of all the gin joints in all the towns in all the world, she walks into mine.”
• The seafood market was probably “just” an early superspreader site, rather than the site of the original spillover event. No bats or pangolins at all, and relatively few mammals of any kind, appear to have been sold at that market, and no sign of SARS-CoV2 was ever found in any of the animals despite searching.
• Most remarkably, Shi and Daszak have increasingly stonewalled, refusing to answer 100% reasonable questions from fellow virologists. They’ve acted more and more like defendants exercising their right to remain silent than like participants in a joint search for the truth. That might be understandable if they’d already answered ad nauseam and wearied of repeating themselves, but with many crucial questions, they haven’t answered even once. They’ve refused to make available a key database of all the viruses WIV had collected, which WIV inexplicably took offline in September 2019. When, in January 2020, Shi disclosed to the world that WIV had collected a virus called RaTG13, which was 96% identical to SARS-CoV2, she didn’t mention that it was collected from a mine in Mojiang, which the WIV had sampled from over and over because six workers had gotten a SARS-like pneumonia there in 2012 and three had died from it. She didn’t let on that her group had been studying RaTG13 for years—giving, instead, the false impression that they’d just noticed it recently, when searching WIV’s records for cousins of SARS-CoV2. And she didn’t see fit to mention that WIV had collected eight other coronaviruses resembling SARS-CoV2 from the same mine (!). Shi’s original papers on SARS-CoV2 also passed in silence over the virus’s furin cleavage site—even though SARS-CoV2 was the first sarbecoronavirus with that feature, and Shi herself had recently demonstrated adding furin cleavage sites to other viruses to make them more transmissible, and the cleavage site would’ve leapt out immediately to any coronavirus researcher as the most interesting feature of SARS-CoV2 and as key to its transmissibility. Some of these points had to be uncovered by Internet sleuths, poring over doctoral theses and the like, after which Shi would glancingly acknowledge the points in talks without ever explaining her earlier silences. Shi and Daszak refused to cooperate with Chan and Ridley’s book, and have stopped answering questions more generally. When people politely ask Daszak about these matters on Twitter, he blocks them.
• The Chinese regime has been every bit as obstructionist as you might expect: destroying samples, blocking credible investigations, censoring researchers, and preventing journalists from accessing the Mojiang mine. So Shi at least has the excuse that, even if she’d wanted to come clean with everything relevant she knows about WIV’s bat coronavirus work, she might not be able to do so without endangering herself or loved ones. Daszak has no such excuse.

It’s important to understand that, even in the worst case—that (1) there was a lab leak, and (2) Shi and Daszak are knowingly withholding information relevant to it—they’re far from monsters. Even in Viral‘s relentlessly unsparing account, they come across as genuine believers in their mission to protect the world from the next pandemic.

And it’s like: imagine devoting your life to that mission, having most of the world refuse to take you seriously, and then the calamity happens exactly like you said … except that, not only did your efforts fail to prevent it, but there’s a live possibility that they caused it. It’s conceivable that your life’s work managed to save minus 15 million lives and create minus $50 trillion in economic value.

Very few scientists in history have faced that sort of psychic burden, perhaps not even the ones who built the atomic bomb. I hope I’d maintain my scientific integrity under such an astronomical weight, but I’m doubtful that I would. Would you?

Viral very wisely never tries to psychoanalyze Shi and Daszak. I fear that one might need a lot of conceptual space between “knowing” and “not knowing,” “suspecting” and “not suspecting,” to do justice to the planet-sized enormity of what’s at stake here. Suppose, for example, that an initial investigation in January 2020 reassured you that SARS-CoV2 probably hadn’t come from your lab: would you continue trying to get to the bottom of things, or would you thereafter decide the matter was closed?

For all that, I agree with Chan and Ridley that COVID-19 might well have had a zoonotic origin after all. And one point Viral makes abundantly clear is that, if our goal is to prevent the next pandemic, then resolving the mystery of COVID-19 actually matters less than one might think. This is because, whichever possibility—zoonotic spillover or lab leak—turns out to be the truth of this case, the other possibility would remain absolutely terrifying and would demand urgent action as well. Read the book and see for yourself.

Searching my inbox, I found an email from April 16, 2020 where I told someone who’d asked me that the lab-leak hypothesis seemed perfectly plausible to me (albeit no more than plausible), that I couldn’t understand why it wasn’t being investigated more, but that I was hesitant to blog about these matters. As I wrote seven months ago, I now see my lack of courage on this as having been a personal failing. Obviously, I’m just a quantum computing theorist, not a biologist, so I don’t have to have any thoughts whatsoever about the origin of COVID-19 … but I did have some, and I didn’t share them here only because of the likelihood that I’d be called an idiot on social media. Having now read Chan and Ridley, though, I think I’d take being called an idiot for this book review more as a positive signal about my courage than as a negative signal about my reasoning skills!

At one level, Viral stands alongside, I dunno, Eichmann in Jerusalem among the saddest books I’ve ever read. It’s 300 pages of one of the great human tragedies of our lifetime balancing on a hinge between happening and not happening, and we all know how it turns out. On another level, though, Viral is optimistic. Like with Richard Feynman’s famous “personal appendix” about the Space Shuttle Challenger explosion, the very act of writing such a book reflects a view that you’re still allowed to ask questions; that one or two people armed with nothing but arguments can run rings around governments, newspapers, and international organizations; that we don’t yet live in a post-truth world.

View Details

I’m about to leave for a family vacation—our first such since before the pandemic, one planned and paid for literally the day before the news of Omicron broke. On the negative side, staring at the case-count graphs that are just now going vertical, I estimate a ~25% chance that at least one of us will get Omicron on this trip. On the positive side, I estimate a ~60% chance that in the next 6 months, at least one of us would’ve gotten Omicron or some other variant even without this trip—so maybe it’s just as well if we get it now, when we’re vaxxed to the maxx and ready and school and university are out.

If, however, I do end this trip dead in an ICU, I wouldn’t want to do so without having clearly set out my values for posterity. So with that in mind: in the comments of my previous post, someone asked me why I identify as a liberal or a progressive, if I passionately support educational practices like tracking, ability grouping, acceleration, and (especially) encouraging kids to learn advanced math whenever they’re ready for it. (Indeed, that might be my single stablest political view, having been held, for recognizably similar reasons, since I was about 5.)

Incidentally, that previous post was guest-written by my colleagues Edith Cohen and Boaz Barak, and linked to an open letter that now has almost 1500 signatories. Our goal was, and is, to fight the imminent dumbing-down of precollege math education in the United States, spearheaded by the so-called “California Mathematics Framework.” In our joint efforts, we’ve been careful with every word—making sure to maintain the assent of our entire list of signatories, to attract broad support, to stay narrowly focused on the issue at hand, and to bend over backwards to concede much as we could. Perhaps because of those cautions, we—amazingly—got some actual traction, reaching people in government (such as Rep. Ro Khanna, D – Silicon Valley) and technology leaders, and forcing the “no one’s allowed to take Algebra in 8th grade” faction to respond to us.

This was disorienting to me. On this blog, I’m used just to howling into the wind, having some agree, some disagree, some take to Twitter to denounce me, but in any case, having no effect of any kind on the real world.

So let me return to howling into the wind. And return to the question of what I “am” in ideology-space, which doesn’t have an obvious answer.

It’s like, what do you call someone who’s absolutely terrified about global warming, and who thinks the best response would’ve been (and actually, still is) a historic surge in nuclear energy, possibly with geoengineering to tide us over?

… who wants to end world hunger … and do it using GMO crops?

… who wants to smash systems of entrenched privilege in college admissions … and believes that the SAT and other standardized tests are the best tools ever invented for that purpose?

… who feels a personal distaste for free markets, for the triviality of what they so often elevate and the depth of what they let languish, but tolerates them because they’ve done more than anything else to lift up the world’s poor?

… who’s happiest when telling the truth for the cause of social justice … but who, if told to lie for the cause of social justice, will probably choose silence or even, if pushed hard enough, truth?

… who wants to legalize marijuana and psychedelics, and also legalize all the promising treatments currently languishing in FDA approval hell?

… who feels little attraction to the truth-claims of the world’s ancient religions, except insofar as they sometimes serve as prophylactics against newer and now even more virulent religions?

… who thinks the covid response of the CDC, FDA, and other authorities was a historic disgrace—not because it infringed on the personal liberties of antivaxxers or anything like that, but on the contrary, because it was weak, timid, bureaucratic, and slow, where it should’ve been like that of a general at war?

… who thinks the Nazi Holocaust was even worse than the mainstream holds it to be, because in addition to the staggering, one-lifetime-isn’t-enough-to-internalize-it human tragedy, the Holocaust also sent up into smoke whatever cultural process had just produced Einstein, von Neumann, Bohr, Szilard, Born, Meitner, Wigner, Haber, Pauli, Cantor, Hausdorff, Ulam, Tarski, Erdös, and Noether, and with it, one of the wellsprings of our technological civilization?

… who supports free speech, to the point of proudly tolerating views that really, actually disgust them at their workplace, university, or online forum?

… who believes in patriotism, the police, the rule of law, to the extent that they don’t understand why all the enablers of the January 6 insurrection, up to and including Trump, aren’t currently facing trial for treason against the United States?

… who’s (of course) disgusted to the core by Trump and everything he represents, but who’s also disgusted by the elite virtue-signalling hypocrisy that made the rise of a Trump-like backlash figure predictable?

… who not only supports abortion rights, but also looks forward to a near future when parents, if they choose, are free to use embryo selection to make their children happier, smarter, healthier, and free of life-crippling diseases (unless the “bioethicists” destroy that future, as a previous generation of Deep Thinkers destroyed our nuclear future)?

… who, when reading about the 1960s Sexual Revolution, instinctively sides with free-loving hippies and against the scolds … even if today’s scolds are themselves former hippies, or intellectual descendants thereof, who now clothe their denunciations of other people’s gross, creepy sexual desires in the garb of feminism and social justice?

What, finally, do you call someone whose image of an ideal world might include a young Black woman wearing a hijab, an old Orthodox man with black hat and sidecurls, a broad-shouldered white guy from the backwoods of Alabama, and a trans woman with purple hair, face tattoos and a nose ring … all of them standing in front of a blackboard and arguing about what would happen if Alice and Bob jumped into opposite ends of a wormhole?

Do you call such a person “liberal,” “progressive,” “center-left,” “centrist,” “Pinkerite,” “technocratic,” “neoliberal,” “libertarian-ish,” “classical liberal”? Why not simply call them “correct”?