The singularity might be my favorite idea in all of AI, and it has a real, specific meaning that I find way more interesting than the way the word usually gets used.
That's it. That's the whole idea, and notice what kind of idea it is: it's a claim about us.
Meanwhile the word gets thrown around as a synonym for AGI. Also for ASI. Also for "the day AI gets really good," or "the day the robots take over," or just a vague wave at some future where everything is weird. People tell me we must be getting close because some benchmark just fell, and I'm like, what are you even measuring? The whole idea is about the place where measurement gives out.
Where the word comes from Most people learned the word from Kurzweil. The Singularity Is Near came out in 2005, sold a mountain of copies, and pretty much welded the word to his particular brand of futurism.
He was about fifty years late.
The first use I know of is from 1958. Stanislaw Ulam wrote a tribute to John von Neumann after he died, and while remembering their conversations he dropped this:
Von Neumann died in 1957, which is why the earliest source we have is a friend quoting him from memory.
One conversation centered on the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue. Stanislaw Ulam, Tribute to John von Neumann, 1958
Two years after the Dartmouth workshop that gave us the term "artificial intelligence."Read the end of that again. Beyond which human affairs, as we know them, could not continue. That's a way bigger claim than technology speeding up, and von Neumann was making it decades before anyone could have meant AI by it.
He was looking at vacuum tubes and punch cards, basically, and still saw the horizon.
The best definition is Vinge's Vernor Vinge taught math and computer science at San Diego State, and he wrote science fiction on the side, which I think is exactly the right mix for this question. In 1993 he gave a paper at a NASA symposium called The Coming Technological Singularity, and that's where a passing remark in a tribute turns into an actual argument.
His opening is about as blunt as anything I've ever seen in an academic paper:
Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended. Vernor Vinge, The Coming Technological Singularity, 1993
But the line that matters—and the reason he picked that particular word—is this one:
In physics, a singularity is where your equations stop giving answers. Vinge is saying the same thing happens to our models of the future once something smarter than us is doing the driving: you keep making predictions, and they just stop coming out.
Here's how I picture it. Forecasting is driving at night: your headlights reach maybe a hundred meters, better data pushes the light a little further out, and the singularity is the moment the black starts at the windshield.
Science fiction writers keep getting to these ideas first, and I think it's because they're paid to take them seriously all the way to the end.I love this definition, and I think grabbing that word from physics might be the best naming decision anyone in this field ever made, because the metaphor smuggles the whole argument in with it. It's an admission that we go blind. It puts the claim exactly where it belongs: on us.
And here's the part almost everyone forgets. Vinge listed four ways superhuman intelligence might show up, and AI is only the first two:
Three of those still have humans in them! That's a much wider frame than the one that survived into the current conversation.
Why it would happen fast There's a third name in this story, and he's the one who explains the mechanism.
In 1965 I.J. Good, a statistician who worked with Turing at Bletchley Park, described what he called an "ultraintelligent machine": a machine good enough at design to build a better machine than itself, which then builds a better one, which then builds a better one.
Imagine having "worked with Turing at Bletchley Park" on your resume.Good called the result an "intelligence explosion." Still the best two words for it.That loop is why people expect this to arrive all at once. The loop feeds itself.
So the lineage goes something like: von Neumann saw the horizon, Good explained why we'd hit it fast, and Vinge told us what it actually is.
My definitions of AGI and ASI Here are the definitions I've settled on, after years of going back and forth on them. I go deeper in My Updated Definitions of AGI vs. ASI.
🧠 "G" is for General, and "S" is for Super.Both of those describe a machine. You could put one on a bench and measure it.
How they fit together The singularity works differently, and I think this distinction is basically the whole reason to care about any of this. AGI and ASI are about the machine. The singularity is about us—specifically, about the moment we lose the ability to see what's coming.
Which means you can have one without the other.
Think about what AGI actually gets you: something as good as a human expert at every cognitive role. That would rearrange the entire economy, and we could still reason about it, because we already know what doctors and lawyers and engineers and analysts do all day—we've been watching human experts work for a few thousand years. It's a world with a billion more professionals in it. Enormous. Also forecastable.
ASI is where it breaks. Once something can do any cognitive role better than any human ever has, it stops being a tool you point at problems and starts being the thing that decides what to do about them, and predicting what a smarter thing will decide would require being as smart as it. Which we just established we aren't.
You can't predict the moves of a chess player who's better than you. That's most of what "better" means.Why I love this idea This is honestly one of the coolest ideas in the entire space, and it gets flattened into a buzzword pretty much daily.
Knowing where it came from helps, because the idea predates the current hype cycle by decades. Von Neumann was talking about it in the fifties. Vinge wrote it up in 1993. Both of them were just thinking out loud, and they got to the heart of it long before anyone had a benchmark to argue over.
And knowing it's a different thing from AGI or ASI changes what you're actually arguing about. When someone says we're close to the singularity, they're making a claim about our forecasting: that we're near the point where our models of the future stop working. What do benchmark scores tell you about that? Almost nothing.
Benchmark scores are the speedometer. The singularity is about the headlights.That framing is a lot more unsettling than a capabilities debate, and I think it's the right one to be having.
So the next time the word comes up, ask which of the three things is being claimed—AGI, ASI, or the horizon itself. I'm going to keep chewing on this one, and if you have a better frame for it, send it my way.
Summary 1. People use the singularity as a synonym for AGI and ASI, and it means something different from both 2. The word goes back to 1958, in Ulam's tribute to von Neumann, describing a point "beyond which human affairs, as we know them, could not continue" 3. Vinge gave it the definition worth keeping, in 1993: "It is a point where our models must be discarded and a new reality rules" 4. I.J. Good supplied the mechanism in 1965 with the ultraintelligent machine that designs its own successor, which is why we'd expect it to hit suddenly 5. AGI and ASI describe a machine's capability, and the singularity describes our own blindness, which makes it a completely different kind of claim 6. The main thing: the singularity is the point past which we can't really predict anything, because at that point it's essentially ASIs running things
Notes1. Ulam's tribute: Stanislaw Ulam, "John von Neumann 1903-1957," Bulletin of the American Mathematical Society, vol. 64, no. 3, May 1958. Earliest known use of "singularity" in this sense. 2. Vinge's paper: The Coming Technological Singularity: How to Survive in the Post-Human Era, presented at the VISION-21 Symposium sponsored by NASA Lewis Research Center and the Ohio Aerospace Institute, March 1993. 3. I.J. Good: "Speculations Concerning the First Ultraintelligent Machine," Advances in Computers, vol. 6, 1965. 4. Vinge's thirty-year window from 1993 has basically closed, and his specific claim remains outstanding. That's also the standing problem with the whole concept: you can only identify a prediction horizon from the far side of it. 5. 🤖 AIL 3: I gave Kai the thesis, the structure, the definitions to include, and the ending. He did the historical research, wrote the prose, and made the header image. Learn more about AIL