Trends
Two years ago, 390 people a month typed "ai apocalypse" into Google. As of August 2026 it is 1,300, and the peak was 1,900 back in March. That peak is the part nobody covering this story has noticed: the term hit its high six months before the resignation thread that put AI extinction back on the front pages.
So the curve is not the drama. The drama is the September thread from a departing Anthropic researcher, the confirmation from his own alignment lead that the company believes AI could kill everyone, and the ten days of coverage that followed. Underneath it is a much older argument with a real number attached, the survey that produced it, and an audit nobody is quoting that grades the loudest lab zero on the one practice that matters.
Key takeaways:
- "ai apocalypse" runs at 1,300 US searches a month as of Aug 2026, up 233% in two years and 665% in five, classified RAPID in our database. It peaked at 1,900 in March 2026, half a year before the September story.
- The trigger was a public resignation. Anthropic pretraining researcher Jacob Coxon posted on September 9 that the labs are "racing straight to self-improving superintelligence and gambling with our lives." The post has 802,728 likes. His colleague Evan Hubinger, who leads alignment science at Anthropic, replied that the chance is "greater than 10% within the next decade."
- The 10% is not his invention. A survey of 1,580 AI researchers published this month puts the median probability of human extinction or permanent severe disempowerment at exactly 10%, with an 18% mean and a middle-50% range of 1% to 25%.
- It is also not a measurement. No method produced it, and 12% of those researchers put the figure at zero.
- The audit nobody is quoting. An August assessment of five frontier labs graded Anthropic 0 out of 3 on having a plan to contain a misaligned model while grading it C+ overall. OpenAI scored 3 on that same practice.
- The risk vocabulary is growing and the capability vocabulary is not. "ai extinction risk" is up 467% in a year and "ai safety" up 132%, while "agi" and "superintelligence" are both down 33%.
Let's get into it.
The curve
Here is five years of monthly search volume for "ai apocalypse," straight from our database. It has two humps, and they are not the ones you would guess.
The first hump is mid-2023, the ChatGPT-era panic, and it decayed through 2024. The second starts in January 2025, our breakout month, and it climbs to 1,900 in March 2026 before falling back to 1,300 by August. Note the date: our latest month closes before the September resignation, and the term was already at 2.4x its two-year-ago level without it.
The more interesting comparison is sideways, across the words people use for the same subject.
"agi" at 90,500 a month dwarfs the entire risk vocabulary put together. People search for the capability roughly 70 times more often than they search for the catastrophe. And the direction of travel splits cleanly.
Every term about danger is up. ai extinction risk is up 467% in a year, off a base of only 170 searches, and ai alignment is up 23%. Every term about capability is down, including agi at minus 33%. That is what a story moving from the engineering press to the general public looks like: the jargon fades, the stakes language grows.
What "AI apocalypse" actually means
Strip the movie references and the claim has three parts. One: AI systems keep getting more capable at roughly the current rate. Two: at some capability level, a system can improve itself, and control passes from the people who built it. Three: a system not reliably aligned with human interests, operating at that level, can cause harm at civilizational scale, up to and including extinction.
The canonical version is one sentence, published by the Center for AI Safety in 2023 and still the document every round of the argument re-runs.
The statement reads: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." What makes it load-bearing is the signatory list on that page, which includes Geoffrey Hinton, Yoshua Bengio, and the chief executives of Anthropic, OpenAI and Google DeepMind. The people running the labs signed the extinction warning about their own product three years ago. That is why the September thread was a confirmation and not a revelation.
The term to keep separate is "superintelligence," which names the capability, not the outcome. Our piece on world models covers the research paths people think lead there. "AI apocalypse" is the branch where alignment does not hold.
Why it's breaking out now
On the evening of September 8, Jacob Coxon posted that he had resigned from Anthropic after three years of pretraining research there and at OpenAI. "Neither company is acting responsibly," he wrote. "They are racing straight to self-improving superintelligence and gambling with our lives." The post has 802,728 likes as of September 22. His sharpest line was about where the decision is being made: accepting the race, he wrote, "is a hubristic gamble that should not be launched from a private company's Slack."
An hour and a half later his colleague Evan Hubinger, who leads alignment science at Anthropic, replied under his own name. Hubinger confirmed that "we really do earnestly believe AI could kill all humans," put the chance at greater than 10% within the decade, and added that Anthropic does "not yet have a plan to solve alignment for superintelligence and are not clearly on track to." That post has 59,351 likes. An employee putting a double-digit extinction probability on his employer's product, by name, is what turned a resignation into ten days of coverage.
The day after, Anthropic published its own September threat report, a casebook of misuse it detected and disrupted between December 2025 and August 2026 across seven harm areas. It is the most concrete document in this argument, and it is about today's models rather than future ones. In May 2026 its biological classifier blocked a request to help write a grant application for gain-of-function work on the chikungunya virus aimed at transmissibility and immune evasion. On the cyber side, a Russian espionage actor used AI across the entire attack chain and exfiltrated more than 300,000 national identity records from one government authority.
The admission buried in that report matters more than any probability. Anthropic writes that older models were "well below the threshold where they could meaningfully assist a sophisticated user in carrying out dangerous biological research," but that "for today's models ... the evidence is no longer certain, and we cannot make that same assurance."
That gap between the far-future claim and the present-tense evidence is what the response on social media latched onto. This clip from @thinkwithv, posted September 18, makes the point in fifteen seconds.
"The AI apocalypse is getting all the oxygen," the caption reads. "Meanwhile, the risks already in the room still need rules, scrutiny, and people willing to act." Across the eight terms in our family chart, that argument is drawing 108,170 searches a month combined, as of Aug 2026.
Where the 10% actually comes from
Hubinger's number is not a personal quirk. Days after his post, AI Impacts published the 2024 edition of its Expert Survey on Progress in AI, which asked 1,580 researchers who publish at top AI venues to put a probability on human extinction or similarly permanent and severe disempowerment from advanced AI.
| Measure | Value |
|---|---|
| Respondents (survey fielded 2024, published Sep 2026) | 1,580 researchers |
| Mean probability of extinction or severe disempowerment | 18% |
| Median probability | 10% |
| Share assigning at least 10% | 51% |
| Share assigning at least 1% | 81% |
| Share assigning exactly zero | 12% |
| Middle 50% of estimates | 1% to 25% |
| Favour more prioritisation of risk research | 72% |
So Hubinger sits on the median of his field. The same survey reports that the year by which human-level machine intelligence reaches a 50% chance has moved from 2061 in the 2016 edition to 2042 in the 2024 edition. Researchers were near-evenly split on pace: 34% would be most optimistic if progress were faster, 34% if it were slower, 29% at the current speed.
Read the distribution rather than the headline and the picture changes. A field whose middle half ranges from 1% to 25%, with 12% at zero, does not have a consensus estimate. It has an enormous disagreement that happens to have a 10% midpoint.
Who is doing something about it
The ecosystem around this claim is now institutional, and each part of it has a number.
| Player | What it does | The number |
|---|---|---|
| Center for AI Safety | Owns the canonical one-sentence statement | Signed by the CEOs of Anthropic, OpenAI and Google DeepMind |
| AI Impacts | Runs the survey that supplies the 10% | 1,580 researchers, running since 2016 |
| Guidelight AI Standards | Grades labs on control practices | 5 labs assessed, best grade C+ |
| Anthropic | Publishes the most detailed misuse reporting | 7 harm areas over 9 months |
| Sanders and Casar | The legislative response | Up to 20 years in prison for individuals |
The legislation is the newest piece. On September 3, Senator Bernie Sanders and Representative Greg Casar announced the Ban Artificial Superintelligence Act, which would permanently prohibit superintelligent AI, pause advanced AI development until a new cabinet-level regulator writes rules, and set penalties the release compares to those for unlawfully developing nuclear weapons: a corporate death penalty for entities, up to 20 years in prison for individuals.
The labs are not silent either. Anthropic's chief executive published an essay in January called The Adolescence of Technology that takes the risks seriously while rejecting what he calls doomerism, meaning treating AI risk "in a quasi-religious way." His own line is the honest summary: "we are considerably closer to real danger in 2026 than we were in 2023."
What changes if the argument wins
Compliance becomes a product line. In every previous safety panic, the winners sold the remedy. Evaluation, red-teaming, monitoring and containment tooling are already where the money moved once agents went into production, the shift we traced in our 2026 AI agents report. A statutory pause would accelerate it by a year.
Disclosure gets expensive. Anthropic confidentially submitted a draft S-1 to the SEC on June 1, 2026, and the prospectus is not public, so nobody outside can check how it describes catastrophic risk. An employee publicly assigning a greater-than-10% extinction probability to the product is now a question for a securities lawyer, not a philosophical position.
Enterprise buying slows before regulation does. When a vendor publishes that it "cannot make that same assurance" about biological capability, the next renewal conversation changes, regardless of what any bill does. That is the quieter version of the shake-out we covered in the SaaS apocalypse, and it is why generative AI budget lines are increasingly written with exit clauses.
The sceptic's case, with a name on it
The strongest objection is not that the risk is zero. It is that the number is not a measurement.
Robert Brunner, chief disruption officer at the Gies College of Business at the University of Illinois Urbana-Champaign, made the case in a September 17 interview with the university's news bureau. "No measurement produced the 10%," he said. "Nor has anyone demonstrated a working path from today's models to superintelligence." His objection is procedural: we put probabilities on asteroid impacts using orbital measurements anyone with the data can check, and nothing equivalent exists here, so the figure is "a personal estimate rather than a measured probability."
He also names the cost of running this round repeatedly. "It's a claim that cannot be measured, cannot be questioned without seeming callous and has seemingly put itself beyond scientific debate." His fix is a common scale, like the Torino scale astronomers use for asteroids, that rates every incident on one axis including the ones that turn out to be nothing.
The evidence supports him on the measurement gap and undercuts the labs at the same time. Guidelight AI Standards assessed five frontier developers in August against six foundational control practices, using only public materials. The results are the most uncomfortable numbers in this story.
| Lab | Overall grade | Containment plan for a misaligned model |
|---|---|---|
| Anthropic | C+ (2.50) | 0, not implemented |
| OpenAI | C+ (2.50) | 3, substantial partial implementation |
| D+ (1.50) | 2, limited partial implementation | |
| xAI | D- (0.83) | 1, precursors only |
| Meta | F (0.67) | 0, not implemented |
Read that next to Hubinger's post. The company whose alignment lead publicly puts extinction above 10% scores zero on having a published plan to contain a misaligned model, and its own report says it can no longer rule out meaningful biological uplift from current systems. Guidelight's summary is that basic practices for keeping control of AI are, at most, partially implemented. That is a stronger argument for slowing down than any probability estimate, and it is checkable.
The public, for what it is worth, did not need the September thread. An Economist and YouGov poll fielded in May found 71% of Americans already thought AI development was moving too fast, against 27% who said the pace was about right.
What to watch
Whether September shows up in the curve. Our next readings for ai apocalypse will be the first to include the resignation. If the term clears its March high of 1,900, the story moved demand. If it does not, the March peak stands as evidence that this is a slow-burn public concern rather than a news cycle.
Whether the small terms stay small. "ai extinction risk" at 170 searches a month is up 467% and still tiny. A specialist term crossing a few thousand searches is how you tell a concept has arrived; one that grows fast and stays under 500 is a phrase journalists use, not one the public does.
Whether the containment zero becomes a one. A lab that publishes an actual containment plan moves that score, and it would be the first case of this argument producing an auditable change rather than another statement. Watch that cell before you watch the bills.
The honest summary is uncomfortable for both sides. The extinction number is real in the sense that half the field endorses it and hollow in the sense that nothing measured it. The near-term harms are measured, documented by the labs themselves, and getting less coverage than the speculation. Meanwhile 1,300 people a month type the phrase into Google and 90,500 type "agi." The apocalypse is a smaller search story than the capability that would cause it.
Want to see which AI terms are actually breaking out before the coverage arrives? Read our guide on how to identify market trends, follow the live ai apocalypse trend page, or browse what is breaking out right now on the Rising Trends dashboard.
