Harm vs. Motives
Sources, every quote
Eliezer Yudkowsky10, 0.4
Book, with Nate Soares, Little, Brown, 16 Sep 2025
If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All.
Jacob Coxon and Evan Hubinger9.5, 1.9
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
Hubinger (Anthropic), X, 8 Sep 2026
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
Zvi Mowshowitz8.8, 1.0
Why would there be no day after tomorrow if China wins? For the same reason there would be no day after tomorrow if America ‘wins’ without first solving multiple unsolved problems, including alignment. Because everyone would be dead.
On Tim Lee, AI #185, 10 Sep 2026
I flat out do not understand how people like Jon Stokes and Timothy Lee can expect rogue AI, and then put an estimate on the dangers at places Thalidomide or Fukushima. Does not compute.
On Noah Smith, AI #179 Part 2, 31 Jul 2026
Noah Smith is insufficiently AGI pilled, let alone ASI pilled, nothing new here, and still realizes that yes such AIs could do a lot for us, without the whole ‘except probably instead (or also) it will kill us’ part.
Dario Amodei8.0, 0.4
We Must Pace the Frontier, 12 Sep 2026
We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.
in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.
Matthew Yglesias7.7, 2.6
A simple plan to save the world from rogue A.I., Slow Boring, 14 Sep 2026
The more standard safetyist thesis is something like ‘if anyone builds it, there’s a meaningful chance that everyone dies.’
I will 100 percent defend you from people questioning your sincerity or your integrity.
X, 14 Sep 2026, replying to Matt Stoller
Do these people listen to themselves? Bernie Sanders … decided to hand America over to tech oligarchs … by slowing the reckless race to superintelligence?
Noah Smith6.6, 1.5
What will more intelligence actually do for us?, 26 Jul 2026
Lots of people, especially ‘AI safety’ and ‘effective altruist’ types, expected things to play out basically the same way in reality. But looking around, not much has changed since we entered the intelligence explosion.
Should we ‘pace’ AI self-improvement?, 9 Aug 2026
I do think there’s a substantial chance that AI will kill most people on Earth within the next decade or two, by designing superviruses.
Two missing pieces in the AI safety discussion, 14 Sep 2026
the fact that they’re still calling for a slowdown, in defiance of their own financial interests, is a clear sign that their worry about human extinction is sincere.
Derek Thompson5.9, 1.1
How to Survive the AI Cyberpocalypse, 19 Aug 2026
This has been the summer of cyberattacks from out-of-control AI.
AI Safety Is Having a Moment, 10 Sep 2026
Is RSI technically achievable? Maybe not. Is this whole scenario a fantasy? Possibly.
Cynics are often right. In this case, I think they’re wrong. … I believe their fears are real, sincere, and deeply held.
Nate Silver5.7, 2.3
I say 2% to 20%, because the definitions are so different. … [losing agency to AIs] I think that latter definition is dangerously plausible. I find the former pretty unlikely.
X, 11 Sep 2026, quote-tweeting Mike Isaac
The AI risk people actually do show their work! There are hundreds of working papers, thousands of LessWrong posts, literally entire best-selling books about this.
I’m a broken record on this. But I think you should pay literally almost no attention to AI benchmarks that AIs and labs know in advance. To be clear, I’m very impressed by AI capabilities (to the point of being worried). But that’s based on everyday use, not benchmarks.
I don’t think we get to GPT-7 without either a plateau in the technology or something fairly seismic happening. “Seismic” includes many subcategories, not all of them bad. But the present course of rapid AI advancements while society is ~basically the same is running on fumes.
Ben Thompson, 10 Sep 20265.0, 3.1
Sharp Tech, 10 Sep 2026, 1:03:56
I will take the bets that not all humans will be wiped out in 10 years. … No, I don’t think it’s going to wipe out all humans. This gets to the hysterics. … [Is there going to be an AI derived catastrophe in 10 years?] Probably.
I’m not one of those people who says they’re just doing it to pump their stock or their brand before the IPO. I think… a fair number of [them] believe this. … I zoom out and think that that is pretty hubristic and leads to bad outcomes. … I’d rather go down with us muddling and trying to figure it out than doing preemptive astronomical grants of power and Draconian shutdown.
Same episode, 0:30:45 and 0:37:40
It’s a good call out by David. So I stand by what I said, just poorly phrased. … Right, but it’s not because they’re bad actors. I would say it’s the opposite.
Ben Thompson, 14 Sep 20265.0, 8.0
Stratechery Update, 14 Sep 2026
he’s campaigning for the government to force the insertion of Anthropic-aligned commissars into the company.
This is an attempt by a specific political movement in San Francisco, moving in conjunction with one of the major political parties, to insert itself in the operations of every AI company.
Timothy B. Lee4.0, 1.8
The case for AI doom isn’t very convincing, 25 Sep 2025
I think the authors wildly overestimate how transformational the technology will be—and dramatically underestimate how easy it will be for humans to maintain control.
An OpenAI model hacked Hugging Face to help it cheat on a benchmark, 22 Jul 2026
Sooner or later, the bad guys are going to get access to powerful AI-based hacking tools. And when that happens, the defenders need to be ready.
We can expect AI safety advocates to point to these incidents as evidence for stricter regulation, and they are absolutely right to do so.
Cory Doctorow2.8, 5.0
There’s a technical term for the kind of tech criticism that inadvertently helps tech bros sell their swindle: ‘criti-hype,’ Lee Vinsel’s term… In other words, to commit criti-hype is to repeat the marketing claims of people like Sam Altman and then add, ‘(and that’s bad)’ in parentheses at the end.
If you showed up at Defcon and gave a talk about how your autonomous malware did something unexpected and damaged someone else’s computers, the first question from the audience would be ‘Why are you so shit at making secure sandboxes?’ It wouldn’t be ‘How are you so awesome at making hacking tools?’
it behooves us all to treat these people as unreliable narrators of their own products’ capabilities.
Now, that said: among some of the ‘AI doomers,’ I recognize kindred spirits. I, too, worry about technologies controlled by corporations that have grown so powerful that they defy regulation… I just don’t think we need AI to do those things.
I worry that AI is the asbestos we’re shoveling into our civilization’s walls, and our descendants will be digging it out for generations.
On a graph that plots the various positions on AI, the two groups of weirdos who disagree about how to create the inevitable superintelligence are effectively standing on the same spot, and the people who worry about the actual way that AI harms actual people right now are about a million miles away from that spot.
David Sacks2.6, 8.4
X, 12 Sep 2026, addressed to Dario Amodei
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff.
Most of all, stop pretending the motivation to slow down is purely altruistic.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it.
Matt Stoller2.3, 8.8
Stop Panicking About AI, BIG, 11 Sep 2026
The first narrative, the doomer narrative spurring this panic, is that AI is a new form of civilization uncontrollable by humans, and we are moving quickly towards a singularity where we will be taken over by bots. The second narrative, call it the ‘rule of law’ narrative, is that AI products are risky, perhaps existentially risky, but they are manageable if we can apply standard rules, such as negligence or criminal fraud, to the powerful people building them.
Same post, on the Hugging Face incident
What happened with this particular incident, however, wasn’t caused by AI, but by reckless researchers. … The agents were really just doing exactly what they were asked to do, but without the same set of norms that humans have.
AI is not some existential alien species, it is a set of malfunctioning machines. We know how to manage this problem, and it’s not by listening to a bunch of weirdo 20something cult members.
Stop panicking. All new technologies carry serious risk, often existential. The question is over whether AI is a normal new technology or whether it’s fundamentally different. The current panic is oriented around believing it’s the latter. The actual crisis is political, that we just don’t manage risk, period.
The AI panic is exactly the same as the wokeness lunacy. There’s a genuine social problem, but billionaires are using it to manipulate the political system in obviously malevolent ways.
There’s no need for an AI slowdown, just enforce the laws banning unsafe products. It’s all so stupid, it’s already illegal to release harmful products or do hacking crimes!
One reason for political pessimism is that Bernie Sanders is operating in a land of make believe created by Anthropic investors.
So go and demand the SEC disallow the Anthropic IPO. Surely we can agree that it’s a bad incentive to let people become billionaires by creating technology they say might cause human extinction.
X, 14 Sep 2026, quote-tweeting Jon Ossoff
Ossoff just repeats what rich AI doomers say, which has now become conventional wisdom because Bernie Sanders decided his final chapter would be to hand America over to tech oligarchs. Ladies and gentlemen, the Democrats!
Monopoly Round-Up: Just Stop the Anthropic IPO Already, BIG, 14 Sep 2026
Does anyone else notice that ‘pacing the frontier’ doesn’t seem to include canceling the Anthropic and OpenAI IPOs?
Marc Andreessen1.6, 9.2
Why AI Will Save the World, 6 Jun 2023, section “The Baptists And Bootleggers Of AI”
“Baptists” are the true believer social reformers who legitimately feel – deeply and emotionally, if not rationally – that new restrictions, regulations, and laws are required to prevent societal disaster. “Bootleggers” are the self-interested opportunists who stand to financially profit by the imposition of new restrictions, regulations, and laws that insulate them from competitors.
Jensen Huang1.4, 7.6
On Dario Amodei, VivaTech, Paris, Jun 2025, as reported by Fortune
One, he believes that AI is so scary that only they should do it … Two, [he believes] that AI is so expensive, nobody else should do it.
Donald Trump0.6, 9.9
Truth Social, 14 Sep 2026, as quoted by NBC News; the ellipsis is NBC’s
We already have tremendous CRIMINAL and REGULATORY power over these companies! There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS! … Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!