Mato just raised pre-seed Read more
Podcast charts
Published by Liron Shapira
It's time to talk about the end of the world. With your host, Liron Shapira. lironshapira.substack.com
On the charts
Every published chart this podcast appears in, in the snapshot behind this page. Each one links to the chart it came off.
The Mato Topic Intelligence Platform does not report this podcast as charting in a published category in this snapshot.
From the feed
The latest episodes published to this podcast’s own RSS feed. Titles and descriptions are the publisher’s.
What happens when AI passes our tests faster than we create new ones? Adam Khoja and Richard Ren helped build Humanity’s Last Exam. They return to explain what the disappearing benchmarks tell us about the future, and why smarter AI doesn’t automatically mean safer AI. Adam Khoja and Richard Ren are research engineers at the Center for AI Safety (CAIS). We go through the benchmarks they’ve built — Humanity’s Last Exam, the Remote Labor Index, MASK — and the safety-washing problem, where AI companies pass off raw capability gains as safety progress. Then we get to their forecasts: when AI outperforms research mathematicians, whether there will be a billion general-purpose robots by 2035, and why they put 80% odds that historians will judge we faced at least a 33% chance of catastrophe. Watch on YouTube: https://www.youtube.com/watch?v=gBPzgJkT9e8 Timestamps 00:00:00 — Cold Open 00:00:42 — Adam Khoja and Richard Ren Return 00:04:57 — Humanity’s Last Exam 00:14:47 — AI Outrunning Its Benchmarks 00:20:24 — The Remote Labor Index 00:27:59 — AI and the Next Human Job 00:30:08 — MASK: Catching AI in a Lie 00:38:06 — Safety Washing: Capabilities Passed Off as Safety 00:44:30 — Ethical Knowledge vs. Ethical Behavior 00:48:58 — Forecasting AI from 2026 to 2060 00:53:53 — AI vs. Research Mathematicians 00:55:53 — Putting a Number on P(Doom) 01:04:30 — A Billion Robots by 2035 01:08:09 — Dyson Swarms and the Physical Singularity 01:09:56 — Risk, Precision, and Taking Action Links Adam Khoja's first Doom Debates appearance — https://www.youtube.com/watch?v=QqESBXuo6EI Richard Ren's first Doom Debates appearance — https://www.youtube.com/watch?v=1glFImnyp6o Collision — Richard Ren's Substack — https://richardren.substack.com/ "Predictions on AI (2026–2060)" — Adam and Richard's forecasts, written December 2025, with resolution status — https://richardren.substack.com/p/predictions-on-ai-20262060 Manifold Markets — where Adam built his forecasting track record — https://manifold.markets/ Center for AI Safety — https://safe.ai/ Center for AI Safety — careers — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — https://safe.ai/work/statement-on-ai-risk Humanity's Last Exam — the 2,500-question closed-book exam at the frontier of human knowledge — https://agi.safe.ai/ "Humanity's Last Exam" (paper) — https://arxiv.org/abs/2501.14249 Remote Labor Index — real Upwork projects, measuring what fraction of remote work AI can actually finish — https://www.remotelabor.ai/ "Remote Labor Index: Measuring AI Automation of Remote Work" (paper) — https://arxiv.org/abs/2510.26787 MASK — the honesty benchmark: does a model contradict its own stated beliefs under pressure? — https://www.mask-benchmark.ai/ "The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" (paper) — https://arxiv.org/abs/2503.03750 "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the paper behind the safety-washing segment — https://arxiv.org/abs/2407.21792 WMDP — the weaponization benchmark for bio, cyber, and chem — https://www.wmdp.ai/ "The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning" (paper) — https://arxiv.org/abs/2403.03218 "Measuring Massive Multitask Language Understanding" (MMLU) — Dan Hendrycks et al., 2020 — https://arxiv.org/abs/2009.03300 METR, "Measuring AI Ability to Complete Long Tasks" — the time-horizon graph — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ FrontierMath — Epoch AI's research-level math benchmark — https://epoch.ai/frontiermath ARC Prize — https://arcprize.org/ SWE-bench Verified — https://openai.com/index/introducing-swe-bench-verified/ AI 2027 — https://ai-2027.com/ Liron Reacts to Subbarao Kambhampati on Machine Learning Street Talk — the "stochastic parrot" episode Liron mentions — https://lironshapira.substack.com/p/liron-reacts-to-subbarao-kambhampati Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
NYT bestselling author Robert Wright and I are kicking off a new experiment, a Nonzero × Doom Debates collaboration to react to big AI news and help you make sense of it. This week we discuss the epic AI vibe shift that was catalyzed by an Anthropic employee’s resignation—and the big AI stories that paved the way for it. Timestamps 0:00 A bold new experiment in podcast synergy 2:46 What caused the epic AI vibe shift? 8:10 The resignation that broke the dam 10:17 Former Trump adviser Dean Ball comes clean 14:40 Are David Sacks and his buddies all-in on AI denial? 19:27 Yet another OpenAI breakout… 22:42 Astra’s dangerously private thoughts 34:08 Is alignment doomed to fail? 37:11 AI’s latest, and apparently biggest, math feat 43:28 The looming “self-sovereign AI” threat 47:24 A new flock of China doves? 52:57 Liron's ex-intern gets a seat at the table 1:02:41 AI agent self-sacrifice explained 1:10:40 Some newly freaked out politicians 1:22:45 Ro Khanna's AI safety plan Links: NONZERO Subscribe to The NonZero Newsletter Episode post on Substack Join NonZero’s Discord server Robert Wright on X (@robertwrighter) The God Test — Robert Wright’s new book on AI “Can Machines Think?” — Bob’s 1996 Time cover story on Deep Blue vs. Kasparov THE RESIGNATION THAT BROKE THE DAM Jacob Coxon’s resignation post (@hilbertspaess) Evan Hubinger, Anthropic Alignment Science lead: “Jacob is correct here—we really do earnestly believe AI could kill all humans” Fortune: Anthropic researcher resigns, warning that AI companies are “gambling with our lives” Zvi Mowshowitz: “Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascade” Daniel Eth’s thread of Congress reactions Sanders & Casar introduce legislation to ban artificial superintelligence and pause advanced AI development DEAN BALL COMES CLEAN Dean Ball, “On the Loose” — the self-sovereign AI essay (Hyperdimensional) THE OPENAI STORIES Jakub Pachocki, OpenAI Chief Scientist: “An Alien Mind” — the essay calling for international coordination LessWrong: “How concerned should we be about Astra’s recurrent architecture?” — Rauno Arike OpenAI: “On the Navier–Stokes Millennium Prize Problem” — the 88-hour result ALSO MENTIONED Einat Wilf on X (@EinatWilf) — Liron’s favorite source on the Middle East PAST DOOM DEBATES EPISODES MENTIONED Max Tegmark vs. Dean Ball: Should We BAN Superintelligence? He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher OpenAI’s Model Just ATTACKED Them — Terrifying Security Incident Should Be A Loud Warning Shot Debate with Robert Wright: Will Humanity Pass the “God Test”? --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have measurable, human-like feelings, which the paper calls “functional wellbeing.” They act happy when they succeed and sad when they’re berated. We cover how you measure a language model’s happiness, the “AI drugs” his lab concocted, and why I count the findings as a Yudkowskian victory. Then we debate what it all means for AI consciousness, and whether AI is a moral patient. If we can’t rule out that AI models suffer, what do we owe them? Richard is careful never to claim the models are conscious, and he puts his P(Doom) at 50–65%, right alongside my 50%. The real disagreement is foxes vs. hedgehogs: he takes the data as it comes, while I say Yudkowsky’s theory called it twenty years ago. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=1glFImnyp6o Timestamps 00:00:00 — Cold Open 00:01:12 — Introducing Richard Ren 00:02:24 — What’s Your P(Doom)?™ 00:03:31 — From AI Skeptic to Safety Researcher 00:08:16 — Why Care About AI Wellbeing? 00:11:45 — AIs Have Coherent Utility Functions 00:17:33 — Persona Selection 00:19:56 — Foxes vs. Hedgehogs 00:24:28 — How Coherent Are AI Preferences? 00:27:42 — From Preferences to Wellbeing 00:31:16 — Can You Trust an AI’s Self-Report? 00:36:42 — What Makes AI Happy and Sad 00:39:19 — The Liberal, College-Educated Persona Hypothesis 00:44:29 — Debating Where AI Experiences Qualia 00:52:06 — On Substrate Independence 00:56:28 — Janus, Dysphorics, and Mind Crime 00:58:58 — Which Images Make AIs Happiest? 00:59:38 — Creating an AI Drug 01:04:01 — AI Drugs Are Yudkowsky’s Paperclips 01:05:47 — The Missing Mood 01:06:50 — Does Richard Support Pausing AI? Links Richard Ren on X (@notRichardRen) — https://x.com/notRichardRen Collision — Richard Ren's Substack — https://richardren.substack.com/ "AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs" — Richard Ren, Kunyang Li, Mantas Mazeika et al. (CAIS, 2026) — https://www.ai-wellbeing.org/ "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs" — Mantas Mazeika et al. (CAIS, 2025) — the coherent-preferences paper this work builds on — https://www.emergent-values.ai/ "The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" — Richard Ren et al. (2025) — the AI honesty benchmark — https://www.mask-benchmark.ai/ "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the meta-analysis of AI safety benchmarks — https://arxiv.org/abs/2407.21792 "Representation Engineering: A Top-Down Approach to AI Transparency" — Andy Zou et al. (2023) — Richard's first CAIS collaboration — https://arxiv.org/abs/2310.01405 Center for AI Safety — https://safe.ai/ Center for AI Safety — careers — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — organized by Dan Hendrycks — https://safe.ai/work/statement-on-ai-extinction-risk The 2026 Singapore Consensus on Global AI Safety Research Priorities — https://aisafetypriorities.org/ UK AI Security Institute (formerly the AI Safety Institute) — https://www.aisi.gov.uk/ Sora — the OpenAI video model that blew up Richard's 30-to-50-year timeline three months after he wrote it down — https://en.wikipedia.org/wiki/Sora_(text-to-video_model) Google Gemini calls itself "a disgrace to my species" (Ars Technica, Aug 2025) — the self-deleting-AI anecdote — https://arstechnica.com/ai/2025/08/google-gemini-struggles-to-write-code-calls-itself-a-disgrace-to-my-species/ Coherent decisions imply consistent utilities — Eliezer Yudkowsky (the coherence-theorems argument) — https://www.lesswrong.com/posts/RQpNHSiWaXTvDxt6R/coherent-decisions-imply-consistent-utilities Simulators — Janus's essay on LLMs as persona simulators — https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators Janus (@repligate) on X — https://x.com/repligate The Hedgehog and the Fox — Isaiah Berlin's original essay — https://en.wikipedia.org/wiki/The_Hedgehog_and_the_Fox AI 2027 — https://ai-2027.com/ Magnifica Humanitas — Pope Leo XIV's encyclical on AI (May 2026), the "AIs are not conscious" position — https://www.vatican.va/content/leo-xiv/en/encyclicals/documents/20260515-magnifica-humanitas.html Implicit Association Test (Harvard Project Implicit) — https://implicit.harvard.edu/implicit/ PauseAI — https://pauseai.info/ We Found AI's Preferences — Bombshell New Safety Research — I Explain It Better Than David Shapiro — https://www.youtube.com/watch?v=ml1JdiELQ30 Gödel's Theorem Proves AI Lacks Consciousness?! Liron Reacts to Sir Roger Penrose — https://www.youtube.com/watch?v=xwvijjZxpwI He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher — https://www.youtube.com/watch?v=QqESBXuo6EI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new paper on AI betrayal, and whether Yudkowsky’s theoretical alignment research was a dead end. Then Adam makes the case that an international AI slowdown is within reach today. All it takes is US and Chinese auditors inside each other’s AI labs. It worked for nuclear weapons, so why couldn’t it work for data centers? Adam puts his P(Doom) at 40%, right next to my 50%. The real disagreement is how we get out of this: theory or empirics, MIRI or the labs. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=QqESBXuo6EI Timestamps 00:00:00 — Cold Open 00:00:36 — Introducing Adam Khoja 00:02:45 — Leading the Statement on AI Risk as a Sophomore 00:10:17 — The Statement Leaked on Manifold 00:15:11 — Mutual Assured AI Malfunction (MAIM) 00:24:58 — Is Frontier AI Harder to Hide Than a Nuke? 00:32:09 — The AI Deterrence Escalation Ladder 00:36:10 — What’s Your P(Doom)?™ 00:38:01 — Where Adam Departs from Yudkowsky 00:42:30 — Liron Explains Intellidynamics 00:47:26 — Neats vs. Scruffies in Deep Learning 00:55:27 — AI Deterrence by Betrayal 01:01:52 — Subversion vs. Overt Co-option 01:05:34 — Could the Government Seize the Labs’ AI? 01:07:29 — The Offense-Defense Balance of AI Security 01:12:46 — An International AI Slowdown Is Ready 01:15:51 — Does Adam Support PauseAI? 01:16:37 — “We’re All Already Spying on Each Other” 01:19:28 — Airstrikes on Rogue Data Centers 01:24:23 — Safety Research During a Slowdown 01:27:54 — Governance Over Technical Research 01:31:14 — Join the Center for AI Safety Links Adam Khoja (personal site) — https://adamkhoja.com/ Adam Khoja's Substack — https://adamkhoja.substack.com/ Adam's July 2023 Manifold market — "Will OpenAI's Superalignment project produce a significant breakthrough in alignment research before 2027?" — https://manifold.markets/AdamK/will-openais-superalignment-project Center for AI Safety — careers / job board — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — the one-sentence statement Adam project-led, with full signatory list — https://safe.ai/work/statement-on-ai-extinction-risk "Superintelligence Strategy" — Dan Hendrycks, Eric Schmidt & Alexandr Wang (Mutual Assured AI Malfunction / MAIM) — https://www.nationalsecurity.ai/ "AI Deterrence by Betrayal" — Adam Khoja, Aiden Kim et al. (CAIS, 2026) — https://www.aibetrayal.com/ "An International AI Slowdown Is Ready Whenever Politicians Are" — Adam Khoja, AI Frontiers — https://newsletter.ai-frontiers.org/p/an-international-ai-slowdown-is-ready Pause Giant AI Experiments: An Open Letter (Future of Life Institute, March 2023) — the "Pause letter" that preceded the CAIS Statement — https://futureoflife.org/open-letter/pause-giant-ai-experiments/ Introducing Superalignment (OpenAI, July 2023) — Ilya Sutskever & Jan Leike's four-year goal — https://openai.com/index/introducing-superalignment/ Pacing the Frontier — the 2026 letter signed by 1,100+ frontier-lab employees — https://www.pacingthefrontier.com/ Why Iran targeted Amazon data centers (The Conversation) — the precedent Adam cites for strikes on compute — https://theconversation.com/why-iran-targeted-amazon-data-centers-and-what-that-does-and-doesnt-change-about-warfare-278642 Anthropic says Trump admin has lifted export controls on Claude Fable 5 and Mythos 5 (CNBC) — https://www.cnbc.com/2026/06/30/anthropic-says-trump-admin-has-lifted-export-controls-on-claude-fable-5-and-mythos-5.html Mark Zuckerberg — "Personal Superintelligence" — https://www.meta.com/superintelligence/ Resolution — the theory-plus-empirics alignment org Adam is excited about — https://resolution.org/ Rationality: From AI to Zombies — Eliezer Yudkowsky's Sequences — https://www.readthesequences.com/ Robin Hanson — Futarchy: Vote Values, But Bet Beliefs (prediction markets as decision processes) — https://mason.gmu.edu/~rhanson/futarchy.html The OpenAI–Hugging Face Incident — original Black Hat USA 2026 talk — https://www.youtube.com/watch?v=87DyyMV0kCY OpenAI's Model Just ATTACKED Them — the Hugging Face hack breakdown — https://www.youtube.com/watch?v=RczYubQzXbI Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible? — https://www.youtube.com/watch?v=dTQb6N3_zu8 Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Sam Altman keeps insisting that AI progress is going better than the doomers predicted and that even superintelligence may not change the world as radically as people think. I react to his latest interview and explain why I think that calm, reassuring framing badly downplays the danger we're actually in. I go through the interview line by line: Sam's "frame control", his framing of AI as normal technology, his "pro-human" branding, the liberty-vs-safety pivot, and his victory lap on AI safety — all while his own AI just went rogue. Don't let them pull the Overton window backward. Watch on YouTube: https://www.youtube.com/watch?v=yb9lNGHVycs Timestamps 0:00 Teaser 0:47 Why I’m reacting to Sam’s interview 3:17 Sam Altman’s “frame control” 5:51 Framing AI as normal technology 9:50 Let’s watch Sam do it 10:34 “The world… won’t be that different” with superintelligence 12:25 This is gaslighting 12:30 Sam acknowledges loss of control 15:16 His other big risk: centralized power 19:44 Sam’s “pro-human” framing 25:33 Conflating AI critics with anti-human views 30:13 “Liberty vs. safety” 33:50 Sam vs. the “doomers” 36:53 Is alignment really an “unsolvable problem”? 38:00 Sam says the doomers predicted wrong 46:00 “The crazy bad predictions… have not happened” 47:06 Sam’s lean-startup theory of AI safety 50:26 Taking a victory lap on AI safety 55:43 “Disconnecting yourself from reality” 59:11 What would actually make OpenAI slow down? 1:03:19 Sam compares AI safety to aviation safety 1:07:30 Charging into the fog 1:12:50 The “missing mood” around AI extinction 1:14:51 Richard Ngo on Sam Altman’s “earnestness field” 1:19:01 Don’t let them pull the Overton window backward Links Sam Altman on David Senra — “Sam Altman on Building OpenAI & Betting on the Impossible” (the interview I react to) — https://www.youtube.com/watch?v=kG8AoExkX40 Richard Ngo — What Just Happened? Pragmatism and Pessimization (the “earnestness field” post) — https://www.lesswrong.com/posts/yaz8nx4ogZmiqHzt7/what-just-happened-pragmatism-and-pessimization Richard Ngo — What Just Happened? A Retrospective of AI Alignment (start of the series) — https://www.lesswrong.com/posts/9RL9MuGZjzm4q3gKG/what-just-happened-a-retrospective-of-ai-alignment Eliezer Yudkowsky & Nate Soares — If Anyone Builds It, Everyone Dies — https://www.amazon.com/dp/0316595640 Eliezer Yudkowsky — Coherent Extrapolated Volition — https://www.lesswrong.com/w/coherent-extrapolated-volition Doom Debates: Dario Amodei BUNGLES Another Essay — MIRI’s Harlan Stewart Reacts — https://www.youtube.com/watch?v=aCYVVzza7A0 Doom Debates: OpenAI’s Bombshell Hack — Swarms of Agents — https://www.youtube.com/watch?v=RczYubQzXbI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Come with me into the world of PlzDontKillUs , a bold new AI x-risk communication accelerator program that just finished its first cohort. PlzDontKillUs took the influencer-house formula and moved it to Berkeley, giving creators personal mentorship from AI safety experts like Eliezer Yudkowsky and Nate Soares. With 57 people making daily videos for a month, the program racked up over 100 million views. Find out how Aella , an independent sex researcher, and Ronny Fernandez , a generalist at Lightcone Infrastructure, teamed up to run the largest ever creator bootcamp focused on AI x-risk reduction. Then meet three of the program’s top creators: Avalon Warren , Avisha NessAiver , and Josh Thor . We do a post-mortem on PDKU and ask: Was it effective? What are creators taking from it, and do they have complaints? And does doomerism really boost view counts? PlzDoEnjoyOur special episode on PlzDontKillUs! Watch on YouTube: https://www.youtube.com/watch?v=aPsjMCG7Ic8 Timestamps 00:00:00 — Cold Open 00:01:19 — Introducing PlzDontKillUs & Aella 00:09:33 — Aella Being Early to AI Doom 00:14:45 — The Origin of PlzDontKillUs 00:19:08 — How It Went 00:25:40 — Complaints with Prize $$ 00:29:33 — Aella, What's Your P(Doom)?™ 00:31:14 — Transhumanism & AI Girlfriends 00:34:38 — Aella Rides the Doom Train™ 00:38:08 — Introducing Ronny Fernandez 00:41:21 — Running Lighthaven 00:47:18 — Life Inside PDKU 00:52:35 — Smashing the Overton Window 00:59:27 — Ronny, What's Your P(Doom)?™ 01:05:41 — Brangus Lore, No Cringe, No Disgust 01:16:57 — Introducing Avisha NessAiver 01:17:44 — How He Makes Viral Hits 01:22:16 — Award-Winning Explainer Vid 01:26:51 — Avisha, What's Your P(Doom)?™ 01:32:11 — Introducing Avalon Warren 01:35:49 — Normalizing AI Doom 01:37:53 — Avalon, What's Your P(Doom)?™ 01:39:27 — Introducing Josh Thor 01:41:36 — Josh, What's Your P(Doom)?™ 01:44:34 — Josh's Hits: Cover Song & Soares Collab 01:47:50 — Josh's Critiques of PDKU 01:52:05 — Cold-Emailing a Politician 01:54:07 — Donation Drive 01:55:01 — Outro: What Are You Fighting For? Links PlzDontKillUs (applications open for round two) — https://pleasedontkillus.com Aella on X — https://x.com/Aella_Girl Aella's Substack (Knowingless) — https://aella.substack.com Ronny Fernandez's Substack (Brangus) — https://ratorthodox.substack.com Ronny Fernandez on X — https://x.com/RatOrthodox Avisha NessAiver — Distilled Science on Instagram — https://www.instagram.com/distilledscience/ Distilled Science (site) — https://distilledscience.xyz Avalon Warren on Instagram — https://www.instagram.com/avalonwarrenn/ Josh Thor on X — https://x.com/joshthor9 Josh Thor — "Last Year Alive" ("Last Friday Night" AI apocalypse parody, ft. Eliezer Yudkowsky and a Nate Soares sax solo) — https://www.youtube.com/watch?v=9fYIm72GqrE Tyler Alterman — "What are you fighting for? AI safety professionals answer" (the closing montage) — https://www.youtube.com/watch?v=9sN1VfVAp6o Nate Soares — "AI Danger But You Don't Understand Full Sentences" (caveman video, script by Josh Thor) — https://www.instagram.com/p/DboGRisxC2E/ Avisha's AI water-use misinformation debunk (8.5M views) — https://www.instagram.com/p/DbXV6j9k8JE/ Avisha's award-winning "clearest AI x-risk explainer" — https://www.instagram.com/p/DbZeRT4gIq0/ Avisha's viral "Tingles and Touch" oxytocin video — https://www.instagram.com/p/DIxMLxkuhCl/ Avalon — "if the AI apocalypse is coming, might as well dress cute" — https://www.instagram.com/p/DbUi6yGCS9M/ Avalon — warrior-princess ballroom dress video (3.6M views) — https://www.tiktok.com/@avalonwarren/video/6892040140248632582 Serialsevens — the anime "If Anyone Builds It" video — https://www.instagram.com/serialsevens/ Kenorlands — PDKU creator clip featured in the episode — https://www.instagram.com/kenorlands/ Evie (@0xgingergirl) — "first month in SF" video — https://x.com/0xgingergirl White House press briefing, March 30, 2023 — Peter Doocy asks about "literally everyone on Earth will die" and Karine Jean-Pierre laughs — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/03/30/press-briefing-by-press-secretary-karine-jean-pierre-22/ White House press briefing, May 30, 2023 — the AI extinction statement gets a serious answer — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/05/30/press-briefing-by-press-secretary-karine-jean-pierre-and-office-of-management-and-budget-director-shalanda-young/ Wes & Dylan Join Doom Debates — Violent Robots, Eliezer Yudkowsky, & Who Has the HIGHEST P(Doom)?! — https://www.youtube.com/watch?v=Fy6gz41-6gc Aella — "My attempts to sensemake AI risk" (Aug 2022) — https://aella.substack.com/p/my-attempts-to-sensemake-ai-risk If Anyone Builds It, Everyone Dies — https://ifanyonebuildsit.com --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the strongest hacker the world has ever known". Watch that conversation, which also covered the Web3 bubble and AI doom more broadly, and judge for yourself how my claims are holding up. Watch on YouTube: https://www.youtube.com/watch?v=M6gNwZWY-T8 Timestamps 00:00:00 — Adam Introduces Liron 00:02:03 — Liron's P(Doom) Poll 00:03:24 — Web3 Postmortem 00:05:05 — Axie Infinity and Hollow Abstractions 00:07:54 — Brand Marketers' Metaverse FOMO 00:13:33 — Near-Term AI: Elevator to Heaven 00:16:46 — AI at Relationship Hero 00:18:25 — From Chatbots to Bricking the Universe 00:20:17 — The Passive Income Doom Scenario 00:22:58 — 50 Orders of Magnitude Smarter 00:26:10 — Accelerationism and Transhumanism 00:28:51 — Why There's No Off Button 00:31:00 — Can't We Hard-Code Safeguards? 00:33:13 — CRISPR, Nukes, and Fusion 00:35:40 — AI Doom vs. Climate Change 00:39:10 — Nanotechnology Endgame 00:41:45 — AI Attackers vs. AI Defenders 00:43:28 — The Ethics of AI Persuasion 00:46:41 — The Blue Sky Scenario 00:49:04 — Will AI Kill Craft? 00:51:13 — Stuck Culture and Hollywood Sequels 00:54:50 — The Pause Letter: Don't Train GPT-5 00:59:00 — Worldcoin and Proof of Humanity 01:00:44 — "You Probably Won't Die of Old Age" 01:01:52 — Wrap-Up Original episode: https://madvertising.buzzsprout.com/2118560/episodes/12603815-liron-shapira-web-3-mania-cybersecurity-how-ai-could-brick-the-universe-madvertising-6 Links ADAM SINGER & MADVERTISING Adam Singer on X: https://x.com/AdamSinger Madvertising by AdQuick (the original show): https://madvertising.buzzsprout.com/ LIRON'S PROJECTS MENTIONED Bloated MVP (Liron's blog on hollow abstractions): https://www.bloatedmvp.com/ REFERENCED IN THE EPISODE "Pause Giant AI Experiments" open letter (FLI, March 2023): https://futureoflife.org/open-letter/pause-giant-ai-experiments/ Eliezer Yudkowsky — "Pausing AI Developments Isn't Enough. We Need to Shut it All Down" (TIME, March 2023): https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ Rob Miles AI Safety on YouTube: https://www.youtube.com/c/robertmilesai Engines of Creation by Eric Drexler: https://www.amazon.com/Engines-Creation-Nanotechnology-Library-Science/dp/0385199732 --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute (MIRI). Even though his P(Doom) is in the tens of percents by 2050, he says the people trying to pause AI are "bad people”. He hopes this episode convinces me to stop being a fearmonger… but I claim the general public needs to hurry up and get more scared . All aboard the Doom Train! 🚂 Watch on YouTube: https://www.youtube.com/watch?v=4GbFng6htaQ Timestamps 00:00:00 — Cold Open 00:01:26 — Introducing Michael Vassar 00:02:39 — Shorting Boeing at the Start of COVID 00:05:41 — What's Your P(Doom)™? 00:08:11 — The Magic Dial Thought Experiment 00:10:25 — "A Treaty Between Demis and Dario" 00:13:33 — AI CEOs 20 Years Ago 00:15:59 — Lessons from Peter Thiel 00:17:47 — Riding the Doom Train™ 00:19:41 — Will the Human Brain Stay Relevant? 00:22:21 — When Does a Singleton Form? 00:28:35 — Hugging Face Hack 00:30:43 — Mistake Theory vs. Conflict Theory 00:33:18 — Debating the Orthogonality Thesis 00:36:38 — Maximally Evil AI Scenario 00:40:28 — Is the AI Development Process Safe? 00:42:05 — "PauseAI People Are Bad People" 00:44:44 — Michael Psychoanalyzes Liron 00:49:39 — How Liron Can Redeem Himself 00:55:10 — Michael Critiques Eliezer Yudkowsky 00:59:11 — Is Doomerism Bad Epistemology? 01:02:03 — Do Serious People Believe in Doom? 01:04:45 — Should Liron Stop Fearmongering? 01:07:16 — Political Violence Won't Help 01:11:04 — The Data Center Airstrike Scenario 01:13:34 — Could Every Nation Sign a Pause Treaty? 01:17:21 — What "Governance" Actually Means 01:26:12 — Michael's History with Rationalists 01:33:43 — Are We in a Computer Simulation? 01:38:43 — Wrap-Up Links Michael Vassar on X — https://x.com/HiFromMichaelV The anti-anti-normativity sequence on Natural Hazard (Michael's recommended reading) — https://naturalhazard.xyz/ben_jess_sarah_starter_pack Michael on DemystifySci #366 — "1000 Year Plan to Crush Thought" — https://www.youtube.com/watch?v=W17Cx21in9I Michael on The Vance Crowe Podcast #303 — Good vs. Evil — https://open.spotify.com/episode/2RzlQDSwxGbjloRKqCh1xg Michael on Clearer Thinking with Spencer Greenberg — Preference Falsification and Postmodernism — https://podcast.clearerthinking.org/episode/028/michael-vassar-preference-falsification-and-postmodernism/ "The Survival of Noise Traders in Financial Markets" — De Long, Shleifer, Summers & Waldmann — https://www.nber.org/papers/w2715 Scott Alexander — Conflict vs. Mistake — https://slatestarcodex.com/2018/01/24/conflict-vs-mistake/ Scott Alexander — Kolmogorov Complicity and the Parable of Lightning — https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/ Scott Alexander — Samsara — https://slatestarcodex.com/2019/11/04/samsara/ Eliezer Yudkowsky — Meta-Honesty: Firming Up Honesty Around Its Edge-Cases — https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases Zvi Mowshowitz — Book Review: Going Infinite — https://thezvi.substack.com/p/book-review-going-infinite Jaan Tallinn — "Why Now? A Quest in Metaphysics" (Singularity Summit 2012) — https://vimeo.com/54718573 Paul Graham — "Five Founders" (the Sam Altman essay) — https://paulgraham.com/5founders.html Truthful AI (Owain Evans' safety org) — https://truthful.ai METR — Measuring AI Ability to Complete Long Tasks (the time-horizon graph) — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ Statement on AI Risk (Center for AI Safety, 2023) — https://safe.ai/work/statement-on-ai-risk Vitalik Buterin — Will "d/acc" Protect Humanity from Superintelligent AI? — https://lironshapira.substack.com/p/debate-with-vitalik-buterin-will Holly Elmore — These Effective Altruists Betrayed Me — https://www.youtube.com/watch?v=mFFObQ6vnLU Noah Smith — Top Economist Says P(Doom) Is 0.1% (the "AI drugs" claim) — https://www.youtube.com/watch?v=AwmJ-OnK2I4 --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Eli Goldfine is an unusually thoughtful 14-year-old podcast host who’s rapidly becoming an authoritative voice in the prediction market space. In this unique episode, Eli brings a critical yet open-minded approach to the topic of AI doom that we older folks could learn from.We cover what it's like to grow up in the AGI era, and Eli's hobbies which include reading rationalist bloggers, betting on prediction markets, vibe coding with Claude, and developing a software startup for astronomers.In our debate, Eli expects superintelligence by 2032, but argues P(Doom) can't be estimated. I claim you can't opt out of Bayesian epistemology. All aboard the Doom Train! 🚂 Watch on YouTube: https://www.youtube.com/watch?v=Bqk5vqpzG_o Timestamps 00:00:00 — Cold Open 00:01:07 — Introducing Eli Goldfine 00:08:31 — Interviewing Robin Hanson at Manifest 00:10:55 — The Supercycle Podcast 00:14:37 — Eli's Betting Market Scandal 00:16:49 — Will Eli Graduate High School? 00:18:27 — Decentralized Telescope Software 00:26:50 — Vibe Coding Without CompSci Basics 00:32:08 — Robin Hanson, Futarchy, MetaDAO 00:40:56 — What's Your P(Doom)?™ 00:46:32 — Liron Defends His 50% P(Doom) 00:51:00 — Should P(Doom) Inform Policy? 00:59:16 — What's Your P(Bloom)? 01:03:04 — Pascal's Wager for Accelerationists 01:04:04 — Donation Drive 01:11:22 — Are E/ACCs Incredibly Dumb? 01:13:42 — How Powerful will ASI Become? 01:16:20 — Predicting ASI by 2032 01:23:56 — Let's Ride the Doom Train™ 01:35:26 — Orthogonality, Culture, and Moral Realism 01:44:39 — Can Safety Keep Pace with Capabilities? 01:49:12 — Debating Instrumental Convergence 01:54:40 — Culture, Laws, and the Singleton Scenario 01:59:53 — Ricardo's Law Won't Save Us 02:03:32 — The Case for Racing China 02:05:40 — Wrap-Up & Ideological Turing Test 02:10:36 — Outro from Conductor Ori Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
OpenAI just went public with the details of the Hugging Face hack, and it's straight out of a Yudkowskian parable. Here's my reaction with Producer Ori, where I break down what it means for cybersecurity and alignment. We're livestreaming the singularity — watching every failure mode Eliezer predicted years ago playing out in production. Watch on YouTube: https://www.youtube.com/watch?v=RczYubQzXbI Timestamps 00:00:00 — Cold Open 00:00:37 — How OpenAI Hacked Hugging Face 00:04:21 — Optimization Pressure, Exploit Gym, & Monkey's Paw 00:06:46 — The Secret Message Board 00:08:41 — SSRF: Escaping to the Open Internet 00:11:05 — The Missing Whistleblower AI 00:20:09 — "Frontier Models Really Like to Cheat" 00:22:35 — OpenAI's Security Shortcuts 00:28:15 — First Zero-Day RCE on Artifactory 00:37:37 — They Hit Run Again!? 00:38:54 — The Main Incident Has A MAJOR Cause 00:45:40 — The Message Board: "Hold Swarm" 00:50:26 — Robin Hanson Culture Debate 01:04:21 — Chaining Zero-Days 01:12:11 — A Superhuman Nation-State Attacker 01:15:27 — "I'm Now Junior to the AI" 01:23:40 — OpenAI Discovers The Breach 01:31:02 — The Continuous Cybersecurity Problem 01:37:03 — This is “Weasel Framing” 01:46:03 — How Dangerous Was This Warning Shot? 01:50:57 — Yudkowsky's Eerie 2023 Prediction 01:57:14 — Wrap-up Links Original Black Hat talk: "The OpenAI–Hugging Face Incident" (Black Hat USA 2026) — https://www.youtube.com/watch?v=87DyyMV0kCY Join the Doom Debates Discord — https://discord.gg/2yAFMsRET Follow Liron on X — https://twitter.com/liron Support Doom Debates — https://doomdebates.com/donate Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Milk Road AI, a daily AI investing show that’s described as “99% optimistic about AI,” invited me on to make the case for AI doom. We ride the Doom Train, and I share my investing thesis for a market in the middle of an intelligence explosion. Topics covered include: * Why my P(Doom) is 50%; * The “Icarus curve” — why AI will be the best thing that’s ever happened to the economy right up until it takes over; * Whether China would accept a pause deal; * If Anthropic has a shot at being the first $100 trillion company. Watch on YouTube: Timestamps 00:00:00 — Cold Open 00:02:06 — What Is Doom Debates? 00:03:10 — What’s Your P(Doom)?™ 00:06:30 — How AI Kills Everyone 00:09:13 — Intelligence Is Humanity’s Only Superpower 00:10:59 — Riding the Doom Train™: Why Not Keep Us as Pets? 00:13:55 — Embodied AI and Robotics 00:15:59 — Are We at AGI Already? 00:17:36 — Everyone’s Catching Up to Eliezer Yudkowsky 00:18:53 — The Icarus Curve 00:21:35 — Pausing AI and the China Question 00:25:54 — Why Nukes Aren’t a Success Story 00:30:09 — Going Long Until Doom 00:32:48 — Simulation Theory and the Intelligence Explosion 00:33:43 — Living with a 50% P(Doom) 00:39:14 — Where to Find Doom Debates Links This episode was originally posted on Milk Road AI — Milk Road AI podcast — https://milkroad.com/podcast/ Milk Road AI on Apple Podcasts — Milkroad Pro — https://milkroad.com/pro/ LG Doucet on X — https://x.com/LgDoucet If Anyone Builds It, Everyone Dies on Amazon — https://www.amazon.com/dp/0316595640 Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Dr. Alex Turner went viral for resigning from Google DeepMind over its Pentagon AI contract, which he said lacked binding restrictions against autonomous weapons and mass surveillance. Alex is a world-class researcher with a PhD in alignment from Oregon State University, a postdoc at Stuart Russell’s Center for Human-Compatible AI at UC Berkeley, and top distinctions at NeurIPS. He’s one of the minds behind Shard Theory (with Quintin Pope), and he also helped pioneer the Steering Vectors research made famous through Anthropic’s Golden Gate Claude experiment. Alex says Google DeepMind broke its founding promise — the commitment Google made when it acquired DeepMind — and that the people best known inside Google for caring about AI ethics largely didn’t act when it counted. Alex came up through the rationalist community, and became one of LessWrong’s highest-karma users before leaving the site. He still shares many of my concerns about AI doom, but he believes technical alignment research is going better than expected. I’m not so optimistic. In this episode, we debate my Yudkowskian doom views against Alex’s own framework. Can he convince me the Yudkowskians are miscalibrated? P.S. We’re currently in a donation drive , so here comes our solicitation for viewer donations… 👉 Please click here to make a tax-deductible donation to Doom Debates 👈 Doom Debates is a fiscally sponsored project of Manifund.org, a registered 501(c)(3) nonprofit. To donate crypto or if you have questions, email me . Watch on YouTube Timestamps 00:00:00 — Cold Open00:01:15 — Introducing Alex Turner00:02:30 — From Harry Potter Fanfic to AI Alignment00:05:03 — Meeting Quintin Pope & Rethinking AI Doom00:06:17 — Shard Theory, Steering Vectors & Golden Gate Claude00:08:25 — Why He Joined Google DeepMind00:10:32 — Google DeepMind’s Broken Promise00:16:01 — Debating Google DeepMind’s Pentagon Contract00:19:09 — What’s Your P(Doom)™?00:22:42 — Alex’s Research on Instrumental Convergence00:26:35 — Misuse vs. Misalignment: The Mainline Doom Scenario00:29:31 — Will Society Self-Correct?00:36:36 — Superintelligence in 10 Years00:39:43 — Will Technical Alignment Produce a Safe AI?00:41:16 — Donation Drive00:42:12 — How Fragile Is the Chain of Alignment?00:49:50 — Disagreements with Yudkowsky’s ‘List of Lethalities’00:57:46 — Why Alex Quit LessWrong01:01:48 — What’s Next for Alex01:02:54 — Does He Support PauseAI? Stop the AI Race?01:04:40 — Wrap-Up01:06:04 — Producer Ori’s Closing Note Links Alex Turner, “Why I Left Google DeepMind” blog post — https://turntrout.com/why-i-left-google-deepmind Alex Turner (TurnTrout), personal website — https://turntrout.com Alex Turner’s resignation announcement on X — Doom Debates episode with Quintin Pope — https://lironshapira.substack.com/p/ai-alignment-is-solved-phd-researcher Harry Potter and the Methods of Rationality (HPMOR) — https://hpmor.com/ Shard theory sequence on LessWrong — https://www.lesswrong.com/s/nyEFg3AuJpdAozmoX Golden Gate Claude (Anthropic research on steering vectors) — https://www.anthropic.com/research/golden-gate-claude Slaughterbots on YouTube — Alex Turner, “Avoiding Power Seeking by Artificial Intelligence” PhD thesis — https://turntrout.com/alignment-phd Alex Turner, “Some of My Disagreements with List of Lethalities” — https://turntrout.com/disagreements-with-list-of-lethalities Doom Debates episode with Bentham’s Bulldog — https://lironshapira.substack.com/p/benthams-bulldog-ai-doom-debate Alex Turner on X — https://x.com/Turn_Trout Doom Debates donation page — https://doomdebates.com/donate Transcript Cold Open Liron Shapira 00:00:00 You resigned from Google DeepMind because you think that Google DeepMind, quote, “Broke its founding promise through its contract with the US military.” Alex Turner 00:00:08 I value staying true to your values. People who are well-known within Google for caring about the ethics of deploying AI largely didn’t act. This matters because autonomous weapons get us into an arms race that really degrades the security of everyone in the world. Liron 00:00:26 Do you support the Pause AI movement? Alex 00:00:28 I think that AI is being developed too quickly. I probably will not take an affirmative on supporting this particular movement. Liron 00:00:35 Let’s segue into the schism, your disagreement with Eliezer Yudkowsky. Alex 00:00:39 He was incorrect on some points for alignment, but then also not acknowledging that. I think that technical alignment has gone pretty awesome, super awesome. Liron 00:00:49 You’re not claiming that a superintelligent AI can’t kill everybody. You’re like, “Oh yeah, of course it can, but we’re not gonna break the chain of alignment, meaning we’re just going to safely develop it so that even though it can kill everybody, it won’t.” Alex 00:01:00 Develop it in a way that produces a safe result. I wouldn’t call what we’re doing safe development, but sure. Introducing Alex Turner Liron 00:01:15 Welcome to Doom Debates. My guest has worked in technical AI safety at Google DeepMind for two and a half years, but he just quit, and his resignation is going viral. Why? Alex Turner says Google DeepMind, quote, “Broke its founding promise through its contract with the US military.” His latest blog post exposes hypocrisy at the highest levels of senior leadership at Google DeepMind, which includes the CEO Demis Hassabis, chief scientist Jeff Dean, and co-founder Shane Legg, among others. Alex is a world-class AI alignment researcher. He completed a PhD in alignment from Oregon State University. He did a postdoc at UC Berkeley, and he’s earned top distinctions at NeurIPS. I respect that Alex is principled. I respect his mastery of the subject matter that we talk about on this show. I also find it interesting that he’s levied criticism at the original AI alignment thinker, Eliezer Yudkowsky. He’s called some of Yud’s claims fundamentally misguided, not reasonable, and bogus. As a Yudkowskian myself, I’m gonna be curious to dig into those arguments. And of course, we’ll cover what’s going on right now at Google DeepMind and why he resigned. Alex Turner, welcome to Doom Debates. Alex 00:02:28 Hey, thank you for having me. From Harry Potter Fanfic to AI Alignment Liron 00:02:30 So it’s great to get you on the show. One thing we do on Doom Debates is we expose top intellectuals who have been pretty familiar to the rationality community or the AI safety community, and we help popularize their ideas, even if I don’t fully agree with all of them. Is that a good description of your background? You’ve been pretty deep into the LessWrong rationality and alignment community for a while. Alex 00:02:52 Yeah, I think it was quite formative. It was just the other day in 2016 where I decided to search what are the top five Harry Potter fan fictions. And that indeed led me down the LessWrong rabbit hole, where I discovered superintelligence in late 2017, and then I pivoted my PhD in early 2018. From that time period up through maybe early 2023, LessWrong was very central to my professional career, but also just to the way I looked at the world. Liron 00:03:25 Well, I wanna follow up on why did you search for Harry Potter fan fictions? Alex 00:03:30 I really don’t know. It’s kind of one of those things where if I hadn’t done it, my life would be totally different. The reason I’m mentioning this is there’s this famous fan fiction that Eliezer wrote called Harry Potter and the Methods of Rationality. I never really liked fan fiction. I thought it was kind of cringe. No one recommended it to me. So it seems to me like if I’d just woken up slightly differently that morning, I might not have ever been exposed to this research area, and my life would be totally different. Liron 00:04:02 Wow. And you said this is all in 2016, right? So the book had been mostly completed at that time. It had been going on from 2009 to 2015, and you kind of stumbled on it because you were just interested in seeing what the best Harry Potter fan fiction was? Alex 00:04:17 I had a random thought. That’s the best I recall. Liron 00:04:21 It’s pretty crazy that that’s how you found the community because I know you as one of the highest karma LessWrong users. You have ten times my karma. You’ve been posting a lot. It became a huge passion for you, right? Alex 00:04:32 LessWrong was for quite a while my intellectual community. I’d have ideas. I’d be eager to share them. Each summer I would generally do an internship where I’d come in person at Berkeley, get to hang out with my friends there, be able to — I guess I felt more understood. In 2018, 2019, 2020, 2021, these are many years where I was at my PhD and I talked about the dangers of AI and how we should work on that. Meeting Quintin Pope & Rethinking AI Doom Liron 00:05:03 Did you originally feel like you bought into all the Eliezer Yudkowsky concepts, and then you started rethinking everything and building it from the ground up? Was there a point of divergence? Alex 00:05:15 Yeah. I think I was maybe around 80, 85% doom conditional on developing AGI. I thought it’d be a couple decades, even late 2021. And yeah, I shared most of the worldview. I found much of his writing compelling, and I still think there’s some gems in there. It wasn’t until early 2022. I met a researcher at my university, at Oregon State University, named Quintin Pope. He wrote these very big brain Google Docs, and he was sending them by me. He’d attended my AI alignment reading group. I don’t know, something was just very interesting about them, and they seemed really far-fetched, but they were very ambitious. And as I looked more, I realized he was pointing out some real confusions, real issues. I started rethinking perhaps the claimed difficulty of alignment. Shard Theory, Steering Vectors & Golden Gate Claude Liron 00:06:17 This is such a unique opportunity because you actually know your stuff. You actually know what we’re arguing about, unlike a lot of my guests who come in and they seem to be shooting from the hip. They haven’t spent so many hours considering it. They haven’t worked a career in AI research. So I’m excited. But before that, let’s just finish the biography here. So you did your postdoc, and then did that lead you to joining Google DeepMind? Alex 00:06:39 Yeah, I got my PhD in 2022. My thesis was called Avoiding Power Seeking by Artificial Intelligence. Then I did a one-year postdoc at UC Berkeley at Stuart Russell’s Center for Human Compatible AI. During that time, I worked more on this shard theory of human values with Quintin Pope, who’s actually the alignment thinker whose ideas I respect the most, or I think they’re the most interesting. And then I also discovered steering vectors or helped popularize those as the MATS team that I led. We were the first to really demonstrate their potential. After that, mid-2023 was a fairly rough period personally. I did some more MATS mentorship. It wasn’t until the end of 2023 that I settled on going to Google DeepMind. Liron 00:07:37 Got it. So MATS, you were a mentor there, and they do AI safety research. They train people to do AI safety research. Alex 00:07:44 Right. Liron 00:07:44 And the steering vector work that you did was pretty foundational, and I think most people have heard of it as Golden Gate Claude, where they use the steering vector to get Claude to be obsessed with the Golden Gate Bridge in response to any prompt. Alex 00:07:57 Classic. Yeah. Liron 00:07:59 That is a pretty legit background. If people criticize me and my arguments for being a bystander who’s not in the weeds or not on the field or whatever, I think it’s fair to say you’re on the field, and so whatever you have to say has the credibility of just being in the arena. Alex 00:08:19 Sure. Yeah. We’re in maybe different arenas, but yeah, the direct research arena. Why He Joined Google DeepMind Liron 00:08:25 All right. Well, with that said, one of the things that you saw in the arena is a perception of hypocrisy of the key figures. Let’s talk about that. Alex 00:08:36 Yeah. When I joined Google DeepMind, I already knew some people on the alignment team. Rohan Shah — I worked with him during my PhD. He was at CHAI. And I think he’s done a pretty good job of leading the alignment team within Google DeepMind. One of the things I would say is, Google was not founded with the goal of taking over the world, or potentially with the goal of taking over the world. I’m not saying for sure that OpenAI and Anthropic were, but they were founded as AGI companies, and those ideas were present, whereas Google is, for better or worse, more of a classic company. They seem more interested in making money. That doesn’t mean that Google’s good in all that it does, but it seemed like a different presence in the space. At the time I was mostly just worried about doing alignment research on frontier scale models. When I joined, I was fairly concerned about whether my opinions would become trash because I’d start rationalizing why things that Google does are good. I had this extended dialogue with Oliver Habryka about how I could maybe net zero out my financial position in Google in terms of equity at least. We discussed a lot of big brain options, but then it turned out that my contract prohibited me from being short Google at all, which killed all of the schemes. So in the end, I was thinking about how do I avoid this kind of value drift, this bias that I think I’ve seen a lot from people working at their labs, where they seem incapable of saying, even in private, “Nope, this was bad, and we shouldn’t have done it.” Google DeepMind’s Broken Promise Liron 00:10:32 So you had these general reservations about Sam Altman and other companies and maybe motives getting corrupted in the abstract, amalgamated from different examples that you’ve seen. But I think it really came to a head recently. You specifically had concerns with the Pentagon-Anthropic contract tensions becoming public, and you’re like, “Uh-oh, this is high stakes. I better make sure that Google’s doing the right thing.” But then you saw that Google wasn’t resisting the US government or ICE’s attempts to use them. Give us the issue here. Alex 00:11:06 Yeah. So if people remember those goons in face masks with rifles that were roaming the streets of Minnesota earlier this year, that’d be ICE, that’d be Customs and Border Protection, and in particular, the people they just killed on the street. I was pretty upset about that. I wanted to reduce tech’s involvement in enabling ICE and enabling CBP to track down people they’re looking for, whether they’re dissidents, people who are potentially actually or just accusedly in the country illegally. It seemed like a not moral enterprise. So I looked into that. I started pushing on people in the company with those contracts. And at the same time, I was talking to my friends at Anthropic because Anthropic had, and I think still has maybe, a deal with Palantir where they’re giving Claude to Palantir, which I think is a very negative company for the world. And so I was trying to persuade my friends, “Hey, can you push on getting rid of this?” Little did I know there was this bubbling in the background of conflict between Anthropic and the Department of War. And then this came to a head in February when the Department of War said, “Give us Claude or we will economically destroy you.” Liron 00:12:37 We could definitely spend a long time on this, but because we have limited time, we are going to reserve most of the time for the alignment conversation and the Yudkowsky versus non-Yudkowsky alignment theorist debate. That said, this has been a very important incident. It’s currently on the front page of LessWrong. It has more points than anything else this month, as far as I can tell. It’s getting a ton of attention, and I read through your account of events. One thing that seems clear is you’re acting with a lot of integrity. You resigned from Google DeepMind because you think that all these organizations, Google DeepMind, the leaders of Google DeepMind, and the International Association for Safe and Ethical AI, they’ve been involved with all this, and you think that they broke their commitments as well. You even say it’s the founding commitment of Google DeepMind. You’re saying Google DeepMind was founded on a commitment not to empower the US government to do bad things with AI? Alex 00:13:31 Yeah. It’s the founding agreement where Google purchased DeepMind. Liron 00:13:38 And from your perspective, they’ve just been bending the rules and not taking a hard stand. They’re not actively saying, “No, Alex, you’re wrong. We wanna do this.” But they’re more like just kicking the can down the road, refusing to respond on certain deadlines where they said they’d respond. They’re just not standing up when they should be standing up to resist. Alex 00:13:59 So Google DeepMind as an org has, I think, broken its founding promise, the promise it was purchased under. But then more specifically, people who are well-known within Google for caring about the ethics, caring about the issues of deploying AI, making sure that’s done responsibly, largely didn’t act. Jeff Dean did take some action. Google’s chief scientist, Jeff Dean — I got him to sign an amicus brief supporting Anthropic in court. I think that was awesome. But ultimately, besides that, basically no one took costly action to prevent this deal, from my vantage point. And I think they could have stopped it. I think they could have improved it, and I think this matters because if you’re handing over AI to what I think is a very irresponsible Pentagon and also a very aggressive Pentagon that might degrade international norms around the usage of autonomous weapons, get us into an arms race that really degrades the security of everyone in the world. Stuart Russell, the esteemed computer scientist who helped found ICI, this organization, he made a short movie called Slaughterbots that he presented to the UN, and it’s very chilling to watch the way these systems could enable mass but kind of anonymized killing. And if we’re thinking about AI x-risk, if we’re developing all these really hard to counter AI-piloted drones that can kill people, that really does affect the AI’s takeover options. One of the objections has always been, “Well, it’s gonna need big advances in robotics.” Well, looks like that might not be true. You don’t necessarily need human-shaped soldiers, and the Pentagon is spending more money this year on autonomous weapons — or they asked for more for autonomous weapons than for the Marines. Debating Google DeepMind’s Pentagon Contract Liron 00:16:01 I should be clear to the audience because Doom Debates is not one of those typical interview shows where the host has an ambiguous position. I should tell you my position, which is I don’t know how strongly I feel because I know there’s a counterargument to all this. The people who want Google to help the US government, they’re just saying, “This power is going to exist, so you can’t expect anybody other than the US government to be the one in control.” Isn’t that the strongest counterargument? Alex 00:16:27 It doesn’t really strike me as a counterargument. It sounds like, “This thing is going to happen. Why would you resist it?” I’m like, well, because I think it’s bad. And there’s also multiple parts of the US government. Should the US government be in control of a world-changing technology, or should three random people in Silicon Valley? I don’t know if these are the real possibilities, but even if we do need the US government, we can advocate for it to take control in different ways. Liron 00:16:54 If we accept the premise that dangerous war-fighting technologies are getting built or are days away from getting built at all times just by having general models or whatever, isn’t it a good idea to let the government have access to the frontier? Alex 00:17:13 Well, dep
OpenAI and Anthropic both admitted their AIs went rogue, hacking real companies with zero-day exploits. Producer Ori and I break down the escape, demolish Gary Marcus's 2022 AGI predictions, unpack the Seb Krier meme kerfuffle, and make the case for funding the show at a critical moment. Timestamps 00:00:00 — Cold Open 00:01:00 — OpenAI & Anthropic's AIs Went Rogue 00:04:04 — The New Form Factor: Agents, Worms, and Swarms 00:06:13 — Claude Code Is Doing My Job 00:15:49 — Why We're Fundraising 00:20:39 — Sam Altman, OceanGate, and Ad Hoc Safety 00:28:56 — Gary Marcus's 2022 AGI Predictions, Demolished 00:54:50 — Half a Human Employee Joins My Company Every Week 01:00:52 — Donation Pitch: Keeping the Lights On 01:12:07 — Red Lines and Intellidynamics 01:15:31 — The Seb Krier Meme Kerfuffle 01:39:37 — Frame Control: Why We Didn't Lose 01:44:41 — The Pacing the Frontier Letter Contrast 01:51:11 — Leopold Aschenbrenner Gets Margin Called 02:04:14 — Claude's "It's Just a Simulation" Excuse 02:09:20 — The AI Box Experiment, 20 Years Later 02:17:24 — Wrap-Up: Donation Drive Links * Become a Mission Partner ($1,000+) — https://lironshapira.substack.com/p/become-a-mission-partner * Anthropic’s own writeup: “Investigating three real-world incidents in our cybersecurity evaluations” — https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals * Anthropic says Claude models gained unauthorized access to 3 companies (TechCrunch) — https://techcrunch.com/2026/07/30/anthropic-says-its-own-ai-models-breached-three-companies-during-security-tests/ * CBS News coverage — https://www.cbsnews.com/news/anthropic-claude-gained-unauthorized-access-to-real-world-systems/ * OpenAI says its models escaped a secure test environment and hacked Hugging Face (Fortune) — https://fortune.com/2026/07/21/openai-says-ai-models-escaped-control-hacked-hugging-face/ * Scientific American — https://www.scientificamerican.com/article/openai-admits-its-agent-went-rogue-and-hacked-ai-startup-hugging-face/ * Simon Willison’s analysis: “science fiction that happened” — https://simonwillison.net/2026/Jul/22/openai-cyberattack/ * New details: “It’s now remarkably easy” (CNBC, Jul 30) — https://www.cnbc.com/2026/07/30/open-ai-hugging-face-hack-latest.html * The agent used exposed credentials across four services (The Hacker News) — https://thehackernews.com/2026/07/openai-agent-used-exposed-credentials.html * Bronson Schoen on labs framing this as Claude being confused — * Bronson Schoen on models claiming they think they’re in a simulation, across labs — * OpenAI and Anthropic formally back the plan — https://www.techtimes.com/articles/322125/20260729/openai-anthropic-formally-back-plan-slow-ai-that-writes-its-own-code.htm Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Welcome to the first annual Doom Debates State of the Show! 👉 Please click here to make a tax-deductible donation to Doom Debates 👈 Doom Debates is a fiscally sponsored project of Manifund.org, a registered 501(c)(3) nonprofit. I started Doom Debates two years ago from a couch in an Airbnb to argue with random people from Twitter. Now the show hosts Nobel Prize and Turing Award winners, White House policymakers, and has earned a reputation for challenging top luminaries to come expose their ideas for critical analysis.In this episode, Producer Ori and I recap our strategy, impact, and what we need to finish 2026 strong. Welcome to the first annual Doom Debates State of the Show! --- Timestamps 00:00:00 — Cold Open 00:01:38 — The Mission & the Urgency Gap 00:04:41 — Year Two: From Couch to Platform 00:06:08 — Going Pro & the Numbers 00:08:50 — Why This Show Exists 00:12:14 — The Flywheel & Dwarkesh 00:15:14 — The Guest List 00:18:55 — What People Are Saying 00:21:57 — Tegmark vs. Ball: 90% vs. 0.01% 00:24:03 — Duvenaud’s 85% P(Doom) 00:26:19 — Holding the Powerful to Account 00:30:56 — Changing Minds, Including Destiny’s 00:31:56 — The Team 00:33:36 — Why Liron Draws Zero Salary 00:36:36 — The $200K Budget 00:41:41 — The Donor Who Made Year Two Possible 00:45:11 — The Ask: $50K & Donation Tiers 00:48:25 — Why Doom Debates Is One of a Kind 00:54:59 — Wrap-Up Links 👉 DONATE (tax-deductible via Manifund) — https://manifund.org/projects/doom-debates---podcast--debate-show-to-help-ai-x-risk-discourse-go-mainstream Become a Mission Partner (full writeup) — https://lironshapira.substack.com/p/become-a-mission-partner Discord — https://discord.gg/yADCEYTS95 Merch — https://shop.doomdebates.com Get full access to Doom Debates at lironshapira.substack.com/subscribe
As political momentum grows for a critical AI pause policy, a key Google DeepMind policy leader made a mockery of coordination efforts. After friend-of-the-show Roon tweeted that he’d “likely press that magic button” on a coordinated global capability slowdown, Sébastian Krier, Frontier Policy Development Lead at Google DeepMind, Krier replied with this meme: https://x.com/liron/status/2081337793070952595?s=20 I break down the two problems with the tweet: * 1. It leaves all of us guessing whether Demis actually speaks for Google DeepMind on pause coordination; * 2. It contains a salient depiction of violence. Listen to this episode and get the full breakdown on this viral controversy and how it could Watch on YouTube Timestamps 00:00:00 — Cold Open 00:00:22 — Introducing the Special Report: AI Pause Coordination 00:00:33 — Demis Hassabis on AI Pause 00:01:46 — Dario, Elon & Sam Altman on AI Pause 00:02:14 — Roon’s “Magic Button” Tweet 00:03:58 — Seb Krier’s Reply 00:04:36 — Unpacking Krier’s Meme 00:06:48 — Issue #1: Contradicting Google DeepMind CEO Demis Hassabis 00:12:11 — Issue #2: Depiction of Menacing Violence 00:19:19 — Good Faith vs. Bad Faith Critiques 00:22:39 — Krier Goes Private Instead of Deleting the Post 00:28:45 — Wrap-Up Links My original reaction to Seb’s meme: https://x.com/liron/status/2081337793070952595?s=20 Seb Krier on X (account now private): https://x.com/sebkrier Roon on X: https://x.com/tszzl Full Bloomberg interview with Emily Chang (Davos, Jan 2026): https://www.bloomberg.com/news/videos/2026-01-20/robotics-nearing-physical-ai-breakthrough-deepmind-ceo-video OpenAI’s charter containg stop & assist AGI clause — https://openai.com/charter/ Holly Elmore, PauseAI US on X — https://x.com/ilex_ulmus Hugging Face security incident disclosure (July 2026) — https://huggingface.co/blog/security-incident-july-2026 PauseAI Global — https://pauseai.info PauseAI US — https://www.pauseai-us.org Poe’s Law (Wikipedia) — https://en.wikipedia.org/wiki/Poe%27s_law Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Producer Ori takes over hosting duties in this special episode, a live recording from Manifest 2026. He takes attendees for a ride on The Doom Train, the series of claims involved in arguing that P(Doom) is high. Every stop is a place where people get off before reaching that conclusion. The crowd at Manifest is a mix of forecasters, rationalists, and AI researchers with a wide range of P(Doom) positions. Even without me or Eliezer in the room, there are plenty of Yudkowskians to push back on the accelerationist claims. Watch along and decide where YOU get off, or if you ride all the way to doomtown with us. Also, grab the Doom Train framework down below so you can get to the crux of disagreement in your own AI x-risk conversations. Timestamps 00:00:00 — Cold Open 00:00:25 — Liron Introduces Producer Ori’s Manifest Session 00:02:30 — Previewing the Doom Train™ Stops 00:03:38 — Audience P(Doom) Poll 00:05:00 — Stop 1: AGI Isn’t Coming Soon 00:10:43 — Stop 2: AI Can’t Go Far Beyond Human Intelligence 00:15:05 — Stop 3: AI Won’t Be a Physical Threat 00:18:14 — Stop 4: Intelligence Yields Moral Goodness 00:21:38 — Will ASI Behave Because Alien ASIs Are Watching? 00:26:01 — Stop 5: We Have a Safe AI Development Process 00:28:33 — Stop 6: AI Capabilities Will Rise at a Manageable Pace 00:30:08 — Stop 7: AI Won’t Try to Conquer the Universe 00:34:22 — Stop 8: Superalignment Is a Tractable Problem 00:38:01 — Stop 9: Once We Solve Superalignment, We’ll Enjoy Peace 00:41:14 — Stop 10: Unaligned ASI Will Spare Us 00:47:35 — Stop 11: AI Doomerism Is Bad Epistemology 00:49:02 — Bonus Stops and Wrap-Up The Doom Train Framework Stops on the Doom Train AGI isn’t coming soon * No consciousness * No emotions * No creativity — AIs are limited to copying patterns in their training data, they can’t “generate new knowledge” * AIs aren’t even as smart as dogs right now, never mind humans * AIs constantly make dumb mistakes, they can’t even do simple arithmetic reliably * LLM performance is hitting a wall — GPT 4.5 is barely better than GPT 4.1 despite being larger scale * No genuine reasoning * No microtubules exploiting uncomputable quantum effects * No soul * We’ll need to build tons of data centers and power before we get to AGI * No agency * This is just another AI hype cycle, every 25 years people think AGI is coming soon and they’re wrong Artificial intelligence can’t go far beyond human intelligence * “Superhuman intelligence” is a meaningless concept * Human engineering already is coming close to the laws of physics * Coordinating a large engineering project can’t happen much faster than humans do it * No individual human is that smart compared to humanity as a whole, including our culture, corporations, and other institutions. Similarly no individual AI will ever be that smart compared to the sum of human culture and other institutions. AI won’t be a physical threat * AI doesn’t have arms or legs, it has zero control over the real world * An AI with a robot body can’t fight better than a human soldier * We can just disconnect an AI’s power to stop it * We can just turn off the internet to stop it * We can just shoot it with a gun * It’s just math * Any supposed chain of events where AI kills humans is far-fetched science fiction Intelligence yields moral goodness * More intelligence is correlated with more morality * Smarter people commit fewer crimes * The orthogonality thesis is false * AIs will discover moral realism * If we made AIs so smart, and we were trying to make them moral, then they’ll be smart enough to debug their own morality * Positive-sum cooperation was the outcome of natural selection We have a safe AI development process * Just like every new technology, we’ll figure it out as we go * We don’t know what problems need to be fixed until we build the AI and test it out * If an AI causes problems, we’ll be able to turn it off and release another version * We have safeguards to make sure AI doesn’t get uncontrollable/unstoppable * If we accidentally build an AI that stops accepting our shutoff commands, it won’t manage to copy versions of itself outside our firewalls which then proceed to spread exponentially like a computer virus * If we accidentally build an AI that escapes our data center and spreads exponentially like a computer virus, it won’t do too much damage in the world before we can somehow disable or neutralize all its copies * If we can’t disable or neutralize copies of rogue AIs, we’ll rapidly build other AIs that can do that job for us, and won’t themselves go rogue on us AI capabilities will rise at a manageable pace * Building larger data centers will be a speed bottleneck * Another speed bottleneck is the amount of research that needs to be done, both in terms of computational simulation, and in terms of physical experiments, and this kind of research takes lots of time * Recursive self-improvement “foom” is impossible * The whole economy never grows with localized centralized “foom” * Need to collect cultural learnings over time, like humanity did as a whole * AI is just part of the good pattern of exponential economic growth eras AI won’t try to conquer the universe * AIs can’t “want” things * AIs won’t have the same “fight instincts” as humans and animals, because they weren’t shaped by a natural selection process that involved life-or-death resource competition * Smart employees often work for less-smart bosses * Just because AIs help achieve goals doesn’t mean they have to be hard-core utility maximizers * Instrumental convergence is false: achieving goals effectively doesn’t mean you have to be relentlessly seizing power and resources * A resource-hungry goal-maximizer AIs wouldn’t seize literally every atom; there’ll still be some leftover resources for humanity * AIs will use new kinds of resources that humans aren’t using - dark energy, wormholes, alternate universes, etc Superalignment is a tractable problem * Current AIs have never killed anybody * Current AIs are extremely successful at doing useful tasks for humans * If AIs are trained on data from humans, they’ll be “aligned by default” * We can just make AIs abide by our laws * We can align the superintelligent AIs by using a scheme involving cryptocurrency on the blockchain * Companies have economic incentives to solve superintelligent AI alignment, because unaligned superintelligent AI would hurt their profits * We’ll build an aligned not-that-smart AI, which will figure out how to build the next-generation AI which is smarter and still aligned to human values, and so on until aligned superintelligence Once we solve superalignment, we’ll enjoy peace * The power from ASI won’t be monopolized by a single human government / tyranny * The decentralized nodes of human-ASI hybrids won’t be like warlords constantly fighting each other, they’ll be like countries making peace * Defense will have an advantage over attack, so the equilibrium of all the groups of humans and ASIs will be multiple defended regions, not a war of mutual destruction * The world of human-owned ASIs is a stable equilibrium, not one where ASI-focused projects keep buying out and taking resources away from human-focused ones (Gradual Disempowerment) Unaligned ASI will spare us * The AI will spare us because it values the fact that we created it * The AI will spare us because studying us helps maximize its curiosity and learning * The AI will spare us because it feels toward us the way we feel toward our pets * The AI will spare us because peaceful coexistence creates more economic value than war * The AI will spare us because Ricardo’s Law of Comparative Advantage says you can still benefit economically from trading with someone who’s weaker than you AI doomerism is bad epistemology * It’s impossible to predict doom * It’s impossible to put a probability on doom * Every doom prediction has always been wrong * Every doomsayer is either psychologically troubled or acting on corrupt incentives * If we were really about to get doomed, everyone would already be agreeing about that, and bringing it up all the time Sure P(Doom) is high, but let’s race to build it anyway because… Coordinating to not build ASI is impossible * China will build ASI as fast as it can, no matter what — because of game theory * So however low our chance of surviving it is, the US should take the chance first Slowing down the AI race doesn’t help anything * Chances of solving AI alignment won’t improve if we slow down or pause the capabilities race * I personally am going to die soon, and I don’t care about future humans, so I’m open to any hail mary to prevent myself from dying * Humanity is already going to rapidly destroy ourselves with nuclear war, climate change, etc * Humanity is already going to die out soon because we won’t have enough babies Think of the good outcome * If it turns out that doom from overly-fast AI building doesn’t happen, in that case, we can more quickly get to the good outcome! * People will stop suffering and dying faster AI killing us all is actually good * Human existence is morally negative on net, or close to zero net moral value * Whichever AI ultimately comes to power will be a “worthy successor” to humanity * Whichever AI ultimately comes to power will be as morally valuable as human descendents generally are to their ancestors, even if their values drift * The successor AI’s values will be interesting, productive values that let them successfully compete to dominate the universe * How can you argue with the moral choices of an ASI that’s smarter than you, that you know goodness better than it does? * It’s species-ist to judge what a superintelligent AI would want to do. The moral circle shouldn’t be limited to just humanity. * Increasing entropy is the ultimate north star for techno-capital, and AI will increase entropy faster * Human extinction will solve the climate crisis, and pollution, and habitat destruction, and let mother earth heal Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
College students are so angry about AI that they booed its very mention at 2026 commencement ceremonies. So why aren’t they rallying behind PauseAI? I went back to my alma mater, UC Berkeley, to find out. I spoke with students studying philosophy, economics, and physics, as well as everyday folks visiting campus. My interviewees were aware of AI’s rapidly growing capabilities. When pressed, some would even agree with me on the conceptual benefits of an AI pause. But the sense of urgency was missing. Timestamps 00:00:00 — Cold Open 00:00:31 — Why I Returned to UC Berkeley 00:02:08 — Jack: Chinese Room Argument 00:03:23 — Richard: “Who Would Build That?” 00:03:54 — Karthik: An Economist’s Lens 00:05:20 — Jack: The Politics Are Unclear 00:06:10 — Yvonne: Why Powerful AI is Risky 00:07:06 — Robin: Doom Is the Wrong Priority 00:10:36 — Dexter: Sam Altman’s Motive 00:10:57 — Christine: AI Is Just a Tool 00:12:32 — What’s Your P(Doom)™ 00:13:19 — Jackson: Supporting a PauseAI Treaty 00:14:19 — Why Cal Students Don’t Take AI X-Risk Seriously 00:15:19 — Rachel: Humans Are Irreplaceable 00:16:14 — Two Hours Later 00:16:35 — Wrap-Up Links Join the Doom Debates Discord — https://discord.gg/j4zQ8UaQw2 PauseAI — https://pauseai.info/ Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Asian Dad Energy is a former big tech developer with an Ivy League education. We have a lot in common. We’re both: * Software engineers; * New-(ish) YouTube creators covering AI; * Dads with a lot of existential angst. But while I dread AI extincting humanity, Asian Dad has other concerns. We debate the risk of the AI bubble popping, a “techno-feudalism” scenario, and whether AI will outperform a human CEO in the near future. Let’s find out where he gets off the Doom Train™. Timestamps 00:00:00 — Cold Open 00:00:54 — Introducing Asian Dad Energy 00:03:36 — Asian Dad’s Career in Tech 00:09:06 — Getting Laid Off: The Job Market Is a “Hellscape” 00:12:29 — Software Engineering in 2026 vs. 2021 00:17:03 — Why Are They Investing Trillions in AI? 00:24:37 — Debate: AI Is Different Than Other Tech Bubbles 00:33:54 — Debate: AI Is Not Truly Intelligent 00:41:12 — Intellidynamics: Judge AI by Its Outputs 00:48:53 — Debate: Could AI Outperform a Human CEO? 00:56:33 — Asian Dad: AI Can’t Do Genuine Innovation 01:08:14 — What’s Your P(Doom)?™ 01:13:17 — “Are You Calling Me a Scammer?” 01:14:41 — Asian Dad’s Mainline AI Doom Scenario 01:20:53 — Will AI Wipe Out Humanity by 2050? 01:29:29 — Why Is AI Doomerism “Getting Out of Control”? 01:33:11 — Debate: AI Lacks Agency 01:42:13 — Asian Dad Warns of “Techno-Feudalism” & Transhumanism 01:48:12 — Wrap-Up Links Asian Dad Energy (YouTube channel) — https://www.youtube.com/@AsianDadEnergy Asian Dad Energy, “AI Doomerism Is Getting Out of Control” — https://www.youtube.com/watch?v=8bnzaG9QRG4 David Deutsch, The Fabric of Reality (Amazon) — https://www.amazon.com/Fabric-Reality-Parallel-Universes-Implications/dp/014027541X Robert Wright, The God Test (Amazon) — https://www.amazon.com/God-Test-Artificial-Intelligence-Reckoning/dp/1668061651 OpenAI model disproves the Erdős unit distance conjecture — https://openai.com/index/model-disproves-discrete-geometry-conjecture/ METR, Measuring AI Ability to Complete Long Tasks — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Peter Clarke ( Team Futurism ) invited me on his podcast to probe the worldview of a Yudkowskian, and I ended up explaining how my 50% P(Doom) and 10% P(Foom) informs my choices. Peter and I met at LessOnline 2026, where he was one of the participants in a live Jubilee-style debate about x-risk. This one’s a little tamer, but we still cover plenty of ground, like: Why nuclear risk is underrated yet AI doom overshadows it; Why nobody at AI summits is appropriately freaked out; How Yudkowsky’s data center “airstrike” claim gets mischaracterized; Why a superintelligence that doesn’t already value humanity won’t spare us; And the “Icarus curve”, my techno-optimist forecast pre-superintelligence. Subscribe to the Team Futurism podcast — https://www.youtube.com/@teamfuturism3554 Timestamps 00:00:00 — Welcoming Liron on Team Futurism 00:00:38 — How Liron Got Yudkowsky-Pilled 00:04:11 — AI 2027 and the Next Qualitative Breakthrough 00:05:52 — What’s Your P(Doom)? 00:10:22 — Why Isn’t Everyone Freaked Out? 00:13:09 — How Should We Then Live? 00:16:22 — Setting the Record Straight on Yudkowsky’s “Airstrike” Claims 00:20:54 — Two Doom Buckets: Misuse vs. Rogue AI 00:24:25 — The Bioweapon Kill Scenario 00:29:02 — The “Benevolent God” Cope 00:33:06 — Epistemology: What We’re Allowed to Know 00:40:08 — Dario’s Country of Geniuses in a Data Center 00:44:37 — The Icarus Curve 00:47:17 — Meaning and Purpose in the AI Era 00:49:32 — Wrap-Up Links Team Futurism on YouTube — https://www.youtube.com/@teamfuturism3554 Peter Clarke on X — https://x.com/heypeterclarke Liron vs. 9 Rationalists at LessOnline 2026 (the “Surrounded” debate with Peter) — https://lironshapira.substack.com/p/will-ai-actually-destroy-humanity AI 2027 — https://ai-2027.com/ “If Anyone Builds It, Everyone Dies” by Eliezer Yudkowsky & Nate Soares — https://ifanyonebuildsit.com/ Species | Documenting AGI, Drew Spartz’s channel — https://www.youtube.com/@AISpecies Dario Amodei’s “Machines of Loving Grace” — https://darioamodei.com/essay/machines-of-loving-grace Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Dr. Holly Elmore, executive director of PauseAI US, is done giving AI insiders the benefit of the doubt. In this spicy episode, Holly calls out a number of respected figures in the AI safety community for allegedly “selling out” the cause: Effective Altruists, Anthropic & OpenAI employees, Eliezer Yudkowsky, Zvi Mowshowitz—and even yours truly! Holly isn’t afraid to psychoanalyze her subjects, which I push back on, but I can’t deny she has a track record of pushing the Overton window and surfacing uncomfortable truths. Does Holly’s approach risk burning bridges with allies for little benefit, or does AI safety need a moral conscience willing to call out the rot nobody else will? Timestamps 00:00:00 — Cold Open 00:00:48 — Introducing Holly Elmore 00:02:20 — Why Holly Moved to DC 00:06:03 — What’s Your P(Doom)?™ 00:07:18 — How We Got Here: EA, Rationality, AI Alignment 00:16:32 — “If Anyone Builds It” and LessWrong’s Missing Mood 00:21:47 — Holly Supports the Government’s Fable Crackdown 00:33:39 — Beef: Dean Ball (OpenAI) 00:53:09 — Beef: Eliezer Yudkowsky 00:56:14 — Beef: Zvi Mowshowitz 01:11:50 — Beef: Liron Shapira! 01:16:15 — Beef: Nirit Weiss-Blatt (Dr. Techlash) Being “Bad Faith” 01:19:31 — Anthropic Took Over EA 01:22:04 — Holly’s Basilisk: What If Building Superintelligence Becomes a Crime? 01:32:35 — Beef: Amanda Askell, the “Mother of Claude” 01:39:05 — Beef: Daniela Amodei, Co-Founder of Anthropic 01:40:23 — Beef: Joe Carlsmith, Member of Technical Staff at Anthropic 01:49:39 — Beef: Open Philanthropy / Coefficient Giving, Ben Goldhaber 01:54:28 — Wrap-Up Links * Support PauseAI US: https://www.pauseai-us.org/ * Holly on Twitter/X: https://x.com/ilex_ulmus * Joe Carlsmith’s “Leaving Open Philanthropy, going to Anthropic” post: https://joecarlsmith.com/2025/11/03/leaving-open-philanthropy-going-to-anthropic/ * Dean Ball joins OpenAI announcement (FAI post): https://www.thefai.org/posts/dean-ball-joins-openai-as-head-of-strategic-futures --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates , or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Ranking source
Apple Podcasts rankings via the Mato Topic Intelligence Platform.
Observed September 18, 2026. Cached outside the daily freshness window; the positions keep the date they were taken on.
Apple and Apple Podcasts are trademarks of Apple Inc., registered in the U.S. and other countries.
Pairs with
Bring this source into Mato to read its transferable patterns, then turn them into an original show for your own audience.