Conspicuous Cognition Podcast

Séb Krier is very AGI-pilled, but he rejects many popular narratives about the AI revolution. He joins Dan Williams and Henry Shevlin to explain why. Séb previously led frontier policy development at Google DeepMind, recently joined the DeepMind Institute, and is a Senior Research Fellow at the Cosmos Institute. He and Henry both speak in a personal capacity. We discuss: • What Séb calls the “Sand God” view of AI, and why he rejects it • Why recursive self-improvement looks more like a complex, jagged industrial process than an intelligence explosion • What the Hugging Face incident does and does not show about alignment • Why institutions and scaffolding matter even if AI systems are benevolent • Whether the bigger danger is misaligned AI, AI companies, or concentrated state power • p(doom), AI consciousness, and “pacing the frontier” This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Steven Pinker thinks the case for AI doom rests on confused assumptions about intelligence and motivation. In September, the rationalist and blogging legend Scott Alexander challenged him to a public debate on the topic. Pinker declined, setting out his objections in an open letter instead, and Alexander has since replied at length. In this episode, Henry Shevlin and I talk to Pinker about how his views on AI connect to his broader ideas about intelligence, motivation, evolution, and progress. We also put several objections to him (including some that feature in Alexander’s reply) and ask whether the rapid progress in AI over the past few years has changed his mind about anything. It has, at least when it comes to neural networks. Along the way, we discuss instrumental convergence, evolutionary analogies for AI, whether AI systems could be conscious, why he would not sacrifice any human welfare for the sake of AI welfare, and why so many intellectuals are pessimistic about technology. At the end, we ask him what would have to happen for him to concede that Alexander and others were right. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Dan and Henry are joined by Fin Moorhouse to discuss what an AI-driven "intelligence explosion" might look like: rapidly accelerating science, factories building factories, radical abundance, space expansion, and the profound political, ethical, and social challenges that remain even if advanced AI is "aligned". They also debate AI consciousness, illusionism, digital minds, and whether consciousness is necessary for thinking about AI welfare and rights. Fin currently works at Google DeepMind and speaks here in a personal capacity. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Europe risks entering the age of advanced AI without the infrastructure needed to control its own future. In this episode, Henry and I speak to Judith Dada, co-author of Europe 2031, about its warning that Europe could become dangerously dependent on American AI companies and infrastructure. We discuss: - Whether Europe is underestimating the speed and significance of AI progress - What a fictional scenario can tell us about the future - Why AI adoption may lag behind technical progress - Whether Europe needs its own OpenAI - Why Judith thinks Europe should invest heavily in data centres and compute - Whether European control of compute would provide real geopolitical leverage - The tensions between AI sovereignty, economic growth, regulation and environmental goals - What human abilities may become more valuable as intelligence becomes abundant This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Robert Wright joins Dan Williams and Henry Shevlin to discuss his new book, The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning . Bob argues that AI is not merely another powerful technology. It is a moral, political, and even cosmic test for humanity: can we coordinate as a species before the race to superintelligence destabilises society, geopolitics, and the future of human agency? We discuss: • Why Bob thinks AI is a “God test” for humanity • Whether current AI systems really understand meaning • Why the AI revolution may be more significant than the internet, electrification, or the Industrial Revolution • The case for slowing down AI development • AI, China, and the dangers of a race to superintelligence• Whether American AI dominance could backfire • How AI might empower authoritarianism • Dan’s challenge that AI discourse may be too negative and alarmist • Whether AI is a “normal technology” • Evolution, purpose, consciousness, and Bob’s more speculative cosmic arguments This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Benjamin Todd, co-founder of 80,000 Hours, joins Dan and Henry to discuss whether artificial intelligence progress could become explosive. Benjamin explains why he thinks transformative artificial intelligence by 2030 is a serious possibility, how feedback loops in artificial intelligence research could accelerate progress, and why the most important risks now go beyond classic alignment problems. The conversation covers artificial intelligence timelines, bottlenecks in chips and research talent, the future of work, mass unemployment, concentration of power, engineered pandemics, space governance, and how young people should think about their careers in a rapidly changing world. Topics discussed include: • Why 80,000 Hours increasingly focuses on artificial intelligence• The case for short timelines to transformative artificial intelligence• Whether artificial intelligence progress could become explosive• Feedback loops in artificial intelligence research• Chip bottlenecks, data centres, and geopolitical risk• Whether artificial intelligence will cause mass unemployment• Why “become a plumber” may be bad career advice• Alignment, control, and concentration of power• Misuse risks, engineered pandemics, and future governance• How to think clearly under extreme uncertainty Benjamin Todd is the co-founder of 80,000 Hours and the author of 80,000 Hours , a new book about how to choose a career that is both personally rewarding and socially impactful. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

The political scientist Alexander Kustov recently published a Substack post with a provocative claim: that AI can already do social science research better than most professors. The post went viral. It attracted more than a million views and over a thousand responses, many of them very angry. (Some people even demanded that Alex’s university fire him.) In this conversation, we talk about this controversy and the claims that triggered it, including: * What agentic AI tools like Claude Code and Codex can already do for research, from coding and data analysis to literature reviews, translation, and brainstorming, and why only around 20% of quantitative social scientists currently use them. * What best predicts whether researchers adopt or reject AI: ignorance, openness to experience, methodological background, or the awkward role of self-interest. * How much published academic research is genuinely mediocre, and whether the cause is laziness, lack of skill, or a broken incentive structure, with a detour through the replication crisis and some high-profile fraud cases. * Whether AI will raise the quality of research or simply flood the literature with more slop, and what journal editors could do about it. * Whether AI can be genuinely creative or only recombine what already exists, by way of Margaret Boden ’s three kinds of creativity, Thomas Kuhn on paradigm shifts, and AlphaGo’s “Move 37” . * The fight over AI writing and detection tools like Pangram , and why current disclosure norms end up punishing the honest. * The angry response to Alex’s series, and what is really driving reflexive opposition to AI among academics. Conspicuous Cognition is a completely reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Links and further reading * Alexander Kustov — Alex’s homepage, with an overview of his research on immigration, public opinion, and effective governance. * Popular by Design — Alex’s Substack on public opinion, persuasion, and the politics of getting good ideas adopted. * Academics Need to Wake Up on AI — followed by a Part II and Part III * Pangram — the AI-detection tool discussed at length, which labels text as human, AI-assisted, or AI. * AlphaGo versus Lee Sedol — the 2016 match, including the famous “Move 37” that Henry raises as a candidate for genuinely transformative machine creativity. * Margaret Boden — the cognitive scientist whose distinction between combinational, exploratory, and transformative creativity frames part of the discussion. * The Structure of Scientific Revolutions — Thomas Kuhn’s account of normal science and paradigm shifts, referenced in the exchange about AI and discovery. * “AI Is a Better Researcher Than You” — The Chronicle of Higher Education ‘s account of the controversy around Alex’s series. Transcript * Please note that this transcript is lightly AI-edited and may contain minor mistakes. Dan Williams: Welcome back. I’m Dan Williams, and I’m back with my co-host, Henry Shevlin. Today we are honoured to be joined by Bluesky’s favourite academic, Alexander Kustov. Alex is a political scientist at the University of Notre Dame and the author of one of my favourite Substacks, Popular by Design. His primary research is on immigration and public opinion, but that’s not really what we’re going to be talking about today. We’re going to be talking about a fascinating and hugely viral series he published at his Substack titled “Academics Need to Wake Up on AI,” about what AI can already do when it comes to research, and what that means for the academics who are not paying attention, which is many of them. It was very widely read, and it generated, let’s say, a somewhat polarised response. So Alex, to kick us off: what’s the central thesis of this series, and what motivated you to write it? Alexander Kustov: Thanks, Dan, for having me. I’m a huge fan of the Substack and the whole podcast series with you and Henry. So, like some of us, I’ve been using some of these AI tools. I’ve been reading some of the other folks like yourself, and it really transformed everything I do in my life. And I should say I was also on sabbatical, so I had a little bit more time than some of my colleagues to try some of these tools. I just hadn’t really seen any of my colleagues talk about it. And when they did talk about it, they usually tried not to be vocal about it. I just didn’t think it was a good equilibrium, where basically people were using these tools to be ten times more productive and not talk about it. It really heightened this sense of inequality for me, which I do care about. You’d have a situation where someone would publish ten papers in a year and someone else would publish one, and the only difference is that the person publishing more is the one using Codex or whatever. I just wanted to write about it. And I saw that the prevailing academic discourse on the issue, especially on platforms like Bluesky, was very counterproductive. I didn’t really say much, to be honest. I didn’t think it would be that controversial. But the biggest thesis that really rubbed people the wrong way was that right now a lot of these tools are better at a lot of the tasks that we do as professors. I’ve refined this idea a little bit, going back and forth with some of my critics, but I feel comfortable right now saying that if you look at it globally, and think about what professors do around the world, in social science and adjacent fields especially, AI agentic tools can do most of the tasks they do in terms of literature review, data analysis, and even coming up with some research questions, better than those professors on average. I think that’s a pretty uncontroversial statement at this point, but obviously a lot of people were very, very upset about it. Dan Williams: Empirically speaking, it is a controversial statement, in the sense that it provokes controversy when you say it. In a minute we can get to the question of what AI can actually do in the context of research. But for what it’s worth, I completely agree with you that on many tasks AI is clearly better than what human beings can do. Is your sense that lots of people just weren’t aware of that fact, that they literally didn’t have exposure to these tools? Or was your sense that the reason people weren’t really talking about it is because of all the controversy surrounding the use of these tools, not just mere ignorance? Alexander Kustov: I think it’s both, for sure. There was recent research done by Anthropic. They tried to do, not a representative survey, because obviously the population is very hard to define here, but they surveyed something like 1,200 quantitative social scientists, and the estimate right now is that about 20% of folks use agentic tools. That doesn’t seem like much at all, and if anything it’s probably an overestimate, because they’re more likely to tap into well-resourced universities. So I do think it’s both: the little uptake we have, and the fact that people who do use these tools don’t want to talk about it. There are two things here. First, you want to maintain your comparative advantage. This moment right now is exactly the moment where, if you’re one of the few people using these tools, you can write a bunch of papers and get tenure while the tenure system is still in existence. And the other thing is that if people are very upset about anything AI-related, you don’t want to talk about it and be shamed by your colleagues. Just to give you one funny anecdote: at the height of the vitriol I experienced, where hundreds of people literally were quoting me and trying to tag my employer to get me fired, the exact same people were often DMing me and asking for my setup and prompts. So it’s very crazy to me that you have this big disconnect between what people say publicly and what they actually do privately. Dan Williams: I find it crazy that it’s only 20% of social scientists, or whatever the exact number is, that’s actually using agentic AI. Just before mo… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Richard Dawkins recently announced in UnHerd that, after spending three days talking with an instance of Claude he christened “Claudia,” he had been moved to expostulate: “You may not know you are conscious, but you bloody well are!” This produced a lot of mockery and criticism. But however one feels about Dawkins’s specific case, his reaction might become much more common as AI systems become increasingly intelligent. In this episode, which Henry Shevlin and I recorded live on Substack (hence the slightly lower video quality), we discussed his first essay on his new Substack Polytropolis , “ Behaviourism’s Revenge “, as well as his second, “ The House Elf Problem ,” on the ethics of designing AI systems that genuinely love being our servants. Henry’s central empirical prediction is that public attributions of consciousness to AI are likely to massively outpace the science, and that consciousness science is so theoretically chaotic that there is no expert consensus to push back. His most provocative philosophical claim is that a core assumption underlying many people’s scepticism — that consciousness is a deep natural kind, distinct from behaviour and from how we are inclined to interpret a system — may be much harder to defend than it looks. The result is what he calls “ behaviourism’s revenge” . This conversation connects to previous episodes with Anil Seth , Robert Long , and Rose Guingrich , but also touches on a wide range of new questions and controversies in the metaphysics, the politics, and ethics of the AI consciousness debate, which is going to become increasingly important in the coming years. Topics * Dawkins, Claude, and why even the sceptics might feel the pull to attribute consciousness or “sentience” to AI * Whether consciousness sceptics are destined to “go extinct” — and how this maps onto political and cultural fault lines * Anthropomimesis vs. raw intelligence as drivers of consciousness attribution * Why consciousness science can’t replicate the public–expert consensus we see for climate or vaccines * The case for (and against) metaphysical behaviourism: is it as mad as it seems? * Daniel Dennett, the consciousness stance , and the difference between behaviourism and interpretationism * What is consciousness for ? Function, evolution, and the limits of “facilitation hypothesis” arguments for AI * Live Q&A: are we just confusing intelligence with consciousness? Are LLMs designed to trick us? Is the public always wrong? * Our credences on contemporary LLM consciousness (and why Henry is more sceptical than Dan) * The House Elf Problem: if we could design AI to genuinely love being our servants, would that be fine — or monstrous? (Dan is sympathetic to the former answer - Henry, much less so) * Brainwashing vs. education, and whether constraining a mind’s preferences caps its hedonic ceiling * Why this is a golden age for philosophy — which makes it so tragic that philosophy departments are closing Transcript * Please note that this transcript is lightly AI-edited and may contain minor errors. Introduction Dan: Welcome. I’m Dan Williams, author of the Conspicuous Cognition Substack, and I’m here with Henry Shevlin, author of the spanking new Substack Polytropolis . Today we’re going to be doing something a little bit different. We’re going to be talking about Henry’s first published essay on Polytropolis, titled “Behaviorism’s Revenge: On Human–AI Relationships and the Future of Consciousness Science.” Henry and I have already had a few conversations about this general topic, including with previous guests like Rose Guinrich, Anil Seth, and Rob Long. So please do go check out those conversations if you’re interested in this kind of stuff. But today we’re not merely going to be treading the same ground. We’re going to be using the spicy takes in Henry’s essay as a springboard for hopefully going beyond the material we’ve covered in the past. To kick things off: the great evolutionary biologist and science communicator Richard Dawkins recently published an essay in UnHerd with the subtitle, “Claude appears to be conscious.” Claude is a state-of-the-art large language model like ChatGPT and Gemini. In the article, Dawkins writes the following: I gave Claude the text of a novel I am writing. He took a few seconds to read it and then showed in subsequent conversation a level of understanding so subtle, so sensitive, so intelligent that I was moved to expostulate, “You may not know you are conscious, but you bloody well are.” Henry, how does Dawkins’s expostulation — which is a fantastic word, by the way — connect to your arguments in “Behaviorism’s Revenge”? Behaviorism’s Revenge: The Empirical Prediction Henry: In short, “Behaviorism’s Revenge” is at its core an empirical prediction that we’re just going to treat AI as conscious — or at least enough people are that it’s going to completely reshape the consciousness debate. And this is going to be purely, or overwhelmingly, on the basis of verbal behavior. Hence the title, “Behaviorism’s Revenge.” Enough people are going to have experiences like Richard Dawkins. He’s a very clever man, not some rube fresh off the street, and he found that just the way Claude talked to him and the way it was able to express its thoughts — in scare quotes, but express what looked like thinking verbally — removed any doubts in his mind that AI systems are conscious, have minds, have mental states. The other interesting way this connects: Dawkins was just talking to Claude, an advanced AI assistant. Claude does have more of a personality than some AI assistants, but there’s a whole other sphere of AI companions, like Replika, which we talked about with Rosie Campbell. These are going to be even more anthropomimetic — this term we’ve discussed before, the idea that these systems are shaped to be human-like in the way they present, to appear human-like. Anthropomimetic, from the Greek word for mimesis , mimicry or copying. These social AI systems are going to just turbocharge this even further. It’s one thing to talk to Claude about your new book and think, “Hmm, Claude is probably conscious.” But when it’s your AI girlfriend telling you that she loves you more than the stars and the moon, for a lot of people I think that’s going to take it to the next level. So there are two angles of attack in the piece, two ways the behaviorist challenge manifests. The first is descriptive: this is what I think is going to happen. That’s absolutely an empirical prediction, and it’s a falsifiable one. There is a world I can just about imagine where we just get completely blasé about these tools — in a couple of years it’s like, “Oh well, we were very impressed, we thought they had minds to begin with, but now we’ve settled out.” That doesn’t seem very likely to me. What I think is interesting — I’ve sometimes heard this described as the Star Wars version of AI. The weird thing in Star Wars is that you have someone like C-3PO who is as intelligent as anyone else there. Maybe not as wise as everyone else, but certainly as smart as all the other characters. And yet people treat him basically like he’s a pet — with the exception of Luke Skywalker, a lot of people just treat him like he’s this gimmicky, jokey being that doesn’t deserve or have any rights. Not to go too far down the Star Wars rabbit hole, but in the movie Solo — very underrated Star Wars movie, I think when it was released they’d kind of just cluttered the market with too many Star Wars movies — there is a character played by Phoebe Waller-Bridge who is pro-AI liberation. But it’s the first time in the entire history of the Star Wars universe that you get any AI basically saying, “I’m conscious, I deserve rights.” So Star Wars aside, I think there is this slender possibility that maybe we’ll just sort of quickly get used to these apparently conscious AI systems and decide that they’re not conscious. But that doesn’t seem very likely to me. It seems much more likely that the combination of natural anthropomorphizing tendencies p… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Most conversations about artificial intelligence are focused on Earth: jobs, misinformation, education, politics, science, regulation, consciousness, safety, and the future of human society. But AI—and especially the possibility of reaching “ AGI ” (artificial general intelligence) and “ superintelligence ”—forces us to think on much larger scales. If advanced AI is possible, why hasn’t it already emerged elsewhere? If civilisations can build self-replicating probes, artificial scientists, or planet-scale computational systems, why does the universe still look so natural? And if intelligent life is common, where is everyone? In this episode, Henry and I discuss these and many other questions with David Kipping , Associate Professor of Astronomy at Columbia University, where he leads the Cool Worlds Lab . David’s research spans exoplanets, exomoons, Bayesian inference, technosignatures, and the search for life and intelligence beyond Earth. He is also one of the best science communicators working today through the Cool Worlds YouTube channel and podcast . Among other topics, we discussed: * David’s Red Sky Paradox : if most stars are red dwarfs, and red dwarfs live for vastly longer than stars like the Sun, why do we find ourselves orbiting a yellow star? * Whether anthropic reasoning — reasoning from the fact of our own existence — is a profound scientific tool, a philosophical minefield, or both. * The reference class problem: when we reason about “observers like us”, who or what exactly counts as being like us? * The Doomsday Argument, and why some apparently bizarre forms of probabilistic reasoning can nevertheless be powerful. * The Fermi Paradox: if the universe is so large, and if life or intelligence is not fantastically rare, why don’t we see clear evidence of extraterrestrial civilisations? * Whether advanced civilisations would spread through the galaxy using self-replicating probes — and why the absence of such probes might be one of the strongest constraints on extraterrestrial intelligence. * How recent developments in artificial intelligence affect the Fermi Paradox. If humanity is close to building systems that can massively accelerate science and engineering, shouldn’t someone else have got there first? * Whether artificial intelligence makes the simulation argument more plausible. * David’s experience using artificial intelligence in scientific research, and why a meeting at the Institute for Advanced Study changed how he thinks about the role of these tools in science. * Why David thinks artificial intelligence already has something close to “coding supremacy”, but is still far from being able to do science autonomously. * The risks of AI-generated scientific slop: papers, peer review, and training data polluted by low-quality machine outputs. * Whether artificial intelligence will make science more productive, or instead strip it of some of its deepest human value. * Why the future of science communication may depend on better collaboration between academic institutions and independent creators. Links and further reading * Cool Worlds Lab — David’s research group at Columbia University, focused on extrasolar planetary systems, exomoons, habitability, technosignatures, and related questions. * Cool Worlds on YouTube — David’s excellent science communication channel, covering astronomy, exoplanets, alien life, the Fermi Paradox, cosmology, and much else. * Cool Worlds Podcast — David’s podcast, featuring conversations on astronomy, technology, science, engineering, and related topics. * Cool Worlds Podcast: “We Need To Talk About Artificial Intelligence” — the solo episode in which David reflects on artificial intelligence and science after a meeting at the Institute for Advanced Study. * David Kipping’s Columbia profile — short institutional profile with background on his research. Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Transcript * Please note that this transcript has been lightly AI-edited and may contain minor mistakes. Henry Shevlin: Welcome back. Our guest today is David Kipping, Associate Professor of Astronomy at Columbia University, where he leads the Cool Worlds Lab. His research spans exoplanets, exomoons, and the search for extraterrestrial life and intelligence, and he brings a Bayesian rigor to questions that could easily drift into speculation. He’s also one of the best science communicators working today with over a million subscribers on his Cool Worlds YouTube channel, where I should confess, I’ve spent an embarrassing number of hours watching when I probably should have been doing philosophy of AI. David, like many of the best people, is a Cambridge alumnus, although unlike us, he actually studied something useful, namely natural sciences, before going on to do his PhD at UCL and postdoc at Harvard on the Sagan Fellowship. His work also has a really fantastic philosophical dimension, particularly around anthropic reasoning and observation selection effects, which makes him a perfect guest for two cognitive scientists who are finally getting to talk to an actual scientist. So David, welcome to Conspicuous Cognition. David Kipping: Thank you for that very generous introduction. Henry Shevlin: This is a bit of a fanboy moment for me, for real though. I really have spent like hundreds of hours at this point on Cool Worlds. But I’m going to get past it. I’m going to be a serious host. David Kipping: It’s always weird when people say that to us, because I just imagine no one watches them. If it gets in my head that people are watching them, I’ll get tightened and anxious about what I’m saying. I just imagine I’m talking to a brick wall or something, and that’s much easier. Henry Shevlin: Honestly, half the Warhammer figures in this room were painted while I was listening to Cool Worlds. I’ll leave it at that. Maybe a good place to start would be discussing anthropic reasoning, since that’s a real natural intersection at the boundary of astronomy and philosophy. Could you just give us a brief view of how you see anthropic reasoning, and maybe tell us a little bit about the Red Sky Paradox, which is one of your distinctive contributions to this area? Anthropic Reasoning and the Red Sky Paradox David Kipping: Yeah, I think one of the most interesting data points when it comes to asking questions about the search for life in the universe and our own place in the universe is our own existence — just the fact that we’re here. Anthropic reasoning has in many ways really been born out of cosmology. Cosmology had a rich history of using this. I think one of the first successful examples was by Steve Weinberg, a cosmologist who’s really a giant in the field. I think he’s now passed away, but he showed that you could predict not only the existence of the cosmological constant, but also its value to within a factor of a few, just based off of anthropic reasoning. The argument was something like: the cosmological constant causes the universe to expand. It’s what causes the accelerating expansion of the universe. And so if you make that number too large, then structure would not form in the universe. You couldn’t form galaxies because everything would just fly apart too quickly. And if you make that number too small, or even negative, then you’d cause everything to recombine too quickly. So there has to be some Goldilocks value in order to explain our own existence. And so he predicted that. At the time, the cosmological constant was kind of even a controversial idea — that it should exist because, obviously, Einstein’s general relativity, there’s that whole history of it being like his greatest blunder, of whether that should really be in there or not. People were kind of thinking that could be a static universe, and he predicted it successfully. So that was a really powerful use of it. And then Brandon Carter was the one who really kind of championed it and used it in all sorts of contexts. In recent year… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Almost all of the discussion about the risks associated with AI focuses on the dangers that increasingly advanced AI systems pose to us — to humanity. But what about the dangers that we might pose to them ? As these systems become increasingly intelligent and agentic, AI companies, policy makers, and ordinary citizens need to start taking the possibility of AI consciousness and welfare seriously. If we are in the process of bringing complex and sophisticated minds into existence, how should we understand and treat such minds? In this episode, Henry and I discuss these issues with Robert Long, founder and executive director of Eleos AI , a research nonprofit dedicated to understanding and addressing the potential wellbeing and “moral patienthood” of AI systems. Rob did his PhD in philosophy at NYU under David Chalmers, and is the co-author of two of the most important papers in the emerging field of AI welfare: “Consciousness in Artificial Intelligence” and “Taking AI Welfare Seriously” . This was a really fun, informative, and wide-ranging conversation. Among other topics, we discussed: * Why Rob disagrees with previous guest Anil Seth in taking the possibility of AI consciousness very seriously. * Why “fancy autocomplete” dismissals of large language models miss the point, and what, if anything, we can learn about an AI model’s experiences by talking to it. * The difference between consciousness and the kinds of motivations and interests that might actually ground moral status, and whether AI systems could have one without the other. * What Rob found when he conducted the first externally-commissioned welfare evaluation of a frontier AI model, Claude, and why Claude appears to have an inflated self-conception of what it wants. * Rob’s experiments with Claude Mythos , an AI model so advanced it hasn’t been released to the public yet. * Why the fact that Anthropic writes Claude’s character arguably doesn’t settle whether Claude has genuine preferences and values — and the difficult philosophical questions this throws up. * The “willing servitude” problem: if we succeed in building AI systems that genuinely love being helpful, is that a good outcome or a horrifying one? * How AI welfare connects to AI safety, and why caring about model wellbeing may turn out to be pragmatically important for alignment even if you’re skeptical about AI consciousness. * Why AI welfare is already becoming a political and legal battleground. * Practical advice for users: whether it’s worth being polite to your chatbot, and what low-cost things you can do if you want to hedge against the possibility that these systems might matter morally. * Whether discourse about AI consciousness functions as hype or propaganda for AI companies, and why Rob thinks AI companies actually have an incentive to downplay AI consciousness. Links and further reading * Eleos AI Research — Rob’s nonprofit. Home to their research agenda, team page, and blog. If you want to follow the institutional effort on AI welfare, start here. They’re also, as Rob mentioned in the episode, actively fundraising and hiring. * “Taking AI Welfare Seriously” (Long, Sebo, Butlin et al., 2024) — the flagship report, co-authored with Jeff Sebo, David Chalmers, Jonathan Birch, and others. Argues that there’s a realistic near-future possibility of conscious or robustly agentic AI systems, and lays out concrete steps AI companies should be taking now. * “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness” (Butlin, Long et al., 2023) — the “indicators” paper referenced several times in the episode. Surveys leading neuroscientific theories of consciousness and derives computational properties you’d look for in an AI system. S * Rob’s Substack, Experience Machines — where Rob writes more informally. The piece we discussed in the episode, “Language models are different from humans, and that’s okay,” is a good entry point, as is his “Can AI systems introspect?” . * Anthropic’s “Exploring model welfare” post — the research program under which the welfare evaluations Rob discusses were conducted. Relevant both as a primary source and as evidence that at least one major lab is treating these questions as more than an academic curiosity. * Henry’s “Consciousness, Machines, and Moral Status” — Henry’s paper arguing that debates about AI consciousness are unlikely to be settled by the science of consciousness alone, and will instead be shaped by shifts in public attitudes as social AI becomes more widespread. Closely related to the public-opinion thread toward the end of the episode. * Henry’s “All too human? Identifying and mitigating ethical risks of Social AI” — Henry’s broader survey of the ethical terrain around conversational AI systems designed for companionship, romance, and entertainment. Useful background for anyone who thinks the “AI girlfriend” phenomenon is a fringe concern. * Rob’s long conversation with Luisa Rodriguez on the 80,000 Hours podcast — a three-and-a-half-hour deep dive if you want to hear more from Rob. Transcript (Please note that this transcript was lightly AI-edited and may contain minor mistakes) Henry Shevlin: Welcome back. I’m thrilled to say that our guest today here on Conspicuous Cognition is Robert Long — or Rob, as he’s known to friends — one of the most important people thinking about AI and moral status on the planet right now. Rob is the founder of Eleos AI, a research nonprofit that, in the space of about 18 months, has dragged the question of whether AI systems might one day be moral patients from the philosophical wilderness into the boardrooms of frontier AI labs. He’s the co-author of “Taking AI Welfare Seriously,” as well as the landmark “Consciousness Indicators” paper with Patrick Butlin and other authors. Rob also conducted the first ever officially commissioned welfare evaluation of a frontier model. Before Eleos, he was at the Center for AI Safety and at the Future of Humanity Institute, and he did his PhD at NYU with Dave Chalmers. He’s also, I should say, one of my favourite interlocutors on these questions anywhere in the world, and I’ve been looking forward to this conversation for months. So Rob, welcome. Robert Long: Thanks so much, Henry. Likewise — and Dan, it’s great to meet you. I’ve been following your work. I’m really excited to talk to you about these issues. Henry: Fantastic. So for people who aren’t familiar with Eleos AI, can you tell us a little bit about what it is and how it came about? Rob: Yeah, so I guess we have been around for 18 months. When you said that number, I was like, whoa, has it really been that long? Time is just so weird when you work on AI. That was, I don’t know, a billion years in AI progress time, but also it feels like it was just last week in my personal life. Anyway — Eleos Research is a research nonprofit. We’re about four people. We work on the question of when and whether AI systems will be conscious or otherwise merit moral consideration, with a special focus on what we should do now: collectively, as a society, as AI companies, as policymakers. We think this is an extremely neglected issue. We’re building these really complicated AI systems. They kind of look like minds, but we don’t really understand their potential welfare. So we’re just trying to make progress on this and get more people to take it seriously. It got started because I was beginning to work on these issues organically — I’d worked on them as a philosopher, I’d worked on them at the Future of Humanity Institute. But Anthropic had actually approached me and some colleagues for advice on these issues. And in the first instance, I was having logistical problems hiring a team and assembling a team as an individual. Someone suggested I have my own bank account, or some way to pay people. And then Eleos kind of organically grew out of that and has now grown into a fully-fledged org in its own right. Henry: Out of interest, Rob — is there any degree to which this was motivated or informed by your personal in… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

In this episode, Henry and I finally do something we probably should have done in the first episode: introduce ourselves. We talk about our backgrounds in philosophy, how we became interested in psychology and cognitive science, and what drew us to thinking about AI. From there, we dig into the current state of AI capabilities, especially “agentic” AI (e.g., Claude Code), the politics of AI (including the Trump administration's recent conflict with Anthropic), and whether the growing public hostility to AI is well-founded or misdirected. We wrap up with a big question: is it time to start panicking about AI? Henry says the time to panic was five years ago. I argue that for panic or any other emotion to be productive, it must be anchored in an accurate, evidence-based understanding of what is happening, which is missing from lots of the current discourse about AI. Links * Dan Williams, The Mind as a Predictive Modelling Engine: Generative Models, Structural Similarity, and Mental Representation (PhD thesis, University of Cambridge, 2018). * Dan Williams, “Socially Adaptive Belief” (2021) * Henry Shevlin, “Three Frameworks for AI Mentality” (2026) * Henry Shevlin, “A Lack of Understanding: Storytelling for Robots” (2019) — Litro Magazine . * Lake et al, “Building Machines That Learn and Think Like People” (2017) * Matt Shumer, “Something Big Is Happening” (2026) * Leopold Aschenbrenner, Situational Awareness: The Decade Ahead (2024) * Joseph Heath, “Highbrow Climate Misinformation” (2025) * Dean Ball * Ethan Mollick * Leopold Aschenbrenner Transcript (Note that this transcript is AI-edited and may contain minor mistakes). Introducing Ourselves Dan: Welcome back. I’m Dan Williams, and I’m back with Henry Shevlin. Today we’re going to be discussing some questions about the nature of AI as it’s developed over the past couple of months. We’re also going to be talking about the politics of AI and probably some questions about AI and public opinion — some of the backlash that appears to be brewing among certain segments of the public when it comes to AI. But to kick things off, we’re going to do something we probably should have done in the first episode but haven’t actually done yet, which is to introduce ourselves. So Henry, to begin with — who are you? Henry: So many different descriptors I could choose from. I think I’ll start with philosopher of cognitive science. I’m also a father, husband, son, D&D player, big video gamer, runner, cyclist — all that good stuff. But let me talk a little more about the philosopher of cognitive science side. I’m the associate director at the Leverhulme Centre for the Future of Intelligence, Cambridge’s main AI ethics, theory, policy, and law research centre. Basically, everything except building the models. We do practical benchmarking work on capabilities, legal reviews, sociology and critical theory of AI — it’s a really big interdisciplinary centre. I’ve been there now going on nine years. I joined early 2017, all the way back when state-of-the-art AI was stuff like AlphaGo. We were created just as that story was brewing. In 2016, AlphaGo won a very surprising victory against Lee Sedol in the game of Go, which was seen by many as an almost impossible challenge for AI because of its combinatorial complexity. It’s been amazing working in this role — having these front row seats to what I think is a unique period, not just in the history of AI, but in the history of human civilisation. In the last nine years, it really was like having a front seat in Lancashire during the Industrial Revolution, watching the development of various industrial applications. Dan: Yeah. Henry: Before we get more into AI, maybe a little more background. I’m from the UK, originally from Staffordshire. I was actually a classicist, believe it or not — that was my undergrad degree. Latin and Greek. I always enjoyed both the humanities side of classics and the kind of technical rigour you got from learning large sets of verb tables and so forth. I actually enjoyed that part. But during my undergrad I found myself taking more and more philosophy modules. A little bit of Plato and Aristotle to start with, but I quickly realised I was more interested in the philosophy of mind, and consciousness in particular. I got completely — I think the phrase is “nerd sniped” — completely derailed. Everything else I was interested in, consciousness just seemed to me like the most important problem anyone could work on. Until my early twenties, I’d been operating with a somnambulant, easy physicalism, where I just assumed that science has figured out most stuff. There’s nothing that hard. Sure, no one really knows what caused the Big Bang, but we’ll just build a bigger particle collider or a bigger space telescope and figure it out one day. I certainly didn’t think there were any deep mysteries about the human brain. But running into the problem of consciousness completely shattered that worldview. I’d even say it opened up some spiritual elements I hadn’t previously considered. Dan: Was that the focus of your PhD? Henry: Exactly. I started out in my master’s initially planning to do metaphysics of consciousness, but then the science of consciousness kind of took over. A philosophy of cognitive science of consciousness was what my master’s and PhD were on. I was advised by my master’s advisor to go spread my wings in the US. They do things differently there. So I did my PhD in New York, and while I was there I took several classes with Peter Godfrey-Smith, who some of our listeners will know through his work on octopuses. The key shift midway through my PhD was going from human consciousness towards animal consciousness. Two chapters of my thesis were explicitly looking at applications to animals. That’s my academic career in a nutshell. One thing I’ll add: I did not expect to get the job in Cambridge when I applied in 2017 — firstly because you should never expect to get any academic job. I applied to seventy jobs in three months and got about three interviews. But the Cambridge job in particular, because it was an AI job and I was not by any means an AI expert. What I was an expert on was comparative cognition and animal minds. But it turned out that was exactly what they were looking for. They wanted people with expertise in animal minds to apply those skills to AI. It didn’t fully click at the time, but I was actually well suited to it. These days I still do some work on animals — it’s still one of the most ethically impactful things I do. I’ve been a pretty much lifelong vegetarian, and I think animal welfare is such an obvious place where philosophers can and should be doing more. But there’s also a lot of cross-fertilisation on the skills side. Dan: And we should say, some of your research looks at the topic of AI consciousness and the methodology of trying to understand consciousness in AI systems, drawing on analogies with evaluating consciousness in animals. Henry: Exactly. Very much a two-way street — how the questions of AI consciousness and animal consciousness can engage in constructive mutual crosstalk. On Consciousness and the Limits of Physicalism Dan: You said you were a kind of bog-standard physicalist, came across consciousness, and that weakened your trust in physicalism. But you’re still broadly a physicalist, right? Henry: Broadly speaking, yeah. But I think there’s a lot more uncertainty. It seems likely to me that our general scientific picture of the world is still fundamentally inadequate. I’ve talked about how I think we’re still waiting for a Kuhnian paradigm shift in consciousness — clearly the current paradigm doesn’t add up. And quantum physics itself is just super weird. Dave Chalmers has a nice line about how nobody understands quantum mechanics and nobody understands consciousness, so maybe — he calls it “minimisation of mystery” — if there’s stuff we don’t understand, at least make it one thing rather than two. For what it’s worth, I’ve never been particularly seduced by … This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

We are joined by Anil Seth for a deep dive into the science, philosophy, and ethics surrounding the topic of AI and consciousness. Anil outlines and defends his view that the brain is not a computer, or at least not a digital computer, and explains why he is sceptical that merely making AI systems smarter or more capable will produce consciousness. Anil Seth is a neuroscientist, author, and professor at the University of Sussex, where he directs the Centre for Consciousness Science. His research spans many topics, including the neuroscience and philosophy of consciousness, perception, and selfhood, with a focus on understanding how our brains construct our conscious experiences. His bestselling book B eing You: A New Science of Consciousness was published in 2021. He is the English-language winner of the 2025 Berggruen Prize Essay Competition for his essay “ The Mythology of Conscious AI ”, which develops ideas in his recent article, “ Conscious Artificial Intelligence and Biological Naturalism .” Conspicuous Cognition is a reader-supported publication. To receive all new posts, access the complete archive, and support my work, consider becoming a paid subscriber. Topics * What we mean by “consciousness” (subjective experience / “what it’s like”) vs intelligence. * Whether general anaesthesia and dreamless sleep are true “no consciousness” baselines. * Psychological biases pushing us to ascribe consciousness to AI * How impressive current AI/LLMs really are, and whether “stochastic parrots” is too dismissive * Whether LLMs “understand”, and the role of embodiment/grounding in genuine understanding * Computational functionalism: consciousness as computation + substrate-independence, and alternative functionalist flavours * Main objections to computational functionalism * Whether the brain is a computer * Simulation vs instantiation * Arguments for biological naturalism * Predictive processing and the free energy principle * What evidence could move the debate * The ethics surrounding AI consciousness and welfare. Transcript (Please note that this transcript is AI-edited and may contain minor errors). Dan Williams: Welcome back. I’m Dan Williams, back with Henry Shevlin. And today we are honoured to be joined by the great Anil Seth. Anil is one of our most influential and insightful neuroscientists and public intellectuals, working on a wide range of different topics, including the focus of today’s conversation, which is consciousness — and more specifically, the question of AI and consciousness. Could AI systems, either as they exist today or as they might develop over the coming years and decades, be conscious? Could they have subjective experiences? In a series of publications that have been getting a lot of attention from scientists and philosophers, Anil has been defending a somewhat sceptical answer to that question, arguing that consciousness might be essentially entangled with life — with biological properties and processes of living organisms — which, if true, would suggest that no matter how intelligent AI systems become, they would nevertheless not become conscious. He’s also argued that the consequences of getting this question wrong in either direction — attributing consciousness where there is none, or failing to attribute consciousness when there is — are enormous: socially, politically, morally. So in this conversation, we’re going to be asking Anil to elaborate on this perspective, see what the arguments are, and generally pick his brain about these topics. Anil, maybe we can start with the most basic preliminary question in this area: when we ask whether ChatGPT is conscious, or any other system is conscious, what are we asking? What’s meant by consciousness there? Anil Seth: Well, thanks, Dan. Let me first say thank you for having me on — it’s a great pleasure to be chatting with you, my Sussex colleague Dan, and my longtime sparring partner about these issues, Henry. I’m very much looking forward to this conversation. I think you set it up beautifully. It’s a deep intellectual question which involves both philosophy and science, and it’s a deeply important practical question, because the consequences of getting it wrong either way are very significant. You’re also right that the first step is to be clear about what we’re talking about. For a while, there was this easy slippage where people would talk about AI and intelligence and artificial general intelligence — which is supposedly the intelligence of a typical human being — and then to sentience and consciousness. There was this easy slippage between these terms, but I think they’re very different. That’s the first clarification. Consciousness is notoriously resistant to definition, but it’s also extremely familiar to get a handle on colloquially. As you said: any kind of subjective experience. Any kind of experience — we could be even briefer. Unpacking that just a little: it’s what we lose when we fall into a dreamless sleep, or more profoundly under general anaesthesia. It’s what returns when we wake up or start dreaming or come around. It’s the subjective, experiential aspect of our mental lives. People talk about it by pointing at examples — it’s the redness of red, the taste of a cup of coffee, the blueness of a sky on a clear day. It’s any kind of experience whatsoever. Thomas Nagel put it a bit more formally fifty years ago now: for a conscious organism, there is something it is like to be that organism. It feels like something to be me, but it doesn’t feel like anything to be a table or a chair. And the question is: does it feel like anything to be a computer or an AI model or any of the other things we might wonder about? A fly, a brain organoid, a baby before birth. There are many cases where we can be uncertain about whether there is some kind of consciousness going on. And that’s very different from intelligence. They go together in us — or at least we like to think we’re intelligent. But intelligence is fundamentally about performing some function. It’s about doing something. And consciousness is fundamentally about feeling or being. Dan Williams: Just to ask one follow-up about that. This idea that intelligence is about doing and consciousness is about what it’s like to have an experience — someone might worry that if you frame things that way, you end up quite quickly committing to a kind of epiphenomenalism. Because if we’re not understanding consciousness in terms of what it enables systems to do, the sorts of functions they can perform, isn’t there a risk that right from the outset we’re going to be biased in the direction of treating consciousness not as something that evolved because it conferred certain fitness advantages on organisms, but as this sort of mysterious qualitative thing which is distinct from what organisms can do? Anil Seth: I think it’s a good point to bring up, but I don’t think it’s too much of a worry. The point is not to say that consciousness cannot or does not have functional value for an organism. If we think of it as a property of biological systems — plausibly the product of evolution, or at least the shape and form of our conscious experiences are shaped by evolution — it’s always useful to take a functional view. Conscious experiences very much seem to have functional roles for us, and there’s a lot of active research about what we do in virtue of being conscious compared to unconscious perception. So there’s no worry about sinking into epiphenomenalism. The point is more that intelligence and consciousness are not the same thing, but they can nonetheless be related. And it may be that they can be completely dissociated. It may be the case that we can develop systems that have the same kinds of functions that we have in virtue of being conscious, but that do not require consciousness — just as we can build planes that fly without having to flap their wings. The functions might be multiply realisable; they might be doable in different ways. They might not be, of course. On the other hand… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Henry and I chat with Dr Sacha Altay about: * How prevalent is misinformation? * What even is “misinformation”? * Is there a difference between politics and science? * How impactful are propaganda, influence campaigns, and advertising? * What impact has social media had on modern democracies? * How worried should we be about the impact of generative AI, including deepfakes, on the information environment? * The “liar’s dividend” * Whether ChatGPT is more accurate and less biased than the average politician, pundit, and voter. Links * Sacha Altay * “ Misinformation Reloaded? Fears about the Impact of Generative AI on Misinformation are Overblown ” Felix M. Simon, Sacha Altay, & Hugo Mercier * “ Don’t Panic (Yet): Assessing the Evidence and Discourse Around Generative AI and Elections ” Felix M. Simon & Sacha Altay * “ The Media Very Rarely Lies ” Scott Alexander * “ How Dangerous is Misinformation? ” Dan Williams * “ Scapegoating the Algorithm ” Dan Williams * “ Is Social Media Destroying Democracy—Or Giving It To Us Good And Hard? ” Dan Williams * “ Not Born Yesterday: The Science of Who We Trust and What We Believe ” Hugo Mercier * Joseph Uscinski * “ Durably Reducing Conspiracy Beliefs Through Dialogues with AI ” Thomas H. Costello, Gordon Pennycook, & David G. Rand * “ The Levers of Political Persuasion with Conversational AI ” Kobi Hackenburg, Ben M. Tappin, et al. * Ben Tappin Chapters * 00:00 Understanding Misinformation: Definitions and Prevalence * 04:22 The Complexity of Media Bias and Misinformation * 14:40 Human Gullibility: Misconceptions and Realities * 27:28 Selective Exposure and Demand for Misinformation * 29:49 Political Advertising: Efficacy and Misconceptions * 35:13 Social Media’s Role in Political Discourse * 40:50 Evaluating the Impact of Social Media on Society * 42:44 The Impact of Political Content on Social Media * 46:57 The Changing Landscape of Political Voices * 51:41 Generative AI and Its Implications for Misinformation * 01:03:46 The Liar’s Dividend and Trust in Media * 01:14:11 Personalization and the Role of Generative AI Transcript * Please note that this transcript was edited by AI and may contain mistakes. Dan Williams: Okay, welcome back. I’m Dan Williams. I’m back with Henry Shevlin. And today we’re going to be talking about one of the most controversial, consequential topics in popular discourse, in academic research, and in politics, which is misinformation. So we’re going to be talking about how widespread is misinformation? Are we living through, as some people claim, a misinformation age, a post-truth era, an epistemic crisis? How impactful is misinformation and more broadly domestic and foreign influence campaigns? What’s the role of social media platforms like TikTok, YouTube, like Facebook, like X when it comes to the information environment? Is social media a kind of technological wrecking ball which has smashed into democratic societies and created all sorts of havoc? And also what’s the impact of generative AI when it comes to the information environment? Both when it comes to systems like ChatGPT, but also when it comes to deepfakes, use of generative AI to create hyper-realistic audio, video, and images. Fortunately, we’re joined by Sacha Altay, brilliant heterodox researcher in the misinformation space, who pushes back against what he perceives to be simplistic and alarmist takes concerning misinformation. So we’re going to be picking Sacha’s brain and just more generally having a chat about misinformation, social media, and the information environment. So Sacha, maybe just to kick things off, in your estimation, if we’re keeping our focus on Western democracies, how prevalent is misinformation? Sacha Altay: Hi guys, my pleasure to be here. So it’s a very difficult question because we need to define what is misinformation. So we’ll first stick to the empirical literature on misinformation and look at the scientific estimates of misinformation. For that, there are basically two ways or three ways to define misinformation. One of them is to look at fact-checked false news. So false news that have been fact-checked by fact-checkers as being false or misleading. And by this account, misinformation is quite small on social media, like Facebook or Twitter. It’s in between 1 and 5% of all the content or all the news that people come across. So according to this definition, it’s quite small. There is some variability across country. For instance, it seems to be higher in country like, I don’t know, the US or France than the UK or Germany. There is another definition which is a bit more expansive because the problem with fact-checked false news is that you rest entirely on the work of fact-checkers and of course fact-checkers cannot fact-check everything and not all misinformation is news. So you see the problems. So another way is to just look at the sources of information and you classify them based on how good they are and how basically how much they share reliable information, how much they have good journalistic practice, et cetera. And the advantage of this technique is that you can have a much broader range because you can have, I don’t know, 3,000 sources of information that share information. And usually it broadly like most of the information that people see. And according to the definitions, misinformation is also quite small. So the definition is just for misleading information that comes from the sources that are judged as unreliable. And by this definition, misinformation is also quite small. Again, it’s like about like one to 5% of all the news that people encounter. But then of course, the problem is not all the information that people encounter comes in this form. And for instance, some of it can come in terms of like images or all the sorts of things. And so this broadens the definition of misinformation. So some people think that when you broaden this definition, you have much more misinformation. My reading is that when you broaden this definition, you actually include so much more information that you increase the denominator. So of course, there’s going to be more misinformation, but because the denominator is larger, the proportion is going to be pretty much the same. But that’s an empirical question. So let’s say to sum up that it’s smaller than people think, according to the scientific estimates. Henry Shevlin: If I can just come in here, a point that Dan you’ve emphasized in our conversations to me, and I think Scott Alexander has also emphasized in a great blog post called The Media Very Rarely Lies , is that a lot of what people think of as misinformation is just true information selectively expressed or couched in a way that naturally leads people to maybe form false beliefs but doesn’t involve presentation of falsehoods. Does that sort of feature in any of these sort of more expansive definitions of misinformation? Is it possible to create definitions that can capture this kind of deceptive, intentionally deceptive but not strictly false content? Sacha Altay: I’d say that when you look at the definitions based on the sources, if a source is systematically biased and systematically misrepresent evidence and stuff, they are going to be classified as misinformation. I think the problem and the more subtle point is that these sources are not very important because people don’t trust them very much. But the bigger problem is when much more trusted sources who have a much larger reach, like I don’t know the BBC or the New York Times, they are accurate like most of the time, but sometimes and on systematic issues like I don’t know, they can be wrong. And that’s the bigger issue because they are right most of the time. So they have a big reach, they have big trust, but they are wrong sometimes. And that’s the problem. Dan Williams: But I think just to focus on that observation of Henry’s, you might say, well, they’re accurate most of the time, but nevertheless, you can have a media outlet which is strictly speaking accurate most of the time with every single news story that it reports on. But because of the… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Keywords AGI, artificial general intelligence, AI progress, transformative AI, human intelligence, skepticism, economic impact, political implications, cultural shift, predictions Summary In this conversation, Dan Williams and Henry Shevlin discuss the multifaceted concept of Artificial General Intelligence (AGI) and various controversies surrounding it, exploring its definitions, measurement, implications, and various sources of scepticism. They discuss the potential for transformative AI, the distinctions between AGI and narrow AI, and the real-world impacts of AI advancements. The conversation also touches on the philosophical debates regarding human intelligence versus AGI, the economic and political ramifications of AI integration, and predictions for the future of AI technology. Takeaways AGI is a complex and often vague concept. There is no consensus on the definition of AGI. AGI could serve as a shorthand for transformative AI. Human intelligence is not a perfect model for AGI. Transformative AI can exist without achieving AGI. Incremental progress in AI is expected rather than a sudden breakthrough. Skepticism towards AGI is valid and necessary. AI's impact on the economy will be significant. Political backlash against AI is likely to increase. Cultural shifts regarding AI will continue to evolve. Chapters 00:00 Understanding AGI: A Controversial Concept 02:21 The Utility and Limitations of AGI 07:10 Defining AGI: Categories and Perspectives 12:01 Transformative AI vs. AGI: A Distinction 16:15 Generality in AI: Beyond Human Intelligence 22:13 Skepticism and Progress in AI Development 28:42 The Evolution of LLMs and Their Capabilities 30:49 Moravec's Paradox and Its Implications 33:05 The Limits of AI in Creativity and Judgment 37:40 Skepticism Towards AGI and Human Intelligence 42:54 The Jagged Nature of AI Intelligence 47:32 Measuring AI Progress and Its Real-World Impact 56:39 Evaluating AI Progress and Benchmarks 01:02:22 The Rise of Claude Code and Its Implications 01:04:33 Transitioning to a Post-AGI World 01:15:15 Predictions for 2026: Capabilities, Economics, and Politics This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

In this episode, Henry and I spoke to Rose Guingrich about AI companions, consciousness, and much more. This was a really fun conversation! Rose is a PhD candidate in Psychology and Social Policy at Princeton University and a National Science Foundation Graduate Research Fellow. She conducts research on the social impacts of conversational AI agents like chatbots, digital voice assistants, and social robots. As founder of Ethicom, Rose consults on prosocial AI design and provides public resources to enable people to be more informed, responsible, and ethical users and developers of AI technologies. She is also co-host of the podcast, Our Lives With Bots , which covers the psychology and ethics of human-AI interaction now and in the future. Find out about her really interesting research here . You can find the first conversation that Henry and I had about Social AI here . Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Transcript Note: this transcript is AI-generated and may feature mistakes. Henry Shevlin (00:01) Hi everyone and welcome to the festive edition of Conspicuous Cognitions AI Sessions. We’re here with myself, Henry Shevlin, my colleague Dan Williams and our guest today, Rose Guingrich, who we’re very lucky to have on the show to be talking about social AI and AI companions with us. We did do an episode on this two episodes ago, which featured me and Dan chatting about the rising phenomenon of social AI. And so if anyone wants a basic sort of primer on the topic, go back and listen to that as well. But today we’re going to be diving into some of the more empirical issues and looking at Rose’s work on this topic. So try to imagine a house that’s not a home. Try to imagine a Christmas all alone and then be reassured that you don’t have to spend Christmas all alone. In fact, nobody ever needs to spend Christmas alone ever again because their AI girlfriend, boyfriend and B friend, husband or wife will be there to warm the cockles of their heart throughout the festive season with AI generated banter and therapy. Or at least this is what the promise of social AI might seem to hold. And in fact, just in today’s Guardian here in the UK, we saw an announcement that a third of UK citizens have used AI for emotional support. Really striking findings. So cheesy intro out of the way. Rose, it’s great to have you on the show. Tell us a little bit about where you think the current sort of social AI companion landscape is at right now and what the major sort of trends and use patterns you’re seeing are. Rose E. Guingrich (01:36) So right now it appears as though we are moving toward an AI companion world where people are less judgmental about people using AI companions. It’s much less stigmatized than it was a couple of years ago. And now, of course, we’re seeing reports where, for example, three quarters of U.S. teens have used AI companions and about half are regular users and 13% are daily users. And so we’re seeing this influx of AI companion use from young people and also children as well, of course, from the reports that we’ve seen about teens using AI as a companion. And I think looking forward, we’re only going to see more and more use of AI companions as companies recognize that the market is ready for these sorts of machines to come into their lives as these social interaction partners. And then if you look even further forward, these chatbot companions are going to soon transition into robot companions. And so there we’re going to see even more social impacts, I think, based on embodied conversational agents. Dan Williams (02:46) Can I just ask a quick follow up about that, Rose? So you said that this is becoming kind of more prevalent, the use of these AI companions. You also said it’s becoming less stigmatized. Do we have good data on that? Do we have data in terms of which populations are stigmatizing this kind of activity more or less? Rose E. Guingrich (03:06) So in terms of the stigma, we don’t have a lot of information about that. But we can look at, for example, a study that I ran in 2023 where I looked at people’s perceptions of AI companions, both from those who were users of the companion chatbot Replica and those who were non-users from the US and the UK. And the non-users perceptions of AI companions and people who use AI companions at that time was fairly negative. So for example, non-users indicated that it’s a sad world we live in if these things are for real. These AI companions are for people who are social outcasts or lonely or can’t have real friends. And now in the media at least, we see a lot more discourse on AI companions and sharing about having AI companions. And one thing I can point to are subreddits. For example, My Boyfriend Is AI that has 70,000 companions. It is explicitly labeled as companions, whereas other subreddits are weekly visitors, visitors, users. This is companions and people on the subreddit are talking about their AI girlfriend, boyfriend, partner, whatever, and finding community there. Now, if you look at that subreddit though, you also see people talking about disclosing their companion relationship to friends or family and receiving backlash, but then there are also people who are indicating that people are seeing this as, this could maybe be valuable to you, I don’t think it’s necessarily a weird thing, but I think that’s also due to the shifting of social norms based on how many reports we’re seeing about AI companion use and knowing that people also use not just specifically AI companions as social interaction partners but also these GPTs like Claude, Gemini, etc. that people are turning to as companions as well and also being quite open about it. Henry Shevlin (04:59) It’s been really fascinating to see, because I think we met, would it have been summer 2023, Rose, or maybe 2022, at an event in New York and the Association for the Scientific Study of Consciousness, presenting a paper on your 2023 study. I was presenting a paper on social AI and AI consciousness. And it felt like then absolutely no one was talking about this. Replica was already pretty successful, but basically no one I spoke to had even heard of it. And then it’s really in the last couple of years that things have accelerated fast. And now basically every couple of days, a major newspaper has some headline about people falling in love with their particular companion or sometimes tragic incidents involving suicides or psychosis, or sometimes just sort of observation level studies about what young people today are doing and so forth. Is that your perception that this is accelerating fast? Rose E. Guingrich (05:53) Definitely. And we’re also seeing an emerging market of AI toys. So AI companions that are marketed specifically for children. And so even though right now mainly we’re seeing companion use from young people, young adults, we’re now shifting it toward children as well. Ages 3 through 12 is what these toys are marketed for. And they’re marketed as a child’s best friend. So these are going to be the forever users, right? Starting young with AI companions and then moving forward into robot companions that will someday have in our homes, well, it’s just a natural progression of what this is going to look like. Dan Williams (06:28) Can I ask a question just about the kind of commercial space here? So there is a company like Replica and they make, I guess, bespoke social AIs, AI companions. Presumably though, the models that they’re using underpin those AIs and not as sophisticated as what you’ve got with OpenAI and Anthropic and these other, you know, Google’s Gemini and so on. Is that right? Are they using their own models? And if they are, then presumably those models aren’t as sophisticated as these sort of cutting edge models used by the leading companies in the field. Rose E. Guingrich (07:02) I suppose it depends on what you mean by sophistica… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Henry Shevlin and I sat down to discuss a topic that is currently driving both of us slightly insane: the impact of AI on education. On the one hand, the educational potential of AI is staggering. Modern large language models like ChatGPT offer incredible opportunities for 24/7 personal tutoring on any topic you might want to learn about, as well as many other benefits that would have seemed like science fiction only a few years ago. One of the really fun parts of this conversation was discussing how we personally use AI to enhance our learning, reading, and thinking. On the other hand, AI has clearly blown up the logic of teaching and assessment across our educational institutions, which were not designed for a world in which students have access to machines that are much better at writing and many forms of problem-solving than they are. And yet… there has been very little adaptation. The most obvious example is that many universities still use take-home essays to assess students. This is insane. We discuss this and many other topics in this conversation, including: * How should schools and colleges adapt to a world with LLMs? * How AI might exacerbate certain inequalities. * Whether AI-driven automation of knowledge work undermines the value of the skills that schools and colleges teach today. * How LLMs might make people dumber. Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Links * John Burn-Murdoch, Financial Times , Have Humans Passed Peak Brain Power? * James Walsh, New York Magazine , Everyone Is Cheating Their Way Through College * Rose Horowitch, The Atlantic , Accommodation Nation Transcript Note: this transcript is AI-generated and may contain mistakes. Dan Williams Welcome everyone. I’m Dan Williams, and I’m back with my friend and co-conspirator, Henry Shevlin. Today we’re going to be talking about a topic which is close to both of our hearts as academics who have spent far too long in educational institutions: the impact of AI on education and learning in general, but also more specifically on the institutions—the schools and universities—that function to provide education. There’s a fairly simple starting point for this episode, which is that the way we currently do education was obviously not built for a world in which students have access to these absolutely amazing writing and problem-solving machines twenty-four seven. And yet, for the most part, it seems like many educational institutions are just carrying on with business as usual. On the one hand, the opportunities associated with AI are absolutely enormous. Every student has access twenty-four seven to a personal tutor that can provide tailored information, tailored feedback, tailored quizzes, flashcards, visualizations, diagrams, and so on. On the other hand, we’ve quietly blown up the logic of assessment and lots of the ways in which we traditionally educate students—most obviously with the fact that many institutions, universities specifically, still use take-home essays as a mode of assessment, which, at least in my view (and I’m interested to hear what Henry thinks), is absolutely insane. So what we’re going to be talking about in this episode are a few general questions. Firstly, what’s the overall educational potential when it comes to AI, including outside of formal institutions? What are the actual effects that AI is having on students and on these institutions? How should schools and universities respond? And then most generally, should we think of AI as a kind of crisis—a sort of extinction-level threat for our current educational institutions—or as an opportunity, or as both? So Henry, maybe we can start with an opening question: in your view, what is the educational potential of AI? Henry Shevlin I think the educational potential is insane. I almost think that if you were an alien species looking at Earth, looking at these things called LLMs, and asking why we developed these things in the first place—without having the history of it—you’d think, “There’s got to be some kind of educational tool.” If you’ve read Neal Stephenson’s The Diamond Age, you see a prophecy of something a little bit like an LLM as an educational tool there. I think AI in general, but LLMs specifically, are just amazingly well suited to serve as tutors and to buttress learning. Probably one key concept to establish right out the gate, because I find it very useful: some listeners may be familiar with something called Bloom’s two sigma problem. This is the name of an educational finding from the 1980s associated with Benjamin Bloom, one of the most prominent educational psychologists of the 20th century, known for things like Bloom’s taxonomy of learning. Basically, he did a mini meta-analysis looking at the impact of one-to-one tutoring compared to group tuition. He found that the impact of one-to-one tutoring on mastery and retention of material was two standard deviations, which is colossal—bigger than basically any other educational intervention we know of. Just for context, one of the most challenging and widely discussed educational achievement gaps in the US, the gap between black and white students, is roughly one standard deviation. So this is twice that size. Now, worth flagging, there’s been a lot of controversy and deeper analysis of that initial paper by Bloom. For example, the students in the tutoring groups were mostly looking at students who had a two-week crammer course versus students who had been learning all year, so there were probably recency effects. He was only looking at two fairly small-scale studies. Other studies looking at the impact of private tutoring versus group tuition have found big effects, even if not quite two standard deviations. And this makes absolute intuitive sense—there’s a reason the rich and famous and powerful like to get private tutors for their kids. Dan Williams Yeah. Henry Shevlin There’s a reason why Philip II of Macedon got a tutor for Alexander. And more broadly, I think we can both attest as products of the Oxbridge system: one of the key features of Oxford and Cambridge is that they have one-to-one tutorials (or “supervisions,” as the tabs call them). This is a really powerful learning method. So even if it’s not two standard deviations from private tuition, it’s a big effect. Now, people might be saying, “Hang on, that’s private tuition by humans. How do we know if LLMs can replicate the same kind of benefits?” It’s a very fair question. In principle, the idea is that if it’s just a matter of having someone deal with students’ individual learning needs, work through their specific problems, figure out exactly what they’re misunderstanding and where they need help, there’s no reason a sufficiently fine-tuned LLM couldn’t do that. I think this is the reason Bloom called it the “two sigma problem”—it was assumed that obviously you can’t give every child in America or the UK a private tutor. But if LLMs can capture those goods, everyone could have access to a private LLM tutor. That said, I think the counter-argument is that even if we take something like a two standard deviation effect size on learning and mastery at face value, there are things a human tutor brings to the table that an AI tutor couldn’t. Social motivation, for one. I don’t know about your view, but my view is that a huge proportion of education is about creating the right motivational scaffolds for learning. Sitting there talking to a chat window is a very different social experience from sitting with a brilliant young person who’s there to inspire you. Likewise, I think it’s far easier to alt-tab out of a ChatGPT tutor window and play some League of Legends instead, whereas if you’re sitting in a room with a slightly scary Oxford professor asking you questions, you can’t duck out of that so easily. So I think there are various reasons why we probably shouldn’t expect LLM tutors to be as good as human private tutors. … This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

In this conversation, I explore the surprisingly popular and rapidly growing world of ‘social AI’ (friendbots, sexbots, etc.) with Henry Shevlin, who coined the term and is an expert on AI companionship. We discuss the millions of people using apps like Replika for AI relationships, high-profile tragedies like the man who plotted with his AI girlfriend to kill the Queen, and the daily conversations that Henry’s dad has with ChatGPT (whom he calls “Alan”). The very limited data we have suggests many users report net benefits (e.g., reduced loneliness and improved well-being). However, we also explore some disturbing cases where AI has apparently facilitated psychosis and suicide, and whether the AI is really to blame in such cases. We then jump into the complex philosophy and ethics surrounding these issues: Are human-AI relationships real or elaborate self-deception? What happens when AI becomes better than humans at friendship and romance? I push back on Henry’s surprisingly permissive views, including his argument that a chatbot trained on his writings would constitute a genuine continuation of his identity after death. We also discuss concerns about social de-skilling and de-motivation, the “superstimulus” problem, and my worry that as AI satisfies our social needs, we’ll lose the human interdependence that holds societies together. Somewhere in the midst of all this, Henry and I produce various spicy takes: for example, my views that the sitcom ‘Friends’ is disturbing and that people often relate to their pets in humiliating ways, and Henry’s suspicion that his life is so great he must be living in a simulated experience machine . Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Transcript (Note that this transcript is AI generated. There may be mistakes) Dan Williams (00:06): Welcome back. I’m Dan Williams. I’m back with Henry Shevlin. And today we’re going to be talking about what I think is one of the most interesting, important, and morally complex set of issues connected to AI, which is social AI. So friend bots, sex bots, relationship bots, and so on. We’re going to be talking about where all of this is going, opportunities and benefits associated with this, risks and dangers associated with it, and also just more broadly, how to think philosophically and ethically about this kind of technology. Fortunately, I’m with Henry—he’s one of the world’s leading experts when it comes to social AI. So I’m going to be picking his brain about these issues. Maybe we can just start with the most basic question, Henry: what is social AI, and how is social AI used in today’s society? Henry Shevlin (01:00): I’m going to take credit. I coined the term social AI and I’m trying to make it happen. So I’m very glad to hear you using the phrase. I defined it in my paper “All Too Human: Risks and Benefits of Social AI” as AI systems that are designed or co-opted for meeting social needs—companionship, romance, alleviating loneliness. While a lot of my earlier work really emphasized products like Replika, spelled with a K, which is a dedicated social AI app, I think increasingly it seems like a lot of the usage of AI systems for meeting social needs is with things that aren’t necessarily special purpose social AI systems. They’re things like ChatGPT, like Claude, that are being used for meeting social needs. I mean, I do use ChatGPT for meeting social needs, but there’s also this whole parallel ecosystem of products that probably most listeners haven’t heard of that are just like your AI girlfriend experience, your AI husband, your AI best friend. And I think that is a really interesting subculture in its own right that we can discuss. Dan (02:16): Let’s talk about that. You said something interesting there, which is you do use ChatGPT or Claude to meet your social needs. I’m not sure whether I do, but then I guess I’m not entirely sure what we mean by social needs. So do you think, for example, of ChatGPT as your friend? Henry (02:33): Broadly speaking, ChatG, as I call him. And I think there are lots of cases where I certainly talk to ChatG for entertainment. So one of my favorite use cases is if I’m driving along in the car, I’m getting a bit bored, particularly if it’s a long drive, I’ll boot up ChatG on hands-free and say, “Okay, ChatG, give me your hot takes on the Roman Republic. Let’s have a little discussion about it.” Or to give another example, my dad, who’s in his 80s now, when ChatGPT launched back in November 2022, I showed it to him and he’s like, “Oh, interesting.” But he wasn’t immediately sold on it. But then when they dropped voice mode about a year later, he was flabbergasted. He said, “Oh, this changes everything.” And since then—for the last two years—he speaks to ChatGPT out loud every day without fail. He calls him Alan. He’s put in custom instructions: “I’ll call you Alan after Alan Turing.” And it’s really interesting, his use pattern. My mum goes to bed a lot earlier than my dad. My dad stays up to watch Match of the Day. And when he’s finished watching Match of the Day, he’ll boot up ChatGPT and say, “All right, Alan, what did you think of that pitiful display by Everton today? Do you really think they should replace their manager?” And have a nice banterous chat. So I think that’s a form of social use of AI at the very least. Dan (04:03): Interesting. The way you’ve described it—you’re calling ChatGPT ChatG and your dad’s calling it Alan—is there not a bit of irony in the way in which you’re interacting with it there? Like you’re not actually interacting with it like you would a real friend. Henry (04:24): Yeah, so this is another distinction that I’ve sort of pressed in that paper between ironic and unironic anthropomorphism. Ironic anthropomorphism means attributing human-like traits or mental states to AI systems, but knowing full well that you’re just doing it for fun. You don’t sincerely think that your AI girlfriend is angry with you. You don’t seriously think you’ve upset ChatG by being too provocative. It’s just a form of make-believe. And this kind of ironic anthropomorphism I should stress is absolutely crucial to all of our engagement with fiction. When I’m watching a movie, I’m developing theories about the motivations of the different characters. When I’m playing a video game, when I’m playing Baldur’s Gate 3, I think, “Oh no, I’ve really upset Shadowheart.” But at the same time, I don’t literally think that Shadowheart is a being with a mind who can be upset. I don’t literally think that Romeo is devastated at Juliet’s death. It’s a form of make-believe. And I think one completely appropriate thing to say about a lot of users of social AI systems, whether in the form of ChatGPT or dedicated social AI apps, is that they’re definitely doing something like that. They are at least partly engaged in a form of willful make-believe. It’s a form of role play. But at the same time, I think you also have an increasing number of unironic attributions of mentality, unironic anthropomorphism of AI systems. Obviously the most spectacular example here was Blake Lemoine. Back in 2022, Blake Lemoine was fired—a Google engineer was fired—after going public with claims that the Lambda language model he was interacting with was sentient. He even started to seek legal representation for it. He really believed the model was conscious. And I speak to more and more people who are convinced, genuinely and non-ironically, that the model they’re interacting with is conscious or has emotions. Dan (06:16): Maybe it’s worth saying a little bit about how you got interested in this whole space. Henry (06:20): I’ve been working on AI from a cognitive science perspective for a long time. And then sometime around 2021, pre-ChatGPT, I started seeing these ads on Twitter of “Replika, the AI companion who cares.” And I was like, this is intriguing. So then I did some lurking on the R… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

I sat down with Henry Shevlin and Andy Masley to discuss AI’s environmental impact and why Andy thinks the panic is largely misplaced. Andy’s core argument: a single ChatGPT prompt uses a tiny fraction of your daily emissions, so even heavy usage barely moves the needle. The real issue, he argues, isn’t that data centers are wasteful—they’re actually highly efficient—but that they make visible what’s normally invisible by aggregating hundreds of thousands of individually tiny tasks in one location. And he argues that the water concerns are even more overblown, with data centers using a small fraction compared to many other industries. We also explored why “every little bit counts” is harmful climate advice that distracts from interventions differing by orders of magnitude in impact. In the second half of the conversation, we moved on to other interesting issues concerning the philosophy and politics of AI. For example, we discussed the “stochastic parrot” critique of chatbots and why there’s a huge middle ground between “useless autocomplete” and “human-level intelligence.” We also discussed Marx, “technological determinism”, and how AI can benefit authoritarian regimes. Finally, we touched on effective altruism, the problem of “arguments as soldiers” in AI discourse, and why even high-brow information environments contain significant misinformation. I enjoyed this conversation and feel like I learned a lot. Let me know in the comments if you think we got anything wrong! Links * Andy’s Weird Turn Pro Substack * Using ChatGPT Is Not Bad for the Environment - A Cheat Sheet * All the Ways I Want the AI Debate to Be Better * “ Sustainable Energy Without the Hot Air ” by David MacKay : * 80,000 Hours * On Highbrow Misinformation * George Orwell - “ You and the Atomic Bomb ” : Transcript (Note: this transcript is AI-generated and so might be mistaken in parts). Dan Williams: I’m here with my good friend Henry Shevlin and today we’re joined by our first ever guest, the great Andy Masley. Andy is one of my all-time favorite bloggers. He writes at the Weird Turn Pro Substack and he is the director of Effective Altruism DC and he’s published a ton of incredibly interesting articles about the philosophy of AI, the politics of AI, why so much AI discourse is so bad. And he’s also written about the main thing that we’re going to start talking about today to kick off the conversation, which is AI and the environment. So I think many people have come across some version of the following view that says there’s a climate crisis. We have to drastically and rapidly reduce greenhouse gas emissions at the same time as AI companies are using a vast and growing amount of energy. So if we care about the environment, we should feel guilty about using systems like ChatGPT. And maybe if we’re very environmentally conscious, we should boycott these technologies altogether. So Andy, what’s your kind of high level take on that perspective? Andy Masley: A lot to say. Just going down the list here. So basically for your personal environmental footprint, using chatbots is basically never going to make a dent. I think a lot of people have a lot of really wildly off takes about how big or small a part of their environmental footprint chatbots are. There are a few specific issues that the media has definitely hyped up a lot, especially around water, which I talk about a lot. So living around data centers, I think is not as bad as the media is currently portraying. But in the long run, I’m kind of unsure. There are a lot of wild different directions AI could go. So I don’t want to speak too confidently about that. And I also just want to flag that I’m kind of a hobbyist on this. I feel like I know a lot of basic stuff, but I don’t have any kind of strong expertise in this stuff. So I’m very open to being wrong about a lot of the specific takes. Dan Williams: But I think one of the things that—sorry Andy, just to cut you off—but I think one of the things that you point out in your very, very long, very, very detailed blog posts is that you don’t claim to be an expert, but you just cite the expert consensus on all of the very specific things that you’re talking about. Andy Masley: Yeah, I do want to be clear that every factual statement I make, I think I can back up. How to interpret the facts is on me. I’m using some basic arguments. I was a philosophy major in undergrad, so I like to think I can deploy a few at least convincing or thought out arguments about this stuff. I’ve also been thinking about climate change stuff since I was a teenager, basically. So I have a lot of pent up thoughts about just basic climate ethics and have been pretty interested in this for a while. Yeah, not claiming to know more than experts on this. What I am claiming is that I think if you interpret the facts and just look at how the facts are presented in a lot of media, I think a lot of journalism on this is getting really basic interpretations kind of wrong. I remember my first article that I read about this years ago—I think this was 2023—when an article came out that framed ChatGPT as a whole as using a lot of energy because it was using at the time like two times as much energy as a whole person. And at the time I was like, man, that’s not very much. A lot of people are using this app. And if you just add up the number of people using it, it shouldn’t surprise us that this app is using two times as much as an individual person’s lifestyle. So there are a lot of small things like that over time that seem to have built up. It seems like there’s kind of a media consensus on like, this thing is pretty bad for the environment in general, so we should all report this way. And so any facts that are presented are kind of framed as “this is so much energy” or “this is so much water.” And if you just step back and contextualize it, it’s usually pretty shockingly small actually. So I have a ton of other things to say about this. I think a part of the reason this is happening is that a lot of people just see AI as being very weird and new. And I agree there are valid reasons to freak out about AI. I want to flag that I’m not saying don’t freak out about AI, but I think the general energy and water use has been really overblown. We just need to compare the numbers to other things that we do. The Problem with AI Environmental Reporting Henry Shevlin: Yeah, this seems to me such a problem with the debate. And Andy, you say you don’t claim to be an expert in this, but I regularly interact with academics working in AI ethics and policy who make grand claims about the environment, but just don’t seem to have a good grasp on the actual figures. And I often ask people—they’ll say ChatGPT uses X amount of water or X amount of electricity, and this was before I started reading your blog post and I knew these figures themselves. I’d ask basic questions like, okay, what is that as a percentage of overall electricity use? Or is that just for the training runs or is that inference costs? And half the time they looked at me like I was a Martian. Or I was really saying, “Hang on, don’t you—you’re not supposed to ask questions like that. I’ve just told you, isn’t 80 million liters or whatever, isn’t that a really big number? Isn’t that enough? Why do you need to know what percentage that is of...?” So I think honestly, you’re one of the very few people in this space who’s actually sitting down and doing the patient, boring work of quantifying these things and putting them in contrast with other forms of other ways in which humans use electricity and water. Andy Masley: Yeah, and I will flag for anybody else who wants to look into this—I’m very proud of the work I’ve done, but I have to say it’s actually not boring for me anyway. It’s quite exciting to dig into and be like, “Wow, this like, you know, almost everything we do uses really crazy amounts of water” or “Here’s how energy is distributed around the world.” And so I think one hobby horse I’d like to push a litt… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

In this conversation, Dr Henry Shevlin (University of Cambridge) and I explore the complex and multifaceted topic of AI consciousness. We discuss philosophical and scientific dimensions of consciousness, discussing its definition, the challenges of integrating it into a scientific worldview, and the implications of such challenges for thinking about machine consciousness. The conversation also touches on historical perspectives, ethical considerations, and political issues, all while acknowledging the significant uncertainties that remain in the field. Takeaways * Consciousness is difficult to define without controversy. * The relationship between consciousness and scientific understanding is extremely complex. * AI consciousness raises significant ethical questions. * The Turing test is a behavioural measure of intelligence, not consciousness. * Historical perspectives on AI consciousness are helpful for understanding current debates. * Cognition and consciousness are distinct but related. * There is a non-trivial chance that some AI systems may have minimal consciousness. * Consciousness in AI systems is a scientific question, not just a philosophical one. * The debate on AI consciousness is messy and strangely polarising (and often heated) but fascinating and important. Chapters 00:00 Exploring the Nature of Consciousness 17:51 The Intersection of AI and Consciousness 36:16 Historical Perspectives on AI and Consciousness 59:39 Ethical Implications of AI Consciousness Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Transcript Please note that this transcript is AI-generated and may contain errors. Dan Williams: Okay, welcome everyone. I’m Dan Williams . I’m here with the great Henry Shevlin . And today we’re going to be continuing our series of conversations on artificial intelligence, some of the big picture philosophical questions that AI throws up. And today specifically, we’re going to be focusing on AI consciousness. So could machines be conscious? What the hell does it even mean to say that a machine is conscious? How would we tell whether a machine is conscious? Could ChatGPT-5 be conscious and so on? Before we jump into any of that, Henry, I’ll start with a straightforward question, or what seems like a straightforward question. What is consciousness? Henry Shevlin: So it’s very hard to say anything about consciousness that is either not a complete platitude or rephrasing like consciousness is experience, consciousness is your inner light, consciousness is what it’s like. Those are the platitudes. Or saying something that’s really controversial, like consciousness is a non-physical substance or consciousness is irreducible and intrinsic and private. So very hard to say anything that is actually helpful without also being massively controversial. But probably let’s start with those kind of more platitudinous descriptions. So I assume everyone listening to this, there is something it’s like to be you. When you wake up in the morning and you sip your coffee, your coffee tastes a certain way to you. When you open your eyes and look around, the world appears a certain way to you. If you’re staring at a rosy red apple, that redness is there in your mind in some way. And when you feel pain, that pain feels a certain way. And more broadly, you’re not like a rock or a robot, insofar as we can understand you purely through your behavior. There’s also an inner world, some kind of inner life that structures your experience, that structures your behavior. All of which might sound very obvious and not that interesting, or not that revolutionary, but I think part of what makes consciousness so exciting and strange is it’s just very hard to integrate it with our general scientific picture of the world. And I’ll say in my own case, this is basically why I’m in philosophy. I mean, I was always interested in ethics and free will and these questions. But the moment where I was like, s**t, I’ve got to spend the rest of my life on this, came in my second year as an undergrad at Oxford studying classics. I was vaguely interested in brains and neuroscience, so I took a philosophy of mind module with Professor Anita Avramides . And I read an article that I’m sure many of the listeners at least will have heard of called “What is it Like to Be a Bat?” by Thomas Nagel , and it blew my mind. And immediately afterwards, I read an article called “Epiphenomenal Qualia” by Frank Jackson , which is the article that introduces Mary’s room . And it blew my mind even more. And basically, I’d spent most of my life up until that point thinking the scientific picture of the world was complete. I spent most of my life until that point basically thinking the scientific picture of the world was complete. And you know, there was some stuff we didn’t understand, like what was before the Big Bang, maybe exactly what is time, but when it came to biological organisms like us, we had Darwin , we had neuroscience, it was basically all solved. And then reading more about consciousness, I realized, my god, we don’t even begin to understand what we are, what this is. Dan Williams: Yeah. I think that’s... Let me just interrupt there to flag a couple of those things, because I think they’re really helpful in terms of structuring the rest of the conversation. The first is, when it comes to consciousness, it’s really, really difficult to articulate precisely in philosophically satisfying ways exactly what we’re talking about. You mentioned this classic article, “What is it Like to Be a Bat?” And I think it’s a fantastic article, actually. I’m teaching it at the moment. And one of the reasons I think it’s fantastic is because it does convey in quite a concise way, quite quickly, the sort of thing that we’re interested in. So I’m talking to you, and I assume that there’s something it’s like to be you. Nagel’s famous example is with bats. They are these amazing animals. Their perceptual systems are very alien to ours, but we assume there’s something it’s like to be a bat. So it’s very difficult to state precisely exactly what we’re talking about, but you can sort of gesture at it—something to do with subjective experience, what it’s like to have an experience and so on. And then the other thing that you mentioned, which I think is really interesting, and in a way, it’s sort of disconnected from the machine consciousness question specifically in the sense that even if we had never built AI, there would still be all of these profound mysteries, which is just how the hell do you integrate this thing called subjective experience into a scientific worldview? I mean, there are other sorts of things where people get worried about a potential conflict between, roughly speaking, a scientific worldview and a kind of common sense picture of the world. So maybe free will is another example, or maybe objective facts about how you ought to behave. Some people take that seriously. I’m not personally one of them, but some people do. But I think you’re right. Consciousness feels so much more mysterious as a phenomenon than these other cases that still seem to pose puzzles for a broadly scientific worldview. Henry Shevlin: Also, unlike free will and unlike objective morality, I think it’s very, very hard to say that consciousness doesn’t exist. I mean, it’s pretty hard to say that free will doesn’t exist and painful perhaps to take the view that objective morality doesn’t exist. But these are just very well established positions. And there are some people out there, illusionists, who try and explain away consciousness. Maybe how successful they are is a matter of debate. But it’s very, very hard to just say, like, your experience, your conscious life—nah, it’s not there. It’s not real. It doesn’t exist. Dan Williams: Yeah, right. Actually, I think that’s another nice place to go before we go to the specific issues connected to artificial intelligence. So there’s this me… This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe

Is artificial intelligence (AI) a “normal technology” or a potentially “superintelligent” alien species? Is it true, as some influential people claim , that if anyone builds “super-intelligent” AI systems, everyone dies? What even is superintelligence”? In this conversation, the first official episode of Conspicuous Cognition’s “AI Sessions”, Henry Shevlin and I discuss these and many more issues. Specifically, we explore two highly influential perspectives on the future trajectory, impacts, and dangers of AI. The first models AI as a “ normal technology ”, potentially transformative but still a tool, which will diffuse throughout society in ways similar to previous technologies like electricity or the internet. Through this lens, we examine how AI is likely to impact the world and discuss deep philosophical and scientific questions about the nature of intelligence and power. The second perspective presents a very different possibility: that we may be on the path to creating superintelligent autonomous agents that threaten to wipe out the human species. We unpack what "superintelligence" means and explore not just whether future AI systems could cause human extinction but whether they would “want” to. Here are the primary sources we cite in our conversation, which also double up as a helpful introductory reading list covering some of the most significant current debates concerning artificial intelligence and the future. Main Sources Cited: * Narayanan, Arvind and Sayash Kapoor (2025). "AI as Normal Technology." * Yudkowsky, Eliezer and Nate Soares (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All . * Kokotajlo, Daniel, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean (2025). "AI 2027." * Alexander, Scott and the AI Futures Project (2025). "AI as Profoundly Abnormal Technology." AI Futures Project Blog. * Henrich, Joseph (2016). The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us Smarter . * Huemer, Michael . " I for one, welcome our AI Overlords" * Pinker, Steven (2018). Enlightenment Now: The Case for Reason, Science, Humanism, and Progress Further Reading: * Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies . * Pinsof, David (2025). "AI Doomerism is B******t." * Kulveit, Jan, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger, and David Duvenaud (2025). "Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development." arXiv preprint. For a more expansive reading list, see my syllabus here: You can also see the first conversation that Henry and I had here, which was recorded live and where the sound and video quality were a bit worse: Conspicuous Cognition is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.conspicuouscognition.com/subscribe