The Deepfake Dialogues

In May 2025, a man killed in a road rage shooting in Arizona appeared to address the court at the sentencing of the man who shot him. The video was AI-generated, created by his sister, Stacey Wales, to give him a voice in a process that had reduced him to evidence. Stacey joins The Deepfake Dialogues to share who her brother Christopher Pelkey was, how her family built the avatar, and why she believes it was worth the risk. In this episode: * Who Chris was, and how the courtroom left him faceless and voiceless * Why Stacey wrote an impact statement in Chris’s voice rather than her own * How the avatar was built from one photo, a single recorded interview and her script * The ethical choices behind it, including disclosing it as AI and stopping short of having Chris forgive the shooter * How the judge and defence responded in court * The public backlash, and whether anyone can speak for the dead This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Generative AI is making scams easier to create, harder to detect, and possible to operate at unprecedented scale. Voice clones can impersonate family members in distress, AI-generated personas can sustain romance scams, and deepfake videos can make trusted public figures appear to promote fraudulent investments. Ben Winters, Director of AI and Privacy at the Consumer Federation of America, joins The Deepfake Dialogues to discuss how AI is changing the scam landscape—and why treating fraud as a problem of individual gullibility misses the companies and technologies that make these scams possible. We discuss: * How voice cloning is transforming “grandparent” and impersonation scams * How generative AI is making romance scams more convincing and scalable * Deepfake investment scams and fake celebrity endorsements * Why older adults are targeted—and why vulnerability cannot be reduced to a lack of tech literacy * Ben’s concept of the “scam stack,” from data brokers and AI companies to social media platforms and financial services * Why Meta and other platforms should face greater scrutiny for the scams they help distribute * The Consumer Federation of America’s lawsuit against Meta over scam advertisements * What stronger safeguards around voice cloning, payments, data brokers, and scam reporting could look like * Why scam prevention education matters, but cannot substitute for systemic accountability This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

A voice is more than a means of communication. It carries our age, identity, personality, and history. For people who cannot speak or are losing their speech, synthetic voice technology can preserve or create something deeply personal: a voice that sounds like their own. Professor Rupal Patel, a speech scientist at Northeastern University and founder of VocaliD, joins me to trace the evolution of synthetic speech from generic computer voices to highly realistic voice clones. We explore how VocaliD combined the identity carried in a recipient’s vocalisations with the clarity of a donated voice, and how the same technology built to restore voices revealed new risks around impersonation, fraud, and consent. We discuss: * Why children and adults who use communication devices were historically given the same generic voices * How VocaliD created personalised voices for people who had never been able to speak clearly * The Human Voicebank Initiative and the people who donated their voices * What it means for someone with ALS or throat cancer to hear a voice that sounds like their own * How synthetic voices were able to defeat bank voice-authentication systems * The challenge of safeguarding a technology that can be used for both accessibility and impersonation * Whether someone’s voice should be recreated after their death * How changes in the voice could serve as early indicators of Parkinson’s, ALS, and other health conditions This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

A politician says something inconvenient on live television, in front of a crowd, with a sign-language interpreter beside him. He later claims the footage was made with AI. Nothing was fabricated. The denial alone was enough to muddy the story. Sam Gregory is Executive Director of WITNESS, the human rights organisation founded in 1992 to help people document what is happening to them and ensure that what they capture is believed. That second task has become increasingly difficult. Sam has worked on synthetic media since 2018, when WITNESS held some of the first global consultations on deepfakes with journalists and frontline human rights defenders under the banner “prepare, don’t panic.” Those communities warned that deepfakes would not only enable convincing fabrications. They would also give powerful people a new excuse to dismiss real evidence, intensifying a problem that human rights defenders had faced for decades. This conversation explores how we can continue to establish the truth when authentic footage can so easily be denied. We discuss: * Why fabrication and denial can now occur around the same event, and sometimes the same image * Detection equity: why accent, skin tone and camera quality can affect how reliably a deepfake is identified * What the Deepfakes Rapid Response Force does when contested media surfaces during a crisis * Why courts are approaching a reckoning with “prove it was me” defences * Where provenance standards such as C2PA genuinely help, and how they could be weaponised * Why “spot the six fingers” was always the wrong lesson to teach This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Anyone’s face, voice, or name can now be synthesised in seconds. So how do you protect a person’s likeness when the unauthorised version can be created from scratch, no manipulation required? Dan Neely, co-founder and CEO of Vermillio and one of TIME’s 100 Most Influential People in AI, is building the guardrails for the generative internet. Vermillio’s TraceID technology creates a neural fingerprint of a person’s likeness and scours the web to detect when their face, voice, or name has been used to generate AI content, whether that’s a deepfake designed to deceive or an unauthorised AI track. Beyond detection, the company is building the infrastructure for authorised AI licensing, so creators can control and even monetise how their likeness is used. We discuss: * How TraceID fingerprints a likeness and matches it against a trillion pieces of AI content a month * Why attribution goes further than simply asking “is this AI or not?” * Why likeness data is “strategic gold” creators shouldn’t hand over to big tech * Where the line sits between an unauthorised deepfake and legitimate parody or fan content * What happens when a model trained in a country with no IP laws uses your face This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

While Congress stalls, state legislatures have quietly become the front line in the fight against deepfakes. This episode features Ilana Beller, who leads Public Citizen’s state legislative work on AI and has helped move model legislation through more than 40 states, focused on the two areas moving fastest: non-consensual intimate deepfakes and election deepfakes. We discuss: * What the Take It Down Act accomplished and left on the table, including the missing private right of action * How intimate deepfake laws work, and why circulation is easier to prosecute than creation * What separates a strong law from a weak one: enforcement and a high bar for consent * App-level bans on nudification tools and barring platforms from advertising them * How election deepfake laws use labeling and pre-election windows while carving out satire and parody * Why these bills draw rare bipartisan support, and the federal moratorium threatening to preempt them This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

What happens when a local councillor becomes the target of a deepfake smear campaign? In March 2026, a deepfake video of Cllr Pete Kilbane, Labour Deputy Leader of City of York Council, began circulating online. In the clip, his digitally fabricated likeness appeared to hand wads of cash to balaclava-clad men, as if paying them to remove flags. The video was fake. The campaign of intimidation surrounding it was not. In this episode, Pete walks through what happened in York after St George’s Cross and Union flags began appearing on lampposts across the city as part of Operation Raise the Colours. Hope Not Hate and other public reporting have linked the movement to figures associated with Tommy Robinson and the far-right party Britain First. We discuss: * How the flag campaign played out in York and why the politics behind it matter * The deepfake video of Pete, how his colleagues first found out, and why “it’s just satire” does not capture the harm * The threats, doxxing, and intimidation faced by council staff and contractors * What the Online Safety Act can and cannot do when fake content spreads through closed groups * Pete’s concerns about the movement’s links to MAGA and the American far right * How synthetic media is being weaponised to erode trust in local democracy * How the council responded to this incident This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Content note: This episode discusses AI-generated child sexual abuse material, non-consensual intimate imagery, sextortion, and sexual abuse. AI-powered “nudify” apps are increasingly being used by students to create deepfake sexual images of classmates, but most schools still do not have clear policies for prevention, reporting, discipline, or victim support. Riana Pfefferkorn, a policy fellow at Stanford HAI, joins DeepFake Dialogues to discuss her policy brief, Addressing AI-Generated Child Sexual Abuse Material: Opportunities for Educational Policy , and what schools, lawmakers, platforms, and communities need to understand about how this is actually playing out. Read the policy brief here: https://hai.stanford.edu/policy/addressing-ai-generated-child-sexual-abuse-material-opportunities-for-educational-policy We discuss: * The difference between fully synthetic AI-CSAM and images depicting real, identifiable children * The lasting harm to victims, and why it often doesn’t end at graduation * How unprepared school responses can make that harm worse * What schools should have in place before an incident happens * Whether student offenders should face criminal punishment or behavioural intervention * The role of “nudify” app bans, platform liability, and payment processors * Grok, xAI, and what happens when mainstream AI tools enable sexualised image abuse * Why prevention has to include consent, dignity, privacy, and digital citizenship This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

AI-generated wildlife videos are flooding social media, and the consequences go far beyond misleading content. Dana Wilson of Wildlife SOS, an Indian nonprofit running 13 rescue centers and emergency hotlines across the country, explains how synthetic animal videos are distorting public understanding of wildlife behavior, putting both people and animals at risk. We discuss: * How Wildlife SOS’s real rescue footage has been stolen and spliced with AI-generated endings * The public safety threat of videos that make dangerous predators look friendly and approachable * AI-generated attack videos stoking fear and calls for culling in communities that coexist with wildlife * Why even “harmless” AI-generated animal content erodes trust in genuine conservation storytelling * How deepfake technology could make these problems far worse as synthetic video becomes harder to detect * How Wildlife SOS is setting internal AI policies to protect the integrity of their work * What creators and platforms should consider before posting AI-generated wildlife content Learn more about Wildlife SOS at wildlifesos.org and watch real rescue videos on their YouTube channel. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

In this episode, I’m joined by Erin D. Dumbacher, Stanton Nuclear Security Senior Fellow at the Council on Foreign Relations, to discuss her recent Foreign Affairs article “How Deepfakes Could Lead to Doomsday.” We explore how deepfakes and AI hallucinations could infiltrate the systems designed to prevent nuclear catastrophe — and why the story of Stanislav Petrov, a Soviet officer who in 1983 chose not to trust his screen when it told him American nukes were incoming, matters more now than ever. We also dig into the Pentagon’s rush to integrate AI across the Department of Defense, why AI trained on nuclear threats has almost no real data to learn from, and why the most dangerous scenario isn’t a machine launching a weapon — it’s bad information cascading through a decision-making chain where the president has minutes, not hours, to act. We discuss: * Deepfakes of Zelensky and Putin during the Ukraine war — and what could have gone differently * The Pentagon’s GenAI.mil rollout and the risks of moving too fast * AI hallucinations vs. social media deepfakes — two distinct threats to nuclear stability * The data problem: training AI when we’ve only had two nuclear attacks in history * How misinformation could trigger cascading crises under compressed nuclear timelines * Reforms to U.S. nuclear launch authority in an era where information can’t be taken at face value This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

We spend a lot of time on this podcast talking about what AI does to information: deepfakes, synthetic media, manipulated reality. But there’s a physical cost to all of it that rarely gets discussed: water. Every AI-generated image, every deepfake video, every ChatGPT query runs on servers that need millions of litres of fresh water to stay cool. And those data centers are increasingly being built in regions already facing drought and water stress. Dr. Kevin Grecksch is Associate Professor of Water and Environmental Governance at the University of Oxford, where he directs the MSc in Water Science, Policy and Management. His research focuses on who gets water, who controls it, and what happens when there isn’t enough. In this episode, he helps us understand the hidden environmental footprint of the AI tools we discuss every week on this show, and what smarter governance could look like. We discuss: * The physical infrastructure behind AI-generated content, and why it needs so much water * Why data centers are being built in drought-prone regions like Spain and the American Southwest * What “water positive by 2030” actually means (Kevin doesn’t hold back) * The irony of AI as climate solution while worsening water stress * Integrative planning: circular systems, waste heat reuse, and localized solutions * What those of us using generative AI should keep in mind This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

What happens when a dictator dies but deepfake technology keeps him in power? Matthew Fecteau, a US Army Information Operations Officer and PhD researcher at King’s College London, recently coined the term “synthetic sovereign” in The Diplomat. His hypothesis: North Korea could use deepfake technology to keep Kim Jong-un ruling from beyond the grave. We discuss: * Why North Korea may be the only country that could pull this off * Whether enough footage of Kim already exists to make this technically feasible * Who would orchestrate a digital puppet regime and their incentives * The “liar’s dividend” and how Pyongyang could dismiss real evidence of Kim’s death as foreign disinformation * What happens when a synthetic Kim appears to authorize a nuclear strike * Why this scenario might paradoxically serve Western stability interests Read Matthew’s article here This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

What happens when a single AI-faked text “receipt” can land you in jail? Melissa Sims, a Florida nurse, says she was arrested for allegedly violating a no-contact order after her ex-boyfriend used an AI-enabled fake text generator to create convincing “evidence.” Despite phone records later indicating no text was sent or received, prosecutors pursued the case for months—costing her nursing license, over $300,000, and months of her life. The no-contact charge was ultimately dropped, and she was later found not guilty on the original domestic violence charge. Now she’s advocating for Melissa’s Law: legislation that would require metadata verification of digital evidence before arrests and create penalties for fabricating AI-generated/synthetic evidence. We discuss: * How AI-enabled text generators can create realistic “receipts” * Why police and prosecutors often don’t authenticate digital evidence * The technical detail that helped expose the fabrication (SMS vs iMessage) * What Melissa’s Law would change in criminal evidence handling * Why laws haven’t kept pace with synthetic media * Why her ex has faced no consequences This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Generative AI is not only about viral deepfakes. It also enables synthetic content at scale, including text, audio, and images that can be produced quickly and micro-targeted. Ales Cap, a UCL PhD researcher studying AI and democracy, joins me to discuss what the evidence suggests about deepfakes in elections so far, why measuring impact is difficult without platform data, and why the biggest risks may be gradual through erosion of shared facts and trust. We also explore ideas for safeguarding electoral integrity and the promise of AI-enabled mass deliberation. We cover: * Deepfakes and election integrity * Synthetic media beyond video * The “infinite content machine” and micro-targeting * Safeguarding elections and mass deliberation This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Deepfake “nudify” apps are exploding online, but who’s building them, and how do they operate in plain sight? Kolina Koltai, an open-source intelligence investigator at Bellingcat, traces the infrastructure that keeps these services profitable: shell companies, copycat domains, redirect sites, and payment workarounds, and how investigations can pressure operators into shutting down. We discuss: * The digital paper trail nudify operators leave behind * How these services evade app stores, payment processors, and hosting platforms * Why targeting users misses the bigger problem: site owners and profit incentives * The idea of “friction,” and why making abuse harder matters even if you can’t stop it completely * The cultural shift needed to prevent this from becoming normalized This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

In 1969, President Nixon had a speech ready in case Apollo 11 ended in tragedy. Fifty years later, artists Francesca Panetta and Halsey Burgund brought that speech to life using deepfake video and early AI voice cloning, showing Nixon delivering words he never actually spoke. Their Emmy Award-winning interactive documentary In Event of Moon Disaster is both a haunting alternate history and a media literacy experiment about how easily the past can be rewritten. We go behind the scenes on the tech, the ethics, and what responsible synthetic media can look like. We discuss: * How the project came together at Harvard and MIT * Why Apollo 11 was the perfect vehicle for this message * The ethics of demonstrating deepfake danger without becoming misinformation * Where deepfake detection and media literacy are headed, and their limits See the project: moondisaster.org This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Deepfakes are supposed to destroy our ability to trust video evidence—but what if that fear rests on flawed assumptions about how we gain knowledge in the first place? Philosopher Dr. Joshua Habgood-Coote (University of Leeds) argues that video has never been a simple window into reality. It’s always been interpersonal knowledge, dependent on trusting the people and institutions behind what was filmed, edited, and shared. In his 2023 paper “Deepfakes and the Epistemic Apocalypse,” he challenges the panic and draws surprising parallels to Victorian spirit photography. We discuss: * Why “seeing is believing” was never quite true for video * How 19th-century newspapers handled widespread photo faking through social norms, not tech * Why the real harms of deepfakes (including non-consensual intimate imagery) are about power and harassment, not just deception * What a society that’s adapted to deepfakes might actually look like This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Content note: This episode discusses non-consensual intimate imagery and sexual abuse. Noelle Martin was 18 when she discovered doctored intimate images of herself across dozens of porn sites. When she spoke out, perpetrators escalated to deepfake videos. A decade later, she’s an Australian lawyer fighting for legal reform—while still being targeted. We discuss: * Why takedowns are often futile, and how perpetrators escalate * The gap between criminalizing distribution vs. creation * What Australia’s 2024 reforms do and don’t solve * Why this requires a coordinated global response * Noelle’s advice for others experiencing NCII abuse This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com

Smartphone footage has transformed how we document war crimes. But what happens when anyone can claim “it’s a deepfake”? Professor Yvonne McDermott-Rees (Swansea University) leads the TRUE Project, investigating how AI-generated media affects trust in user-generated evidence of human rights violations. In this episode, we explore: • Whether the “liar’s dividend” is actually showing up in courtrooms • What mock jury experiments reveal about how people evaluate digital evidence • Why the biggest risk may be real footage being dismissed—not fakes entering court Learn more: trueproject.co.uk This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thedeepfakedialogues.substack.com