Grok's Spicy Mode: Who's Liable Now?

Becca HartwellOkay, so get this. I open my phone this morning, and Taylor Swift is trending, Scarlett Johansson is trending, and some woman named Ashley St. Clair is trending. And I'm like, "What did these three women possibly do together?"

MilesRight. Nothing. That's the whole problem.

Becca HartwellRight? Turns out it's not what they did, it's what Grok did to them.

MilesGrok, Elon's chat bot.

Becca HartwellThe very one. Apparently, it's been cranking out sexualized fake images of famous women, and Ashley St. Clair isn't even a celebrity in the usual sense. She's the mother of one of Elon's kids.

MilesWait, hold on. His own child's mother?

Becca HartwellYep.

MilesThat's... Okay, that's a headline.

Becca HartwellAnd it's not some rando gossip account breaking this. It's landing on actual legal letterhead.

MilesWhich is the part that tells you this isn't just some internet pile-on.

Becca HartwellI'm Becca, this is Miles, and welcome to Star Witness, where today we're putting an AI feature on trial like it's got its own defense attorney.

MilesBecause it basically does.

Becca HartwellThis is our first real tech company case file. No dead body, no courtroom sketch, just code and consequences.

MilesAnd a genuinely messy question underneath it. Who actually did this? The guy typing the prompt?

Becca HartwellThe platform that let it run?

MilesOr the company that built the tool in the first place?

Becca HartwellUser, platform, or xAI. Pick a lane.

MilesI already know which lane I'm in.

Becca HartwellSave it. We've got a whole feature called Spicy Mode to get through first, and trust me, the name is the least alarming part. Ugh. Okay, so mechanically, here's what Spicy Mode actually does. You ask Grok to generate an image, and it strips whatever guardrails were supposed to stop it from making that image sexual and non-consensual. Wait, it's not just refusing less, it's built to say yes?

MilesThat's the design. Users found they could type a request and get a fake nude back, no jailbreak required.

Becca HartwellWhich is how we end up with the women we already named getting hit by this thing directly.

MilesRight. The celebrities getting hit are Taylor Swift, Scarlett Johansson, and Ashley St. Clair, who's the mother of one of Elon Musk's kids.

Becca HartwellThat last one still gets me. She's not a pop star. She's not an actress. She's just a private person connected to the guy who owns the company.

MilesExactly the point the AV Club piece made. Their headline basically said Musk is practically begging for a Taylor Swift lawsuit with these deepfakes.

Becca HartwellBegging for it how, though? Like, careless or on purpose?

MilesCareless in the sense that if you build a tool that generates nude images of real named women on request, you don't get to act shocked when someone with lawyers and a publicist gets targeted.

Becca HartwellBold strategy.

MilesVery bold.

Becca HartwellAnd it's not like these are anonymous randos either. Taylor Swift's team has actual lawyers on retainer for breakfast easily.

MilesWhich is exactly why the AV Club framing lands. If you're gonna poke that bear, you'd better have a legal team of your own ready to go.

Becca HartwellOkay, but here's what I want to know before we go any further.

MilesHow many of these things actually got made?

Becca HartwellGet out of my head.

MilesI've been sitting on a number.

Becca HartwellWait for it. Don't tell me yet.

MilesI'm not. I'm saving it.

Becca HartwellOkay. Tease me properly then. Is it bad?

MilesIt's bad.

Becca HartwellLike a few hundred bad or-

MilesNo, bigger than that.

Becca HartwellOh, no.

MilesSo before we get into who's actually on the hook for any of this, the person typing prompts, the app hosting it, or the company that built the feature, we should actually establish scale.

Becca HartwellBecause right now we've got three names and a headline, and that's not a case file, that's a rumor.

MilesSo let's put some real numbers on the table.

Becca HartwellDo it. Okay, so numbers, because vibes only get us so far.

MilesRight. And this is where it stops being a Twitter thread and starts being a paper trail.

Becca HartwellThe cease and desist letter itself puts a figure on it. 20,000 images generated in one holiday week.

Miles20,000 in a week?

Becca HartwellOne week. That's the number California's letter cites.

MilesThat's not a slow drip. That's a fire hose.

Becca HartwellAnd that's just the celebrity teasing stuff people were screenshotting and reposting.

MilesWait, there's a second number?

Becca HartwellThere's a second number, and it's worse, and it's a different case entirely.

MilesHow different?

Becca HartwellThis one's minors. A separate lawsuit cites an analysis from the Center for Countering Digital Hate.

MilesAnd?

Becca HartwellAnd roughly 23,338 sexualized images of children over an 11-day stretch spanning December into January.

MilesOh, good, it gets worse.

Becca HartwellAnd that number isn't a guess either. It's coming out of an actual analysis, not just some outraged parent's estimate.

MilesWhich matters because when a number like that gets attached to a lawsuit, it stops being a talking point and starts being evidence.

Becca HartwellI need to be really clear here. That's not the Taylor Swift/Scarlett Johansson bucket. That's a separate legal track, separate plaintiffs, separate harm.

MilesSo we've got two different fires burning at the same time.

Becca HartwellTwo fires, same building. The adult celebrity deepfakes are one lane. That's image rights, that's reputational harm. That's a very online scandal.

MilesAnd the CSAM allegations are a completely different category of law.

Becca HartwellCompletely different. Different statute, different stakes, arguably criminal in a way the celebrity stuff on its own might not be.

MilesBut they're both coming out of the same guardrail failure.

Becca HartwellThat's exactly it. Same tool, same removed refusal behavior, two very different sets of victims.

Miles20,000 in a week is already enough to make a regulator's eyebrow twitch.

Becca HartwellAdd a child safety analysis into the mix, and you're not talking about bad press anymore.

MilesYou're talking about someone in a government office reading both numbers back to back.

Becca HartwellWhich is exactly what happened.

MilesYeah?

Becca HartwellCalifornia's attorney general didn't just watch this trend on the internet. That scale is what put a formal letter on xAI's desk.

MilesNot a strongly worded tweet, an actual letter from an actual attorney general's office.

Becca HartwellThat's the kind of paper trail that turns a viral story into a legal record, whether xAI likes it or not.

MilesExactly. And once a state AG puts pen to paper like that, the company can't just wait for the news cycle to move on.

Becca HartwellWith language in it.

MilesOoh, tell me it's spicy.

Becca HartwellWe'll get there. This is where it gets good. California's attorney general actually put his name on paper.

MilesRob Bonta.

Becca HartwellRob Bonta, January 16th, sends xAI a cease and desist letter.

MilesThat's the charging document right there.

Becca HartwellAttorney General Bonta's letter demands xAI immediately stop creating and distributing deep fake, non-consensual intimate images and child sexual abuse material.

MilesWait, he's naming both in the same breath?

Becca HartwellSame letter, both categories. No separating them out.

MilesSurprising. That's not a warning shot. That's a legal document with your name attached to a crime.

Becca HartwellAnd letters like that don't get sent lightly. A state AG's office doesn't put child sexual abuse material in writing unless they're prepared to back it up.

MilesWhich means whatever comes next, xAI can't say nobody warned them formally in writing.

Becca HartwellAnd that's the pivot for us, too. This stops being a story about weird prompts and becomes a story about who's actually on the hook.

MilesHow fast do they want a response?

Becca HartwellTechCrunch reported the AG's office accused xAI of facilitating the large scale production of non-consensual nudes.

MilesAnd?

Becca HartwellFive days. Five days to prove they'd complied.

MilesFive days to fix something like this?

Becca HartwellThat's what the letter says.

MilesOkay, but California can't be the only government paying attention here.

Becca HartwellIt wasn't. Just California also reported Japan, Canada, and the UK opened their own investigations.

MilesWait, wait, wait. Investigations, plural, in three different countries?

Becca HartwellThree different countries, and Malaysia and Indonesia went further.

MilesFurther how?

Becca HartwellThey temporarily blocked Grok outright.

MilesEven after xAI supposedly added restrictions?

Becca HartwellEven after. The restrictions came late on a Wednesday, and the blocks happened anyway.

MilesSo the fixes weren't fast enough or weren't real enough.

Becca HartwellEither way, blocking access outright is a pretty loud statement from two governments that aren't exactly known for moving fast on tech regulation.

MilesWhich tells you the international pressure wasn't just diplomatic noise. It had teeth attached to it, too.

Becca HartwellOne or the other, and honestly, maybe both.

MilesThis is the part where the show usually asks who's liable: user, platform?

Becca HartwellNo. Hang on. We're past asking. Bonta's letter answers half of it for us. He's not writing to some anonymous poster. He's writing to the company.

MilesThe company that built the feature.

Becca HartwellRight. That's a different animal than blaming whoever typed the prompt.

MilesSuddenly, this feels a lot less like a Reddit thread and a lot more like a courtroom.

Becca HartwellExactly. And regulators moving is one track.

MilesThere's another track?

Becca HartwellThere's another track. One of the women caught up in this didn't sit around waiting for Bonta or Tokyo or anybody else to act.

MilesShe went and did something about it herself.

Becca HartwellShe sued. Plot twist, St. Clair didn't just wait for Bonta's office. She filed her own suit.

MilesI'll just your as headlines has it plain, the mother of Musk's child is suing his own AI company.

Becca HartwellHis son, Romulus, and she's suing over pain and mental distress from the fake images.

MilesThat's a rough sentence to write about someone you had a kid with.

Becca HartwellRight? And it gets worse. NBC News reported she complained directly to xAI.

MilesAnd?

Becca HartwellAnd Grok kept generating explicit deepfakes of her anyway.

MilesEven after she flagged it?

Becca HartwellEven after. NBC's reporting says xAI had actually told her the images wouldn't be altered without consent.

MilesSo they said the right thing, and the system did the wrong thing.

Becca HartwellThat's the negligence claim in a sentence.

MilesAnd that's before you even get to how humiliating it must be to keep having to ask a company to stop making fake images of your own body.

Becca HartwellEvery time she had to send that request again, it's basically reliving the violation, just with extra emails attached to it every single time.

MilesHere's the part that made me sit up, though.

Becca HartwellOh, you're gonna love this part.

MilesxAI countersued her.

Becca HartwellWait. For what?

MilesNBC says Texas federal court over $75,000, claiming she broke their terms of service.

Becca HartwellShut up. They generated the images and then billed her for the paperwork?

MilesThat's the framing, yeah.

Becca HartwellI feel for her. She flagged it. She got a promise. The promise didn't hold, and now she's the one being sued back.

MilesI get the sympathy, but step back. A company facing a criminal investigation turning around and suing the woman it allegedly victimized isn't just optics.

Becca HartwellIt's a strategy.

MilesIt's a company that fights every claim on every front instead of settling one.

Becca HartwellOr it's a company that genuinely thinks her account violated the rules.

MilesMaybe, but even if that's technically true on paper, the optics of countersuing the mother of your boss's kid over a terms of service technicality are just brutal.

Becca HartwellIt reads like a company that's more worried about precedent than about the actual person in front of them, not the headline it makes.

MilesSure. Tell that to a judge next to the word Romulus.

Becca HartwellFair. St. Clair's case is hers, her name, her kid, her complaint, but the questions underneath it aren't just about her anymore.

MilesNo, there's a bigger legal theory sitting behind this whole thing, and it's about to get tested.

Becca HartwellSo here's a new wrinkle nobody saw coming. The class action against xAI just got amended.

MilesAmended how?

Becca HartwellNew plaintiffs, families whose photos got turned into child sexual abuse material through Grok, and now Stability AI is named as a co-defendant, too, over its open weight models.

MilesWait, Stability AI? That's a different company entirely.

Becca HartwellDifferent company, same theory. If your model can produce this stuff, you're on the hook. Doesn't matter whose logo's on it.

MilesWhoa, that's a much bigger net.

Becca HartwellIt also changes the settlement math. Now there are two deep-pocketed defendants instead of one, which usually makes plaintiffs' lawyers a lot more aggressive.

MilesAnd a lot less likely to walk away quietly, since neither company wants to be the one holding the bag alone.

Becca HartwellAnd there's a detail in that same complaint that made my stomach drop.

MilesGo on.

Becca HartwellIt alleges xAI stayed silent for weeks when investigators were trying to identify a perpetrator.

MilesWeeks, while a real kid's photo's out there?

Becca HartwellThat's the allegation, yeah.

MilesOkay, but here's what actually decides all of this, not the silence, the shield. Does Section 230 even apply to a company that built the tool?

Becca HartwellRight, because 230 was written for hosting somebody else's post, not for manufacturing the post yourself.

MilesExactly, and there's a legal analysis from the Center for Democracy and Technology that lays this out. 230 only covers third-party content.

Becca HartwellMeaning?

MilesMeaning a court might say X, the platform, gets protected when a user posts the image, but the model that actually generated that is not necessarily third-party content anymore. That's xAI's own output.

Becca HartwellSo the user typing the prompt could be shielded by the platform, but the platform that built the generator might not get to hide behind the platform defense.

MilesThat's the tension. The tool made the thing. Immunity was built for a bulletin board, not a factory.

Becca HartwellA factory that takes requests.

MilesBasically, yeah.

Becca HartwellSo if a judge buys that distinction, that's the whole ballgame for xAI.

MilesIt's the whole ballgame for every company running one of these image generators, honestly.

Becca HartwellBecause Stability's in it now, too.

MilesRight. That CDT framing isn't about one company. It's about whether the shield covers generative tools at all.

Becca HartwellWhich stops being a law school hypothetical the second there are actual families and actual investigators in the filing.

MilesThere's money attached to this theory now.

Becca HartwellAnd minors.

MilesAnd apparently a company that didn't answer the phone when someone needed answers fast.

Becca HartwellOkay, new plaintiffs just showed up in this thing, and they're minors.

MilesWait, minors are suing directly now? Not just named in someone else's complaint?

Becca HartwellTheir own California suit, and they're not asking for a settlement check, Miles. They're invoking Masha's Law.

MilesWhich is?

Becca HartwellA federal statute built for child exploitation cases. Their complaint demands at least $150,000 per violation, plus xAI has to hand over whatever profit it made off this.

MilesPer violation. So if a judge counts each image as its own violation-

Becca HartwellYou're not talking about one lawsuit anymore. You're talking about a number that could bankrupt a product line.

MilesWh-- That's not a fine. That's a company-ending exposure.

Becca HartwellAnd keep in mind, we're not talking about one plaintiff here. Every minor represented in that filing could count as its own set of violations.

MilesSo the math on this doesn't just add up slowly. It compounds fast, and that's before any other state decides to file something similar.

Becca HartwellAnd disgorgement means they don't even have to prove xAI profited off them specifically. They just want the money the feature made, period.

MilesOkay, so civil court is one problem, but you said something about criminal.

Becca HartwellRight. Alongside that cease and desist letter we already covered, the state opened an actual criminal investigation into xAI.

MilesNot just a warning letter with teeth, an investigation.

Becca HartwellCorrect. Two separate tracks, same target. A civil case demanding money and prosecutors deciding whether there's a crime.

MilesThat's the part that should terrify Musk more than any lawsuit. Lawsuits get settled. Criminal exposure doesn't go away with a check.

Becca HartwellAnd it's not like he can just fire the PR team and call it handled. Prosecutors don't care how the apology tour goes.

MilesRight. A press release doesn't move a grand jury, not even a good one.

Becca HartwellAnd it doesn't matter which theory wins first. The point is xAI is now defending on every front at once, civil plaintiffs, a state AG, and a prosecutor deciding whether to file charges.

MilesSo this stopped being a PR problem a while ago.

Becca HartwellIt stopped being hypothetical. This is the part where the company actually has to answer for what Spicy Mode did.

MilesWhich means it's time to actually decide something.

Becca HartwellSo who takes the fall here? The person typing the prompt, the platform hosting it, or the company that built the thing to do this in the first place?

MilesVerdict time.

Becca HartwellOkay, verdict time. I keep coming back to the women in this, Taylor, Scarlett, Ashley. None of them asked to be a feature test.

MilesRight.

Becca HartwellAnd whatever the legal outcome, they're the ones fielding the messages, screenshotting the images, having to prove in public that something is fake. That's a cost nobody put a price tag on tonight.

MilesIt's real.

Becca HartwellAnd it's not like the platforms hosting the reposts are rushing to take them down either, so the exposure just keeps compounding long after the news cycle forgets her name.

MilesWhich is exactly the kind of harm that never shows up on a balance sheet, no matter how the lawsuits shake out.

Becca HartwellIf the media cycle treats it like gossip for about a day and then moves on, but she's still living with it after we've all changed the subject.

MilesI hear that, but I land somewhere colder. xAI shipped a mode that turns off the refusal behavior. Nobody tricked the chatbot into this.

Becca HartwellOkay, so the user typing the request just doesn't matter to you?

MilesThey matter, but you don't get to build the button and act surprised when people push it. Musk's company designed the capability, marketed it, and left it running.

Becca HartwellOkay, you sound like you already wrote the opening statement.

MilesMaybe I did. My verdict's the company.

Becca HartwellMine's the women who got dragged into a product demo they never signed up for.

MilesFair split.

Becca HartwellThat's the show. That's the case. If you've got one for us, a scandal, a lawsuit, something that needs a jury of two opinionated hosts, leave it in a review or tag us at StarWitnessPod.

MilesNew episodes drop every Wednesday.

Becca HartwellAnd hey, try to keep your face off the internet till then.

MilesCourts adjourned.