Billion-Dollar Bets, AI That Learns From Its Own Mistakes, and the Scammers Who Got Too Good

LaurenBye.

DerekHey, hey, hey, welcome back to Tech Insider Weekly. I'm Lauren and oh man, Derek, we have got a week's worth of stories that I cannot wait to rip into.

Speaker 3Lauren, same. Like I tried to sleep last night and I kept thinking about the Anthropic thing.

DerekOkay, so same, we'll get into that. But first, the full menu.

Speaker 3Okay, so get this. Three separate startups each raised $1.3 billion in the same news cycle. Go out.

DerekThree.

Speaker 3Same number.

DerekSeparately. That's not a coincidence, that's a vibe.

Speaker 3Right? And while that's happening, smaller AI startups are getting absolutely squeezed between big tech and demanding VCs. Eric Newcomer over at the New Stack had a pretty sobering read on that.

DerekNodding. We're also going to talk about AI assistance because Apple and Google both dropped major overhaul announcements on the same day. Bloomberg covered Google's Android AI push and MacRumors reported Siri is getting a full chat interface in iOS.

Speaker 3Siri learning to complete a sentence plot twist, which

DerekTruly. And then security, because Google identified the first confirmed AI-built zero-day exploit, according to both Axios and The Verge. Hackers used AI to find it.

Speaker 3is not great.

DerekNot great at all. Plus Chinese deepfake software targeting executives on Zoom and Teams. Joseph Cox at 404 Media got a copy of the actual software.

Speaker 3Okay, yeah, we need to talk about that one.

DerekAnd we will, starting right now with the AI startup world. Let's get into it. Okay, so get this. Anthropic just taught its AI to dream.

Speaker 3I'm sorry, what?

DerekNo, like literally, they're calling it dreaming. VentureBeat covered the announcement, and the idea is that Claude's agents can now review their past sessions, spot where they went wrong, and self-correct over time, which

Speaker 3So it's learning from its mistakes.

Dereksounds great until you sit with it for a second and think, wait, what if it learns the wrong lesson?

Speaker 3Ah, yes, the classic AI ice. I studied the best scammer problem.

DerekExactly! Anthropic unveiled it at their second annual Code with Claude developer conference in San Francisco, and the pitch is real. Enterprises have been asking for self-correcting agents before they'll trust them with actual production workloads.

Speaker 3That makes sense. You can't hand the keys to something that makes the same mistake on loop.

DerekOh,

Speaker 3But the dreaming label? Come on. Someone in marketing was absolutely having a moment.

Derek100%. But here's the thing. If it actually works, this is a pretty big deal for enterprise automation. Agents that get better without you babysitting them?

Speaker 3That's the dream. Pun very much intended.

DerekPun earned. Okay, so pivot because this week did not stop there.

Speaker 3Not even close.

DerekSo get this. Three separate companies all raised $1.3 billion, same news cycle.

Speaker 3The same number three times?

DerekThree times! The New York Times had the story on a startup called AMP. AMP, which is building what they're describing as an AI grid, basically infrastructure to distribute AI compute more widely.

Speaker 3Okay, so picks and shovels. I get that bet.

DerekThen Forbes' Rashi Shrivastava's piece covers Scribe, founded in 2019 by Jennifer Smith. Their whole thing is a browser extension that sits on your laptop, watches what you do, records what you do, and uses that to help AI agents eventually take over your repetitive tasks.

Speaker 3So it watches you work.

DerekSilently

Speaker 3Wow.

Derekwatching, learning.

Speaker 3I mean, I get the use case, but also hard pass on installing that personally.

DerekSame. And then the third 1.3 billion went to a compute access platform covered by AI business, focused on widening who can actually afford to run AI at scale.

Speaker 3So you've got infrastructure, surveillance as a service, and democratized compute, three totally different bets on where AI actually lands.

DerekAll worth the same exact amount, it's almost like investors are just throwing equal portions at every possible future.

Speaker 3A diversified portfolio of AI anxieties.

DerekExactly what it is. But here's where it gets uncomfortable, because not everyone's raising that kind of money. Eric Newcomer's piece in the New Stack came out of the AI Agent Conference in New York this week, and the vibe from smaller startups was... Not great.

Speaker 3What's the worry?

DerekIt's the squeeze. Big players like Google, OpenAI, Anthropic, they can run models at a loss. They have the balance sheets for it. Smaller AI startups are trying to compete on the same tech, but on a cloud compute budget that makes every API call feel like bleeding out.

Speaker 3Plot twist, the VCs still want hyper growth.

DerekRight. One founder at the conference put it pretty... Pretty bluntly to newcomers, they're trying to find space where they won't get trampled by one of the big models just absorbing their use case.

Speaker 3That's the real tension. It's not whether AI is working. It's whether there's room to build a company around it if you're not already enormous.

DerekAnd the billion dollar rounds we just talked about? Those aren't going to scrappy five person teams figuring it out. That capital is concentrating.

Speaker 3Which means the survival question for everyone else is, is, okay, what's your actual moat?

DerekNot a contract, a niche, something a model update can't just erase overnight.

Speaker 3So who's actually cracking that? Who's landing the contracts that buy them time?

DerekOh, we have some answers on that front. Some of them are pretty surprising. Okay, so here's what that contract competition looks like in practice. TechCrunch reported this week that Amazon Ring evaluated more than 40 voice AI vendors last year and picked a startup called Vapi. Ring now routes 100% of its inbound calls through them.

Speaker 3A hundred percent? Like no fallback?

DerekNo fallback. And that deployment helped Vapi close a $50 million Series B at a $500 million valuation, according to TechCrunch.

Speaker 3So the moat we were just asking about, winning one big enterprise contract, can basically build it overnight.

DerekThat's the whole story. Voice AI isn't a novelty anymore. Ring isn't running it as a pilot. It's infrastructure.

Speaker 3And the timing is interesting because both Apple and Google are completely overhauling their AI assistance right now, essentially at the same moment.

DerekBloomberg reported today that Google is unveiling a bunch of new Android AI features. and they framed it explicitly as a head of Apple Siri revamp.

Speaker 3And on the Apple side, MacRumors broke a report today that iOS 17 is getting a full Siri redesign. It moves to the Dynamic Island, gets a dedicated app for the first time.

LaurenAnd they're positioning it to compete directly with ChatGPT and Claude.

DerekSiri competing with Claude, we've come a long way.

LaurenI mean, Siri adds some ground to cover. The thing is, Lauren, both of these announcements landed on the same day, which tells you the race isn't really about who has better technology, it's about who owns the interface you already reach for.

DerekRight, and that's actually where I think the consumer bet gets interesting and a little weird. After Droid Life flagged this earlier in the month, Google is internally testing an agent called Remy. Gemini-powered and based on code strings inside the latest Google app, it can communicate with others, share documents, and make purchases.

LaurenAutonomously.

DerekAutonomously on your behalf, it learns your preferences and just acts.

LaurenGreat. My phone has opinions and a credit card.

DerekRight? So on one hand, incredibly convenient. On the other hand, has anyone stopped to ask whether people actually want their assistants spending money without confirmation?

LaurenI'm on both sides a little bit, like auto-renewing a subscription is one thing, but if Remy decides I need a new blender because I searched smoothie recipes once.

DerekThat's how you end up with four blenders.

LaurenYeah, yeah, four blenders and a juicer you didn't ask for.

DerekThe trust question here is real, though. You're essentially giving an agent access to your payment methods. And telling it to use judgment—and judgment based on what—your search history from a bad Tuesday?

LaurenWhich is why I think the winner in this space won't just be whoever has the smartest model—it'll be whoever you trust not to embarrass you.

DerekWith a smirk, Apple has spent years building that trust around privacy. Google has the data advantage, neither has fully won the assistant game yet.

LaurenAnd Vapi just proved a scrappy startup can walk in and take the c Take the contract from under both of them, if the execution is there.

DerekNodding. So the phone in your pocket is about to get a lot more opinionated. The question is whether you're comfortable with how it got that way.

LaurenSlightly sobered, and here's where it gets a little darker-the same tools making your assistants smarter and more autonomous turns out they're making other things smarter and more autonomous, too, not in a good way.

DerekSerious tone. Yeah, we need to talk about what's happening on the security side. guide. So flip that on its head. The same AI making our phones smarter?

LaurenYeah?

DerekAlso making scammers significantly better at their jobs.

LaurenCool.

Speaker 3Cool, cool, cool.

DerekSo let's start with Braintrust. TechCrunch reported this week that the company got breached, unauthorized access into one of their AWS cloud accounts which exposed customer API keys.

LaurenOkay, so the context here is what kills me.

DerekRight?

LaurenBraintrust is literally in the business of testing AI reliability. Their whole pitch is, we evaluate whether your AI systems can be trusted.

DerekAnd then they had to email every customer and say, Please rotate your keys.

LaurenOof look!

DerekYeah. According to Lorenzo Franceschi-Bicchierai at TechCrunch, they said they've communicated with one impacted customer and found no evidence of broader exposure. So maybe contained, but still.

LaurenNothing builds confidence in your AI testing company like a breach email on a Monday morning.

DerekHard agree. But here's where this week goes from embarrassing to genuinely alarming. Google's Threat Intelligence Group published a report saying they identified and stopped what may be the first known zero-day exploit built with AI assistance.

LaurenWait, first confirmed one?

DerekFirst confirmed one in the wild, yeah. And per the Verge's Stevie Bonifield, this wasn't a test, this was actual cyber crime threat actors planning a mass exploitation event targeting two-factor authentication.

LaurenSo they used AI to find the vulnerability and write the exploit. The exploit?

DerekYep. The Hacker News covered this too, framing it as AI being put to use for the full pipeline, vulnerability discovery and exploit generation, not just one or the other.

LaurenThat's the part that matters. Before, you needed serious expertise at every step. Now you're compressing that.

DerekExactly. And Google caught it, which is good, but the structural question you have to ask is how many... He didn't get caught.

LaurenYeah, defense got better, offense got the same upgrade.

DerekOn the same day.

LaurenSo what does defense even look like now?

DerekI mean, That's the uncomfortable answer, Right? You're using AI to detect AI-generated threats. It's basically a mirror match.

LaurenAurora Boost, but make it a startup pitch.

DerekPerfect. Okay, but I need to talk about the deepfakes because this is where it gets genuinely surreal. Real.

LaurenOh no, what did Joseph Cox find?

DerekSo 404 Media got access to a piece of Chinese software called Haotian AI. It's marketed to scammers and it does real-time face and voice replacement on WhatsApp, Zoom, Teams, you name it.

LaurenReal time, like live on a call?

DerekLive on a call. Cox literally tested it on a Teams call. and watch his own face (five o'clock shadow included) appear on someone else's body!

LaurenWow! That is the stuff of nightmares.

DerekThe name it's marketed under? hello boss, because the primary use case is impersonating executives to authorize wire transfers.

LaurenHow charming.

DerekIt's running on a gaming laptop, not a data center, not specialized hardware; a gaming laptop.

LaurenThat's the thing that gets me. The barrier's basically gone.

DerekRight. And there's no clean solution, which is a weird place to sit with all the optimism we were just talking about earlier.

LaurenThough it does reframe the bigger picture, doesn't it? The same week this is happening, billions in capital are flowing into AI globally, not just here.

DerekWhich is actually where we're headed next because the investment story gets a lot more interesting when you zoom out past Silicon Valley. Okay, so the capital is still moving, and not just in Silicon Valley anymore.

LaurenOkay, so get this: Reuters reported something I genuinely did not see coming this week.

DerekSkyroot.

LaurenIndia's Skyroot just became its country's first billion dollar space tech startup. GIC, Sherpa Palo, BlackRock, all backing it.

DerekBlackRock in a space tech startup in India.

LaurenI know! And the thing is, this changes the narrative, right? Right? Space infrastructure stopped being just an American story a while ago, but a billion dollar valuation? That makes it impossible to ignore.

DerekIt's the signal that serious institutional money is going global on this, not just VC side bets.

LaurenExactly. And that connects to what Crunchbase News head out yesterday on European AI funding.

DerekYeah, Janae Tara's piece. Roughly half of European venture funding in 2026. These six so far has been

LaurenIt's been AI-related.

DerekHalf.

LaurenWow.

DerekAnd it includes three brand new Frontier model companies, not just someone else's infrastructure with a coat of paint.

LaurenOkay, but I'm on both sides a little bit here. Half sounds massive until you remember the absolute dollar figures in the U.S. still dwarf Europe.

DerekFair point. The gap is closing,

LaurenRight.

Derekbut closing slow.

LaurenRight, right. So the question isn't whether Europe is investing, it's whether they're investing f***ing. acting fast enough to shape the models that matter.

DerekAnd that's the billion dollar question right there.

LaurenOK, switching gears, because there's a story from Fortune this week that I think is a sleeper hit.

DerekSure a day?

LaurenYou read it?

Derekchuckling] An a16z alum and an ex Apple ML lead walking to a seed round.

LaurenTwenty million dollars led by Accel, and their whole pitch is bringing AI to what to what they're calling real economy businesses, the ones that don't do quarterly earnings calls.

DerekI mean, that is a huge addressable market. Like most businesses are not Fortune 500.

LaurenThat's exactly the bet. But here's the thing I keep turning over. Is this a contrarian stroke of genius or is it a sign that the upmarket enterprise AI pitch is just getting harder?

DerekDeadpan. Why not both?

LaurenYeah, why not both?

DerekAnd it loops back to what we were talking about. Talking about earlier, the survival question-if the top end is getting crushed by big tech running models at a loss, maybe the smart play isn't going deeper, it's going wider instead.

LaurenReal economy AI-that might be where the next durable companies get built.

DerekThe money's global now, the opportunity might be too.

LaurenOkay, that was a packed one. If there's one thing I keep coming back to, it's that trust question we kept circling, whether it's an AI agent spending your money or a deepfake boss on a Zoom call.

DerekRight, and the dreaming conversation kind of set the tone early because if AI is learning from its own mistakes, the question of what lesson it's actually learning matters a lot.

LaurenA lot.

DerekThe through line this week is honestly this, the technology is moving fast. But trust is the thing nobody's figured out yet.

Speaker 3Love that. Derek, thanks for being the voice of reason as always.

DerekYou make it sound like a full-time job.

Speaker 3It is. All right, if you liked today's episode, subscribe wherever you listen, leave us a review, and tag us if there's a founder or story we need to cover. New episodes drop every Wednesday.

DerekThanks for hanging out with us. See you next week.