Billion-Dollar Bets, AI That Learns From Its Own Mistakes, and the Scammers Who Got Too Good

Lauren: Bye. Derek: Hey, hey, hey, welcome back to Tech Insider Weekly. I'm Lauren and oh man, Derek, we have got a week's worth of stories that I cannot wait to rip into. Speaker 3: Lauren, same. Like I tried to sleep last night and I kept thinking about the Anthropic thing. Derek: Okay, so same, we'll get into that. But first, the full menu. Speaker 3: Okay, so get this. Three separate startups each raised $1.3 billion in the same news cycle. Go out. Derek: Three. Speaker 3: Same number. Derek: Separately. That's not a coincidence, that's a vibe. Speaker 3: Right? And while that's happening, smaller AI startups are getting absolutely squeezed between big tech and demanding VCs. Eric Newcomer over at the New Stack had a pretty sobering read on that. Derek: Nodding. We're also going to talk about AI assistance because Apple and Google both dropped major overhaul announcements on the same day. Bloomberg covered Google's Android AI push and MacRumors reported Siri is getting a full chat interface in iOS. Speaker 3: Siri learning to complete a sentence plot twist, which Derek: Truly. And then security, because Google identified the first confirmed AI-built zero-day exploit, according to both Axios and The Verge. Hackers used AI to find it. Speaker 3: is not great. Derek: Not great at all. Plus Chinese deepfake software targeting executives on Zoom and Teams. Joseph Cox at 404 Media got a copy of the actual software. Speaker 3: Okay, yeah, we need to talk about that one. Derek: And we will, starting right now with the AI startup world. Let's get into it. Okay, so get this. Anthropic just taught its AI to dream. Speaker 3: I'm sorry, what? Derek: No, like literally, they're calling it dreaming. VentureBeat covered the announcement, and the idea is that Claude's agents can now review their past sessions, spot where they went wrong, and self-correct over time, which Speaker 3: So it's learning from its mistakes. Derek: sounds great until you sit with it for a second and think, wait, what if it learns the wrong lesson? Speaker 3: Ah, yes, the classic AI ice. I studied the best scammer problem. Derek: Exactly! Anthropic unveiled it at their second annual Code with Claude developer conference in San Francisco, and the pitch is real. Enterprises have been asking for self-correcting agents before they'll trust them with actual production workloads. Speaker 3: That makes sense. You can't hand the keys to something that makes the same mistake on loop. Derek: Oh, Speaker 3: But the dreaming label? Come on. Someone in marketing was absolutely having a moment. Derek: 100%. But here's the thing. If it actually works, this is a pretty big deal for enterprise automation. Agents that get better without you babysitting them? Speaker 3: That's the dream. Pun very much intended. Derek: Pun earned. Okay, so pivot because this week did not stop there. Speaker 3: Not even close. Derek: So get this. Three separate companies all raised $1.3 billion, same news cycle. Speaker 3: The same number three times? Derek: Three times! The New York Times had the story on a startup called AMP. AMP, which is building what they're describing as an AI grid, basically infrastructure to distribute AI compute more widely. Speaker 3: Okay, so picks and shovels. I get that bet. Derek: Then Forbes' Rashi Shrivastava's piece covers Scribe, founded in 2019 by Jennifer Smith. Their whole thing is a browser extension that sits on your laptop, watches what you do, records what you do, and uses that to help AI agents eventually take over your repetitive tasks. Speaker 3: So it watches you work. Derek: Silently Speaker 3: Wow. Derek: watching, learning. Speaker 3: I mean, I get the use case, but also hard pass on installing that personally. Derek: Same. And then the third 1.3 billion went to a compute access platform covered by AI business, focused on widening who can actually afford to run AI at scale. Speaker 3: So you've got infrastructure, surveillance as a service, and democratized compute, three totally different bets on where AI actually lands. Derek: All worth the same exact amount, it's almost like investors are just throwing equal portions at every possible future. Speaker 3: A diversified portfolio of AI anxieties. Derek: Exactly what it is. But here's where it gets uncomfortable, because not everyone's raising that kind of money. Eric Newcomer's piece in the New Stack came out of the AI Agent Conference in New York this week, and the vibe from smaller startups was... Not great. Speaker 3: What's the worry? Derek: It's the squeeze. Big players like Google, OpenAI, Anthropic, they can run models at a loss. They have the balance sheets for it. Smaller AI startups are trying to compete on the same tech, but on a cloud compute budget that makes every API call feel like bleeding out. Speaker 3: Plot twist, the VCs still want hyper growth. Derek: Right. One founder at the conference put it pretty... Pretty bluntly to newcomers, they're trying to find space where they won't get trampled by one of the big models just absorbing their use case. Speaker 3: That's the real tension. It's not whether AI is working. It's whether there's room to build a company around it if you're not already enormous. Derek: And the billion dollar rounds we just talked about? Those aren't going to scrappy five person teams figuring it out. That capital is concentrating. Speaker 3: Which means the survival question for everyone else is, is, okay, what's your actual moat? Derek: Not a contract, a niche, something a model update can't just erase overnight. Speaker 3: So who's actually cracking that? Who's landing the contracts that buy them time? Derek: Oh, we have some answers on that front. Some of them are pretty surprising. Okay, so here's what that contract competition looks like in practice. TechCrunch reported this week that Amazon Ring evaluated more than 40 voice AI vendors last year and picked a startup called Vapi. Ring now routes 100% of its inbound calls through them. Speaker 3: A hundred percent? Like no fallback? Derek: No fallback. And that deployment helped Vapi close a $50 million Series B at a $500 million valuation, according to TechCrunch. Speaker 3: So the moat we were just asking about, winning one big enterprise contract, can basically build it overnight. Derek: That's the whole story. Voice AI isn't a novelty anymore. Ring isn't running it as a pilot. It's infrastructure. Speaker 3: And the timing is interesting because both Apple and Google are completely overhauling their AI assistance right now, essentially at the same moment. Derek: Bloomberg reported today that Google is unveiling a bunch of new Android AI features. and they framed it explicitly as a head of Apple Siri revamp. Speaker 3: And on the Apple side, MacRumors broke a report today that iOS 17 is getting a full Siri redesign. It moves to the Dynamic Island, gets a dedicated app for the first time. Lauren: And they're positioning it to compete directly with ChatGPT and Claude. Derek: Siri competing with Claude, we've come a long way. Lauren: I mean, Siri adds some ground to cover. The thing is, Lauren, both of these announcements landed on the same day, which tells you the race isn't really about who has better technology, it's about who owns the interface you already reach for. Derek: Right, and that's actually where I think the consumer bet gets interesting and a little weird. After Droid Life flagged this earlier in the month, Google is internally testing an agent called Remy. Gemini-powered and based on code strings inside the latest Google app, it can communicate with others, share documents, and make purchases. Lauren: Autonomously. Derek: Autonomously on your behalf, it learns your preferences and just acts. Lauren: Great. My phone has opinions and a credit card. Derek: Right? So on one hand, incredibly convenient. On the other hand, has anyone stopped to ask whether people actually want their assistants spending money without confirmation? Lauren: I'm on both sides a little bit, like auto-renewing a subscription is one thing, but if Remy decides I need a new blender because I searched smoothie recipes once. Derek: That's how you end up with four blenders. Lauren: Yeah, yeah, four blenders and a juicer you didn't ask for. Derek: The trust question here is real, though. You're essentially giving an agent access to your payment methods. And telling it to use judgment—and judgment based on what—your search history from a bad Tuesday? Lauren: Which is why I think the winner in this space won't just be whoever has the smartest model—it'll be whoever you trust not to embarrass you. Derek: With a smirk, Apple has spent years building that trust around privacy. Google has the data advantage, neither has fully won the assistant game yet. Lauren: And Vapi just proved a scrappy startup can walk in and take the c Take the contract from under both of them, if the execution is there. Derek: Nodding. So the phone in your pocket is about to get a lot more opinionated. The question is whether you're comfortable with how it got that way. Lauren: Slightly sobered, and here's where it gets a little darker-the same tools making your assistants smarter and more autonomous turns out they're making other things smarter and more autonomous, too, not in a good way. Derek: Serious tone. Yeah, we need to talk about what's happening on the security side. guide. So flip that on its head. The same AI making our phones smarter? Lauren: Yeah? Derek: Also making scammers significantly better at their jobs. Lauren: Cool. Speaker 3: Cool, cool, cool. Derek: So let's start with Braintrust. TechCrunch reported this week that the company got breached, unauthorized access into one of their AWS cloud accounts which exposed customer API keys. Lauren: Okay, so the context here is what kills me. Derek: Right? Lauren: Braintrust is literally in the business of testing AI reliability. Their whole pitch is, we evaluate whether your AI systems can be trusted. Derek: And then they had to email every customer and say, Please rotate your keys. Lauren: Oof look! Derek: Yeah. According to Lorenzo Franceschi-Bicchierai at TechCrunch, they said they've communicated with one impacted customer and found no evidence of broader exposure. So maybe contained, but still. Lauren: Nothing builds confidence in your AI testing company like a breach email on a Monday morning. Derek: Hard agree. But here's where this week goes from embarrassing to genuinely alarming. Google's Threat Intelligence Group published a report saying they identified and stopped what may be the first known zero-day exploit built with AI assistance. Lauren: Wait, first confirmed one? Derek: First confirmed one in the wild, yeah. And per the Verge's Stevie Bonifield, this wasn't a test, this was actual cyber crime threat actors planning a mass exploitation event targeting two-factor authentication. Lauren: So they used AI to find the vulnerability and write the exploit. The exploit? Derek: Yep. The Hacker News covered this too, framing it as AI being put to use for the full pipeline, vulnerability discovery and exploit generation, not just one or the other. Lauren: That's the part that matters. Before, you needed serious expertise at every step. Now you're compressing that. Derek: Exactly. And Google caught it, which is good, but the structural question you have to ask is how many... He didn't get caught. Lauren: Yeah, defense got better, offense got the same upgrade. Derek: On the same day. Lauren: So what does defense even look like now? Derek: I mean, That's the uncomfortable answer, Right? You're using AI to detect AI-generated threats. It's basically a mirror match. Lauren: Aurora Boost, but make it a startup pitch. Derek: Perfect. Okay, but I need to talk about the deepfakes because this is where it gets genuinely surreal. Real. Lauren: Oh no, what did Joseph Cox find? Derek: So 404 Media got access to a piece of Chinese software called Haotian AI. It's marketed to scammers and it does real-time face and voice replacement on WhatsApp, Zoom, Teams, you name it. Lauren: Real time, like live on a call? Derek: Live on a call. Cox literally tested it on a Teams call. and watch his own face (five o'clock shadow included) appear on someone else's body! Lauren: Wow! That is the stuff of nightmares. Derek: The name it's marketed under? hello boss, because the primary use case is impersonating executives to authorize wire transfers. Lauren: How charming. Derek: It's running on a gaming laptop, not a data center, not specialized hardware; a gaming laptop. Lauren: That's the thing that gets me. The barrier's basically gone. Derek: Right. And there's no clean solution, which is a weird place to sit with all the optimism we were just talking about earlier. Lauren: Though it does reframe the bigger picture, doesn't it? The same week this is happening, billions in capital are flowing into AI globally, not just here. Derek: Which is actually where we're headed next because the investment story gets a lot more interesting when you zoom out past Silicon Valley. Okay, so the capital is still moving, and not just in Silicon Valley anymore. Lauren: Okay, so get this: Reuters reported something I genuinely did not see coming this week. Derek: Skyroot. Lauren: India's Skyroot just became its country's first billion dollar space tech startup. GIC, Sherpa Palo, BlackRock, all backing it. Derek: BlackRock in a space tech startup in India. Lauren: I know! And the thing is, this changes the narrative, right? Right? Space infrastructure stopped being just an American story a while ago, but a billion dollar valuation? That makes it impossible to ignore. Derek: It's the signal that serious institutional money is going global on this, not just VC side bets. Lauren: Exactly. And that connects to what Crunchbase News head out yesterday on European AI funding. Derek: Yeah, Janae Tara's piece. Roughly half of European venture funding in 2026. These six so far has been Lauren: It's been AI-related. Derek: Half. Lauren: Wow. Derek: And it includes three brand new Frontier model companies, not just someone else's infrastructure with a coat of paint. Lauren: Okay, but I'm on both sides a little bit here. Half sounds massive until you remember the absolute dollar figures in the U.S. still dwarf Europe. Derek: Fair point. The gap is closing, Lauren: Right. Derek: but closing slow. Lauren: Right, right. So the question isn't whether Europe is investing, it's whether they're investing f***ing. acting fast enough to shape the models that matter. Derek: And that's the billion dollar question right there. Lauren: OK, switching gears, because there's a story from Fortune this week that I think is a sleeper hit. Derek: Sure a day? Lauren: You read it? Derek: chuckling] An a16z alum and an ex Apple ML lead walking to a seed round. Lauren: Twenty million dollars led by Accel, and their whole pitch is bringing AI to what to what they're calling real economy businesses, the ones that don't do quarterly earnings calls. Derek: I mean, that is a huge addressable market. Like most businesses are not Fortune 500. Lauren: That's exactly the bet. But here's the thing I keep turning over. Is this a contrarian stroke of genius or is it a sign that the upmarket enterprise AI pitch is just getting harder? Derek: Deadpan. Why not both? Lauren: Yeah, why not both? Derek: And it loops back to what we were talking about. Talking about earlier, the survival question-if the top end is getting crushed by big tech running models at a loss, maybe the smart play isn't going deeper, it's going wider instead. Lauren: Real economy AI-that might be where the next durable companies get built. Derek: The money's global now, the opportunity might be too. Lauren: Okay, that was a packed one. If there's one thing I keep coming back to, it's that trust question we kept circling, whether it's an AI agent spending your money or a deepfake boss on a Zoom call. Derek: Right, and the dreaming conversation kind of set the tone early because if AI is learning from its own mistakes, the question of what lesson it's actually learning matters a lot. Lauren: A lot. Derek: The through line this week is honestly this, the technology is moving fast. But trust is the thing nobody's figured out yet. Speaker 3: Love that. Derek, thanks for being the voice of reason as always. Derek: You make it sound like a full-time job. Speaker 3: It is. All right, if you liked today's episode, subscribe wherever you listen, leave us a review, and tag us if there's a founder or story we need to cover. New episodes drop every Wednesday. Derek: Thanks for hanging out with us. See you next week.