Published by Teller's Tech - DevOps, SRE and Cloud Podcast
Ship It Weekly is a short, practical recap of what actually matters in DevOps, SRE, cloud infrastructure, and platform engineering. Each episode, your host Brian Teller walks through the latest outages, releases, tools, and incident writeups, then translates them into “here’s what this means for your systems” instead of just reading headlines. Expect a couple of main stories with context, a quick hit of tools or releases worth bookmarking, and the occasional segment on on-call, burnout, or team culture. This isn’t a certification prep show or a lab walkthrough. It’s aimed at people who are already working in the space and want to stay sharp without scrolling status pages, cloud updates, and blogs all week. You’ll hear about things like cloud provider incidents, Kubernetes and platform trends, Terraform and infrastructure changes, and real postmortems that are actually worth your time. Most episodes are 15–30 minutes, so you can catch up on the way to work or between meetings. Every now and then there will be a “special” focused on a big outage or a specific theme, but the default format is simple: what happened, why it matters, and what you might want to do about it in your own environment. If you’re the person people DM when something is broken in prod, or you’re building the cloud and platform everyone else ships on top of, Ship It Weekly is meant to be in your rotation.
Listen on Apple PodcastsThis week on Ship It Weekly : Telstra’s mobile network jumped back to 2006 after a timing device restarted with the wrong date, disrupting calls, data sessions, and hundreds of emergency calls. DoorDash explains how Entity Cache, built with Envoy and Valkey, handles more than 1.5 million requests per second and uses stale-data policies, invalidation, and fallback behavior as a reliability layer. The latest MCP release candidate removes protocol-level sessions, making servers easier to scale behind ordinary load balancers while leaving teams responsible for authentication, tracing, retries, rate limits, and application state. GitHub and PyPI are also adding friction to package automation. Dependabot now delays routine updates by three days, while PyPI blocks new files from releases older than 14 days. Links Telstra outage https://www.telstra.com.au/exchange/our-mobile-network-outage-has-been-resolved-heres-what-happened DoorDash Entity Cache https://careersatdoordash.com/blog/high-performance-proxy-cache-for-doordash-services/ MCP specification release candidate https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/ Dependabot package cooldown https://github.blog/changelog/2026-07-14-dependabot-version-updates-introduce-default-package-cooldown/ PyPI release-file restrictions https://blog.pypi.org/posts/2026-07-22-releases-now-reject-new-files-after-14-days/ Amazon ECS Action Logs https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-ecs-action-logs/ Network Load Balancer listener rules https://aws.amazon.com/about-aws/whats-new/2026/07/aws-network-load-balancer-supports-listener-rules/ Amazon Managed Prometheus limits https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-managed-service-prometheus-1500m-metrics-workspace/ PixelSmash in FFmpeg https://jfrog.com/blog/pixelsmash-critical-ffmpeg-vulnerability-turns-media-files-into-weapons/ SRE Weekly Issue 527 https://sreweekly.com/sre-weekly-issue-527/ This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W31/ More episodes https://shipitweekly.fm/
This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Jay Lark of Hookbridge about webhook reliability, retries, idempotency, replay, security, local development, and what happens when a simple HTTP POST becomes production infrastructure. Jay is a Principal DevOps Engineer and the founder of Hookbridge, a service focused on making webhook delivery more reliable and easier to operate. We start with the basic model: one service sends an HTTP POST to another when something happens. The happy path is easy. The problems begin when an endpoint is unavailable, responds too slowly, receives duplicate events, or gets them out of order. Jay explains what “at least once delivery” means and why receivers must expect duplicates. We talk about idempotency, event IDs, retry behavior, availability during deployments, and why returning a 200 does not prove downstream processing succeeded. We also dig into observability and security. Teams need enough visibility to know whether a webhook arrived, whether signature verification passed, what response was returned, and where processing failed. Jay breaks down HMAC signatures, timestamps, replay protection, and why a valid signature still does not replace normal business-logic validation. Local development is another source of friction. External providers cannot send events directly to localhost , so developers often rely on temporary tunnels, staging deployments, copied payloads, or mocks. Jay explains how Hookbridge uses a fixed URL and local client to forward real webhook traffic to a developer’s machine. We also talk about n8n, self-hosted OpenClaw systems, Hookbridge pull endpoints, and when polling may be simpler or safer than exposing another inbound endpoint. The big takeaway: design the failure path before a webhook becomes business-critical. Verify the sender, expect duplicate and out-of-order events, build enough visibility to debug failures, and have a replay strategy before the first incident. Highlights • Why webhooks are harder than “just an HTTP POST” • What at-least-once delivery means for receivers • Why idempotency, retries, and event ordering matter • What teams need for webhook observability and debugging • Where HMAC signatures, timestamps, and replay protection fit • Why local webhook development is still awkward • When polling or pull-based delivery may be a better fit • When teams should stop building webhook infrastructure themselves Links Hookbridge: https://hookbridge.io Hookbridge local development CLI: https://www.hookbridge.io/cli.html Hookbridge pull endpoints: https://www.hookbridge.io/pull.html Jay Lark on LinkedIn: https://www.linkedin.com/in/jay-lark-ba7a3b5/ OpenClaw: https://openclaw.ai More episodes and show notes: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This week on Ship It Weekly : AWS CloudFormation Express mode promises faster infrastructure feedback by reporting deployments complete before extended resource stabilization finishes. Apache Spark 4.2 adds native vector operations and nearest-neighbor joins, giving some teams a way to keep AI data workloads closer to the platforms they already run. GitLab’s latest research says AI is helping developers generate and commit code faster, but review, testing, governance, and deployment are not accelerating at the same pace. Then, a platform engineering case study from Sevdesk shows how minimum viable governance, useful feedback, and progressive enforcement can improve compliance without turning the platform team into another approval queue. The theme this week: making one part of the system faster does not automatically improve the whole system. In the lightning round: OpenShift 4.22.5 receives an important security update, outdated autoscaling thresholds disrupt GitHub Actions, Cloudflare experiences an incident where some POST requests fail to reach customer origins, and GitHub Code Quality becomes generally available—with automatic billing attached. The episode closes with Reid Savage reflecting on their first year managing Honeycomb’s SRE team and the difference between what a capable team could do and what it should do. Links CloudFormation Express mode https://aws.amazon.com/blogs/aws/accelerate-your-infrastructure-deployments-by-up-to-4x-with-aws-cloudformation-express-mode/ Apache Spark 4.2 https://spark.apache.org/releases/spark-release-4-2-0.html GitLab AI Accountability Report https://about.gitlab.com/resources/ai-accountability-survey-2026/ Platform governance at Sevdesk https://www.infoq.com/presentations/platform-engineering-team-compliance/ OpenShift 4.22.5 security update https://access.redhat.com/errata/RHSA-2026%3A37585 GitHub Actions incident https://github.com/orgs/community/discussions/201795 Cloudflare status https://www.cloudflarestatus.com/ GitHub Code Quality GA https://github.blog/changelog/2026-07-20-github-code-quality-is-now-generally-available/ Could vs. Should — Reid Savage https://www.honeycomb.io/blog/could-should-first-year-managing-sre-team This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W30/ More episodes and full show notes https://shipitweekly.fm/
This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Mat Ryer of Grafana Labs about AI observability, production agents, evals, telemetry cost, guardrails, and what changes once AI moves beyond demos and into systems teams actually depend on. Mat is Senior Director of AI at Grafana Labs, where he focuses on how AI fits into observability and production systems. We talk about Grafana Assistant, why AI observability is not just logs, latency, and HTTP 200s, and how teams can measure whether agents are actually helping. Mat gets into evals, LLM-as-judge patterns, traces as a way to think about conversations, user feedback, tool choice, model changes, and the cost of collecting new telemetry. We also dig into UX and trust. If an AI assistant gives you a wall of text, you still have to decide whether to believe it. If it can show the graph, deep link into Grafana, apply filters, and expose the source data, that becomes a much more useful operating experience. The big takeaway: start small, enhance workflows you already have, build feedback loops, and treat production AI like something you actually have to operate. Highlights • Why AI demos are easy, but production AI is harder • Why agents need observability, evals, and guardrails • How Grafana thinks about AI Assistant and AI observability • Where LLM-as-judge patterns, traces, tool calls, and feedback fit • Why telemetry cost problems may repeat with AI workloads • Why UX matters when operators need to trust the answer • Where AI can help SRE and platform teams today Links Mat Ryer on LinkedIn: https://www.linkedin.com/in/matryer/ Mat Ryer on GitHub: https://github.com/matryer Grafana Labs: https://grafana.com Grafana Assistant: https://grafana.com/products/cloud/ai-assistant/ Grafana AI Observability: https://grafana.com/docs/grafana-cloud/machine-learning/ai-observability/ Grafana Adaptive Telemetry: https://grafana.com/products/cloud/adaptive-telemetry/ Grafana MCP server: https://github.com/grafana/mcp-grafana OpenTelemetry: https://opentelemetry.io Prometheus: https://prometheus.io Grafana Loki: https://grafana.com/docs/loki/latest/ More episodes and show notes: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This week on Ship It Weekly : Datadog tracked coordinated GitHub API enumeration, xAI’s Grok Build CLI reportedly uploaded repo data without redaction, AWS Security Hub added Network Scanning and exposure impact analysis, and Microsoft says AI-powered vulnerability discovery is changing patch pressure. The theme: visibility is not ownership. A GitHub API map does not revoke a token. An exposure finding does not close a port. A patch bulletin does not patch the fleet. And an AI coding tool reading your repo is still access. Brian covers GitHub as a production surface, AI coding tool data boundaries, cloud exposure based on reachability and blast radius, and why patching needs to look more like production operations than spreadsheet theater. Also, the Ship It Weekly shop is open at shop.tellerstech.com with Ship It Weekly t-shirt designs. Use coupon code SHIPTHESTORE for 20% off your order for the next few weeks. Links Datadog: Coordinated GitHub API enumeration https://securitylabs.datadoghq.com/articles/coordinated-github-api-enumeration/ The Verge: Grok Build CLI repository upload report https://www.theverge.com/ai-artificial-intelligence/965600/spacexai-grok-build-repository-upload AWS Security Hub Network Scanning https://aws.amazon.com/about-aws/whats-new/2026/07/aws-security-hub-network-scanning/ AWS Security Hub impact analysis for exposure findings https://aws.amazon.com/about-aws/whats-new/2026/07/impact-analysis-aws-security-hub/ Microsoft: Windows vulnerability management and AI-powered discovery https://blogs.windows.com/windowsexperience/2026/07/09/evolving-windows-vulnerability-management-to-meet-the-speed-of-ai-powered-discovery/ SRE Weekly Issue 525 https://sreweekly.com/sre-weekly-issue-525/ HalluSquatting / hallucinated package risk https://www.endorlabs.com/learn/slopsquatting-when-ai-agents-hallucinate-malicious-packages AWS Lambda Managed Instances for Java cold starts https://aws.amazon.com/blogs/compute/eliminating-java-cold-starts-with-aws-lambda-managed-instances/ This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W29/ Ship It Weekly shop https://shop.tellerstech.com/ More episodes and full show notes https://shipitweekly.fm/
This week on Ship It Weekly : Amazon EKS added Kubernetes version rollbacks, Novee Security published Cordyceps research on GitHub Actions supply chain risk, Tenet Security showed how fake telemetry can hijack AI coding agents, and Amazon CloudWatch added alarms directly from log queries. The theme: safety nets are getting better, but the blast radius is getting wider. Rollback buttons, log alarms, zone-aware routing, secret scanning, and AI agent workflows all help, but they do not replace ownership. Brian covers why EKS rollbacks are useful but not a substitute for real upgrade discipline, why GitHub Actions YAML is production code with credentials, how fake Sentry telemetry can become hostile agent context, and why easier log-based alarms can also mean easier pager noise. In the lightning round: ECS Service Connect zone-aware routing, etcd 3.7, GitHub innersource advisories, secret scanning metadata improvements, and CloudWatch Application Signals service events. Links Amazon EKS Kubernetes version rollbacks https://aws.amazon.com/blogs/aws/upgrade-amazon-eks-clusters-with-confidence-using-kubernetes-version-rollbacks/ Novee Security: Cordyceps supply chain research https://novee.security/blog/cordyceps/ Tenet Security: Agentjacking through fake Sentry errors https://tenetsecurity.ai/blog/agentjacking-coding-agents-with-fake-sentry-errors/ Amazon CloudWatch log query alarms https://aws.amazon.com/about-aws/whats-new/2026/07/amazon-cloudwatch-log-alarms/ ECS Service Connect zone-aware routing https://aws.amazon.com/about-aws/whats-new/2026/07/ecs-service-connect-zone-aware/ etcd 3.7 announcement https://etcd.io/blog/2026/announcing-etcd-3.7/ GitHub innersource security advisories https://github.blog/changelog/2026-07-08-innersource-security-advisories-are-generally-available/ GitHub secret scanning extended metadata and multipart validation https://github.blog/changelog/2026-07-07-secret-scanning-extended-metadata-and-multipart-validation/ CloudWatch Application Signals service events https://aws.amazon.com/about-aws/whats-new/2026/06/cloudwatch-service-events/ Our Links This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W28/ More episodes and full show notes https://shipitweekly.fm/
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It: Conversations episode, I talk with Evan Phoenix of Miren about why deployment is still painful, what teams keep getting wrong when they try to simplify it, and why small teams may need better defaults more than more platform knobs. Evan is the CEO of Miren. He previously worked on Terraform Enterprise and Waypoint at HashiCorp, and he also built Puma and Rubinius. We talk about deployment as the “final boss” of software delivery. Not because teams do not know how to ship code, but because deployment is where everything collides: runtimes, registries, secrets, networking, cloud services, databases, rollbacks, and the internal platform nobody wants to touch anymore. A big theme is opinionated tooling. Engineers often say they want flexibility, but many teams are really asking for good defaults, a clear happy path, and fewer decisions to own. We also get into Terraform Enterprise, Terraform Cloud, Terragrunt, OpenTofu, Waypoint, Kubernetes, Heroku, ECS, container registries, and how AI changes the deployment conversation. AI can generate infrastructure code, but when that setup breaks, someone still has to understand it and be on call for it. Highlights • Why deployment is still painful after years of platforms and abstractions • What Evan learned from Terraform Enterprise and Waypoint • Why Terraform structure, state, modules, and repo layout remain hard • Why OpenTofu gained traction beyond the Terraform licensing change • Why Kubernetes can be too much surface area for some teams • What small teams actually need from deployment tooling • How AI changes infrastructure and deployment workflows • Why generated infrastructure still needs ownership and accountability Links • Miren: https://miren.dev • Miren on GitHub: https://github.com/mirendev • Evan Phoenix: https://evanphx.dev • Evan on Bluesky: https://bsky.app/profile/evanphx.dev • Evan on Linkedin: https://www.linkedin.com/in/evanphoenix/ Things mentioned • Terraform Enterprise: https://developer.hashicorp.com/terraform/enterprise • Terraform Cloud: https://developer.hashicorp.com/terraform/cloud-docs • Terragrunt: https://terragrunt.gruntwork.io • OpenTofu: https://opentofu.org • HashiCorp Waypoint: https://github.com/hashicorp/waypoint • Knative: https://knative.dev Our links More episodes + show notes: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This week on Ship It Weekly : Amazon Q Developer and the AWS language servers had a pair of trust-boundary CVEs, JFrog found hijacked npm and Go packages using hidden VS Code tasks to run malware when a workspace opens, AWS WAF had HTTP/2 request-body inspection issues, and AWS introduced Lambda MicroVMs for running user-generated and AI-generated code in isolated sandboxes. The bigger theme: execution is the boundary now. The repo, the IDE, the AI assistant, the WAF, and the sandbox all sit at the point where something gets to run, inspect, block, or decide. Before execution, trust is a policy. After execution, trust is a blast radius. In the lightning round, Brian covers GitHub’s record advisory volume, Git 2.55, Valkey 9.1 on Amazon ElastiCache, and a quick Fable 5 callback now that Anthropic’s Fable 5 is back online. Links AWS security bulletin: Amazon Q / AWS language server CVEs https://aws.amazon.com/security/security-bulletins/2026-047-aws/ JFrog: Hijacked npm packages using VS Code tasks https://research.jfrog.com/post/hijacked-npm-vscode-tasks-blockchain/ AWS security bulletin: AWS WAF HTTP/2 inspection issues https://aws.amazon.com/security/security-bulletins/2026-048-aws/ AWS Lambda MicroVMs https://aws.amazon.com/blogs/aws/run-isolated-sandboxes-with-full-lifecycle-control-aws-lambda-introduces-microvms/ GitHub Advisory Database record volume https://github.blog/security/supply-chain-security/inside-the-advisory-database-and-what-happens-when-vulnerability-volume-breaks-records/ Git 2.55 highlights https://github.blog/open-source/git/highlights-from-git-2-55/ Amazon ElastiCache Valkey 9.1 https://aws.amazon.com/blogs/database/announcing-valkey-9-1-for-amazon-elasticache/ Claude Fable 5 and Mythos 5 model docs https://platform.claude.com/docs/en/about-claude/models/introducing-claude-fable-5-and-claude-mythos-5 This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W27/ More episodes and full show notes https://shipitweekly.fm/
This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It: Conversations episode, I talk with Kat Traxler of Vectra AI about AI security, the zero-day clock, IAM, cloud risk, AI-assisted bug hunting, and why the scariest future security problems may still start with the boring fundamentals teams already struggle with today. Kat is a Principal Security Researcher at Vectra AI focused on abuse techniques and vulnerabilities in the public cloud, especially around the intersection of cloud security, AppSec, IAM, managed identities, and insecure-by-design flaws. We talk about the current AI security mood, from the excitement around faster research and bug hunting to the fear that AI could shrink the window between vulnerability disclosure and exploitation. Kat explains the “San Francisco Consensus,” why the zero-day clock is getting so much attention, and why she thinks the facts may be real while some of the conclusions are overextended. The bigger theme here is that AI is absolutely changing security work, but it does not erase the fundamentals. Attackers still take the lowest-friction path that works. For most teams, that still means credentials, IAM, misconfigurations, known vulnerabilities, and systems that were never threat-modeled as deeply as people assume. Highlights • Why AI security feels exciting and unsettling at the same time • What the “San Francisco Consensus” means and why people are talking about the zero-day clock • How AI may shrink the time between vulnerability disclosure and exploitation • Why Kat is skeptical of the full “zero-day apocalypse” narrative • Why credentials, IAM, misconfigurations, and known vulnerabilities still matter most for many teams • How AI helps narrow the search space in bug hunting and security research • Where AI is useful for code-level bugs, and where it still struggles with context and threat modeling • Why human expertise still matters when using AI for writing, research, and cloud security analysis • Why IAM remains hard because it sits at the intersection of people, access, and technology • What insecure-by-design flaws are, and why AI may not solve those anytime soon Kat / Vectra AI links • Kat Traxler at Vectra AI: https://www.vectra.ai/about/author/kat-traxler • Kat’s site: https://kattraxler.cloud/ • The San Francisco Consensus: https://kattraxler.cloud/the-san-francisco-consensus/ • Kat on X: https://x.com/NightmareJS • Vectra AI: https://www.vectra.ai/ Our links More episodes + show notes + links: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This week on Ship It Weekly : containerd disclosed a batch of CRI plugin vulnerabilities, Datadog tested PostgreSQL high availability on Kubernetes and found that failover is not useful if it cannot happen safely, AWS DevOps Agent and Datadog MCP Server moved AI incident response closer to real production workflows, and Amazon EKS added customer-routed control-plane egress. The bigger theme: the control plane keeps getting wider. Runtimes, databases, incident agents, API-server egress, credentials, the cloud console, and object metadata are all becoming part of the production blast radius. And when something breaks, users do not experience your architecture diagram. They experience waiting. In the lightning round, Brian covers GitHub self-service credential revocation for incident response, AWS Management Console Private Access without internet connectivity, Vercel Connect and short-lived agent credentials, and Amazon S3 annotations. Links containerd CRI plugin vulnerabilities / AWS security bulletin https://aws.amazon.com/security/security-bulletins/2026-046-aws/ Datadog: PostgreSQL high availability on Kubernetes https://www.datadoghq.com/blog/engineering/postgresql-ha-kubernetes/ AWS DevOps Agent and Datadog MCP Server https://aws.amazon.com/blogs/devops/production-ready-autonomous-incident-resolution-with-aws-devops-agent-now-ga-and-datadog-mcp-server/ Amazon EKS customer-routed control-plane egress https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-control-plane-egress-through-your-vpc/ GitHub self-service credential revocation for incident response https://github.blog/changelog/2026-06-24-self-service-credential-revocation-for-incident-response/ AWS Management Console Private Access https://aws.amazon.com/about-aws/whats-new/2026/06/aws-management-console-private/ Vercel Connect https://vercel.com/blog/introducing-vercel-connect Amazon S3 annotations https://aws.amazon.com/blogs/aws/amazon-s3-annotations-attach-rich-queryable-context-directly-to-your-objects/ Marc Brooker: Waiting, latency, MTTR, and the inspection paradox https://brooker.co.za/blog/2026/06/19/waiting.html This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W26/ More episodes and full show notes https://www.shipitweekly.fm
This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It: Conversations episode, I talk with Joel DeStefano from Guardsquare about mobile app security, why it is different from backend and cloud security, and why scanning alone is not enough once an app is shipped into the real world. We talk about the shift in trust model that happens with mobile apps. In backend and cloud systems, teams usually have more control over the runtime, infrastructure, policies, and monitoring. With mobile, the app becomes a public artifact running on someone else’s device, in an environment you do not fully control. The bigger theme here is that mobile security is not just “scan it before release.” Scanning matters, but teams also need to think about app hardening, obfuscation, runtime protection, monitoring, and whether the app connecting back to their APIs is genuine and uncompromised. Highlights • Why mobile changes the trust model compared to backend and cloud systems • What DevOps, SRE, and platform teams should understand about mobile app risk • Why scanning is useful, but not enough by itself • The danger of assuming app store approval means an app is secure • Why “we do not store sensitive data in the app” can be a misleading security argument • How attackers can reverse engineer apps, inspect workflows, and learn how the app talks to backend APIs • What code hardening and obfuscation actually help protect against • Why runtime checks matter for rooted devices, compromised environments, debuggers, hooking frameworks, overlays, and accessibility abuse • The difference between Android and iOS security assumptions • Why the OS is not responsible for protecting your app’s business logic • How mobile security should fit into CI/CD without destroying release velocity • What should block a release versus what should become tracked risk • Why testing, hardening, runtime protection, and monitoring should work together as one strategy • How AI may speed up attackers without fundamentally changing the need for strong security fundamentals • Joel’s advice for improving mobile security posture: start with the app’s critical workflows, backend interactions, and real business risk Joel / Guardsquare links • Guardsquare: https://hubs.ly/Q04fJgkJ0 • Guardsquare Blog: https://www.guardsquare.com/blog OWASP mobile security links • OWASP Mobile Application Security: https://owasp.org/www-project-mobile-app-security/ • OWASP MASVS: https://mas.owasp.org/MASVS/ Our links More episodes + show notes + links: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This episode of Ship It Weekly is about default trust getting punished. Brian covers Oracle’s emergency PeopleSoft advisory for CVE-2026-35273, npm v12 changing install-script defaults, GitHub Agentic Workflows moving away from long-lived personal access tokens, and Anthropic disabling Fable 5 and Mythos 5 after a U.S. export-control directive. The common thread: legacy ERP systems, package installs, CI/CD agents, and AI models all become production risks when teams trust the default without checking what that trust can actually do. In the lightning round, Brian covers Tekton CloudEvents moving to a dedicated events controller, NVIDIA Triton Inference Server 26.04 changing inference defaults, AWS Nitro Isolation Engine bringing formal verification to Graviton5-based isolation, and Homebrew 6.0 adding explicit trust for third-party taps. The bigger theme: production does not care why you trusted the default. It only cares what that default was allowed to do. The bigger theme: production does not care why you trusted the default. It only cares what that default was allowed to do. Links Oracle PeopleSoft CVE-2026-35273 advisory https://www.oracle.com/security-alerts/alert-cve-2026-35273.html npm v12 breaking changes https://github.blog/changelog/2026-06-09-upcoming-breaking-changes-for-npm-v12/ GitHub Agentic Workflows no longer need PATs https://github.blog/changelog/2026-06-11-agentic-workflows-no-longer-need-a-personal-access-token/ Anthropic Fable 5 / Mythos 5 access statement https://www.anthropic.com/news/fable-mythos-access Tekton Pipelines releases https://github.com/tektoncd/pipeline/releases NVIDIA Triton Inference Server 26.04 release notes https://docs.nvidia.com/deeplearning/triton-inference-server/release-notes/rel-26-04.html AWS Nitro Isolation Engine https://aws.amazon.com/blogs/compute/aws-nitro-isolation-engine-formally-verifying-the-hypervisor-in-the-aws-nitro-system/ Homebrew 6.0.0 https://brew.sh/2026/06/11/homebrew-6.0.0/ This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W25/ More episodes and show notes https://shipitweekly.fm/
This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It: Conversations episode, I talk with Francois Richard, Engineering Director at Meta, about reliability at scale, how AI is changing production risk, what teams actually learn from incidents, and why recovery practice matters just as much as prevention. We talk about the proactive and reactive sides of reliability, why SLOs should represent a promise to users instead of just another dashboard number, how incident reviews should drive real system improvements, and how teams can practice recovery before production forces the lesson on them. The bigger theme here is that reliability is not just about avoiding failure. It is about knowing what happens when prevention fails. That means practicing regional failure, understanding overload behavior, improving incident response, using AI carefully during investigation, and making reliability targets match the actual lifecycle and importance of the system. Highlights • Why reliability work starts with both prevention and recovery • The difference between reactive incident response and proactive reliability engineering • How Meta thinks about disaster recovery testing and regional failure practice • Why an SLO should be treated like a promise to users, not just a dashboard metric • How SLO trends help teams decide when to invest more in reliability or take more product risk • What engineers actually learn during the “pressure cooker” of an incident • Why incident reviews should produce follow-up work, not just a nicer explanation of what broke • The difference between finding the cause of an incident and improving the system • Where AI agents can help with incident investigation, telemetry, metrics, and query building • Why AI-generated code can increase change volume while reducing human context • How faster code generation changes the kinds of reliability problems teams should expect • Why recovery practice matters, especially for region loss, traffic spikes, overload, and restart behavior • What smaller DevOps and SRE teams can learn from Meta-scale reliability patterns • Why not every system needs six nines, especially early in a product lifecycle • How to think about reliability investment based on user promise, product maturity, and operational risk • Why At Scale Systems & Reliability is focused on the infrastructure behind AI and the use of AI to operate large-scale systems Francois’ links • LinkedIn: https://www.linkedin.com/in/francoisrichard/ At Scale links • Systems & Reliability 2026: https://bit.ly/4xd2FdG • At Scale Conferences: https://atscaleconference.com/ Our links More episodes + show notes + links: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This episode of Ship It Weekly is about the hidden glue holding production together. Brian covers Coinbase’s May 7 outage postmortem, where an AWS us-east-1 cooling failure exposed the difference between being “multi-AZ” on paper and actually being able to recover when stateful, low-latency systems are tied to a failed zone. Then he looks at Meta’s AI-assisted Instagram support issue and why account recovery is identity infrastructure, not just customer support. If AI can influence password resets, email changes, MFA resets, or account ownership flows, that workflow needs to be treated like a production control plane. The episode also covers AWS AgentCore CLI CVE-2026-11393, where collaborator metadata could break out into generated Python code during agent import, and an Apigee cross-tenant issue from Google’s Apigee security bulletins that shows why tenant isolation has to be tested beyond the obvious happy path. Links Coinbase May 7 outage postmortem https://www.coinbase.com/blog/a-postmortem-of-our-may-7-2026-outage Meta AI support / Instagram account recovery reporting https://www.theverge.com/tech/945658/meta-ai-support-chatbot-exploit-instagram-accounts AWS AgentCore CLI CVE-2026-11393 https://aws.amazon.com/security/security-bulletins/2026-040-aws/ AgentCore CLI GitHub advisory https://github.com/aws/agentcore-cli/security/advisories/GHSA-m4x6-gwgp-4pm7 Google Apigee security bulletins https://docs.cloud.google.com/apigee/docs/security-bulletins/security-bulletins Cloudflare real-time threat intel WAF rules https://blog.cloudflare.com/realtime-threat-intel-waf-rules/ AWS Lambda tenant isolation with event source mappings https://aws.amazon.com/blogs/compute/integrating-event-source-mappings-with-aws-lambda-tenant-isolation-mode/ Amazon OpenSearch Serverless next generation https://aws.amazon.com/about-aws/whats-new/2026/05/amazon-opensearch-serverless-next-generation-generally-available/ GitHub Enterprise Managed Users IP allow list coverage https://github.blog/changelog/2026-06-08-ip-allow-list-coverage-for-emu-namespaces-in-general-availability/ This week’s On Call Brief https://www.tellerstech.com/on-call-brief-news/2026-W24/ More episodes and show notes https://shipitweekly.fm/
This episode of Ship It Weekly is about automation’s hidden boundaries. Brian covers Kiro CLI CVE-2026-9255, where piped stdin could act like user approval, Amazon Braket SDK CVE-2026-9291 and the very normal Python pickle risk hiding inside quantum job results, AWS Organizations finally emitting CloudTrail events when accounts join or leave an org, and KEDA updates that remind us autoscaling upgrades are production behavior changes. The bigger thread this week is that automation does not remove boundaries. It moves them. Approval paths, trusted data, account membership, scaling signals, platform access, and AI-generated output all need clear ownership and visibility. Brian also covers Kubernetes Dashboard being archived with Headlamp as the path forward, Google Cloud Remote MCP Server for AlloyDB, Apache Kafka 4.3.0, and Atlassian’s AI-native SDLC productivity claims. Sponsored by @Scale: Systems & Reliability, happening June 25 at the Meydenbauer Center in Bellevue, Washington. Register at https://bit.ly/4xd2FdG Links Kiro CLI CVE-2026-9255 https://aws.amazon.com/security/security-bulletins/2026-035-aws/ Amazon Braket SDK CVE-2026-9291 https://aws.amazon.com/security/security-bulletins/2026-036-aws/ AWS Organizations CloudTrail account events https://aws.amazon.com/about-aws/whats-new/2026/05/aws-organizations-cloudtrail/ KEDA v2.20.0 release https://github.com/kedacore/keda/releases/tag/v2.20.0 KEDA v2.19.0 release https://github.com/kedacore/keda/releases/tag/v2.19.0 Kubernetes Dashboard archived / Headlamp path forward https://kubernetes.io/blog/2026/06/04/dashboard-archived-what-now/ Google Cloud Remote MCP Server for AlloyDB https://cloud.google.com/blog/products/databases/alloydb-remote-mcp-server-now-ga Apache Kafka 4.3.0 https://www.confluent.io/blog/apache-kafka-4-3-release-announcement/ Atlassian AI-native SDLC productivity claims https://www.atlassian.com/blog/software-teams/ai-native-sdlc This week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W23/ More episodes and show notes https://shipitweekly.fm/
This episode of Ship It Weekly is about trusted tools becoming production dependencies. Brian covers a rough GitHub supply chain week, including the compromised Nx Console VS Code extension tied to exposed GitHub internal repositories and the Megalodon campaign abusing GitHub Actions workflows across thousands of public repos. The bigger thread this week is that the tools around production are increasingly part of production. Brian also covers Railway’s GCP account suspension outage, Discord’s voice outage during a Kubernetes migration, AWS changing SDK retry behavior, CVE-2026-9133 in the RabbitMQ AWS plugin, and a Reddit story about stolen AWS keys turning into a $14,000 Bedrock bill. Brian also touches on OpenTelemetry graduating from the CNCF, Claude Code security risk, GitLab Secrets Manager, Google Cloud AI spend caps, and a Redshift Python driver RCE. Full source list and extra links are available on this episode’s page at shipitweekly.fm . Links Nx Console compromise https://www.stepsecurity.io/blog/nx-console-vs-code-extension-compromised Megalodon GitHub Actions attack https://www.stepsecurity.io/blog/megalodon-mass-github-actions-secret-exfiltration-across-5-500-public-repositories Railway GCP outage https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outage Discord voice outage https://discord.com/blog/behind-the-scenes-of-the-3-25-26-voice-outage AWS SDK retry changes https://aws.amazon.com/blogs/developer/announcing-updated-retry-behavior-for-aws-sdks-and-tools/ RabbitMQ AWS plugin CVE-2026-9133 https://aws.amazon.com/security/security-bulletins/2026-034-aws/ AWS Bedrock cost spike Reddit thread https://www.reddit.com/r/aws/comments/1tm3ydo/aws_bedrock_cost_spike_14000_usd/ This week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W22/ More episodes and show notes https://shipitweekly.fm/
This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It: Conversations episode, I talk with Jake Warner, founder and CEO of Cycle.io, about private cloud, bare metal, Kubernetes fatigue, and why some teams are rethinking how much infrastructure complexity they actually want to carry. We talk about why bare metal and private cloud are getting interesting again, especially around cost, performance, data sovereignty, compliance, and platform ownership. Jake explains how Cycle approaches infrastructure as a pool of resources, why he thinks in terms of “environments as code” instead of traditional infrastructure as code, and how teams can run containers and VMs together across bare metal, cloud, and hybrid environments. The bigger theme here is that this is not really a “cloud versus bare metal” conversation. It is about choosing the right level of abstraction. Sometimes Kubernetes is the right answer. Sometimes managed cloud services make sense. And sometimes teams just need a more opinionated platform that lets developers ship without requiring a large DevOps army to keep everything running. Highlights • Why some teams are moving back toward private cloud and bare metal • The role of cost, data sovereignty, compliance, and performance in infrastructure decisions • Why bare metal does not have to mean going back to old-school racking and stacking pain • How Cycle turns raw compute into a private cloud-style resource pool • Why Jake thinks about “environments as code” instead of only infrastructure as code • What “no DevOps army required” means in practice for engineering-heavy teams • Why some companies need VMs and containers running together on the same platform • Where Kubernetes still makes sense, especially for highly customized infrastructure needs • Why opinionated platforms can be valuable when teams want fewer knobs and better defaults • Active-active thinking, failover risk, and why application-level replication often matters more than platform-level storage magic • Why bandwidth, performance density, and predictable pricing can make bare metal attractive again • The weird continued gravity of AWS us-east-1, even for teams trying to move workloads elsewhere • How AI workloads, GPUs, and hype cycles fit into the private cloud and platform conversation • Jake’s advice for modernizing hybrid or on-prem infrastructure: containerize first, then look hard at your dependencies Jake’s links • Cycle.io : https://cycle.io/ • Cycle Slack community: https://slack.cycle.io/ • Jake Warner on LinkedIn: https://www.linkedin.com/in/jakewarner/ Our links More episodes + show notes + links: https://shipitweekly.fm On Call Brief: https://oncallbrief.com
This episode of Ship It Weekly is about secrets, agents, risky defaults, and follow-up work that never gets done. Brian covers the CISA contractor GitHub leak involving AWS keys, internal docs, Terraform, Kubernetes, Argo CD, and CI/CD context, plus AWS DevOps Agent doing automated RCA across Datadog, Elasticsearch, CloudTrail, and EKS. Brian also covers MS Copilot Studio computer-using agents, Claude Code in Bitbucket Agentic Pipelines, CVE-2026-46333 and Kubernetes seccomp defaults, GitHub OIDC for Dependabot, Java pods getting OOMKilled, LLM-generated SQL that can be wrong but still run, and why postmortem action items die without ownership. Sponsored by Guardsquare https://hubs.ly/Q04fJgkJ0 Links CISA GitHub leak https://blog.gitguardian.com/how-we-got-a-cisa-github-leak-taken-down-in-26-hours/ AWS DevOps Agent RCA https://aws.amazon.com/blogs/devops/automate-root-cause-analysis-across-datadog-and-elasticsearch-with-aws-devops-agent/ Microsoft Copilot Studio computer-using agents https://techcommunity.microsoft.com/blog/copilot-studio-blog/computer-using-agents-in-microsoft-copilot-studio-are-now-generally-available/4519427 Atlassian Agentic Pipelines with Claude Code https://support.atlassian.com/bitbucket-cloud/docs/agentic-pipelines/ CVE-2026-46333 https://nvd.nist.gov/vuln/detail/CVE-2026-46333 Kubernetes seccomp https://kubernetes.io/docs/reference/node/seccomp/ GitHub OIDC for Dependabot and code scanning https://github.blog/changelog/2026-05-19-expanded-oidc-support-for-dependabot-and-code-scanning/ Java pods OOMKilled in Kubernetes https://dzone.com/articles/java-pod-oomkill-kubernetes LLM-generated SQL risks https://readyset.io/blog/why-llms-write-incorrect-sql-and-what-that-means-for-your-database Postmortem action items https://incident.io/blog/why-do-post-mortem-action-items-fail-how-to-make-incident-follow-ups-actually-get-done On Call Brief https://www.tellerstech.com/on-call-brief/2026-W21/ More episodes + show notes https://shipitweekly.fm/
This episode of Ship It Weekly is about AI agents moving from helpful coding assistants into real operational actors. Brian covers GitHub making Copilot cloud agent tasks available through a REST API, Auth0 bringing authentication and authorization to MCP servers, Red Hat positioning Ansible as a trusted execution layer for agentic IT operations, and OpenAI Daybreak pushing AI deeper into security research and remediation. The bigger thread this week is authority: what these agents can reach, what they can change, who approved the action, and who owns the outcome when something breaks. Brian also covers Discord’s ScyllaDB automation work, AWS GuardDuty crypto mining detection, queues and back pressure, and a Datadog PostgreSQL case where an index scan was still painfully slow. Sponsored by Guardsquare https://hubs.ly/Q04fJgkJ0 Links GitHub Copilot cloud agent tasks via REST API https://github.blog/changelog/2026-05-13-start-copilot-cloud-agent-tasks-via-the-rest-api/ GitHub REST API endpoints for agent tasks https://docs.github.com/en/rest/agent-tasks/agent-tasks Auth0 Auth for MCP is now generally available https://auth0.com/blog/auth0-auth-for-mcp-servers-generally-available/ Red Hat on Ansible as the execution layer for agentic IT https://www.redhat.com/en/about/press-releases/red-hat-establishes-ansible-automation-platform-trusted-execution-layer-it-operations-agentic-era OpenAI Daybreak https://openai.com/daybreak/ Discord automates ScyllaDB clusters at scale https://discord.com/blog/how-discord-automates-scylladb-clusters-at-scale AWS GuardDuty crypto mining detection and prevention https://aws.amazon.com/blogs/security/detecting-and-preventing-crypto-mining-in-your-aws-environment/ Queues do not absorb load, they delay failure https://dzone.com/articles/queues-dont-absorb-load-they-delay-bankruptcy Datadog on inefficient PostgreSQL index scans https://www.datadoghq.com/blog/detect-inefficient-index-scans-with-dbm/ This week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W20/ More episodes and show notes https://shipitweekly.fm/
This episode of Ship It Weekly is about modern reliability getting squeezed from both directions. Old-school failures still hit hard, like broken DNSSEC, kernel privilege escalation bugs, and GitOps behavior changes. But newer automation layers add a second kind of risk, where AI agents, machine identity, and cloud control planes can do real damage fast when authority is too broad. Brian covers the Cursor and PocketOS production database wipe, the .de DNSSEC outage and Cloudflare’s response, Bluesky’s April outage postmortem, Argo CD v3.1.16 reaching end of life plus the v3.4.1 behavior change, Linux kernel CVE-2026-31431 under active exploitation, and why Google Cloud Agent Identity and AWS MCP Server GA both point to agents becoming first-class infrastructure actors. Sponsored by Guardsquare https://hubs.ly/Q04fJgkJ0 Links Cursor / PocketOS production database wipe https://www.tellerstech.com/on-call-brief/2026-W19/ Cloudflare on the .de DNSSEC outage https://blog.cloudflare.com/de-tld-outage-dnssec/ Bluesky April 2026 outage postmortem https://pckt.blog/b/jcalabro/april-2026-outage-post-mortem-219ebg2 Argo CD releases: v3.1.16 final release and v3.4.1 behavior change https://github.com/argoproj/argo-cd/releases Linux kernel CVE-2026-31431 https://nvd.nist.gov/vuln/detail/CVE-2026-31431 AWS bulletin for CVE-2026-31431 https://aws.amazon.com/security/security-bulletins/rss/2026-026-aws/ Google Cloud Agent Identity https://cloud.google.com/blog/products/identity-security/whats-new-in-iam-security-governance-and-runtime-defense AWS MCP Server is now generally available https://aws.amazon.com/blogs/aws/the-aws-mcp-server-is-now-generally-available/ Cross-region disaster recovery for Amazon EKS using AWS Backup https://aws.amazon.com/blogs/containers/cross-region-disaster-recovery-for-amazon-eks-using-aws-backup/ Google Ads new data retention policy starting June 1, 2026 https://ads-developers.googleblog.com/2026/05/new-data-retention-policy-for-google.html This week’s On Call Brief https://www.tellerstech.com/on-call-brief/2026-W19/ More episodes and show notes https://shipitweekly.fm/
Bring this source into Mato to analyze its transferable patterns and turn them into an original show concept for your audience.
Create a show inspired by this