At KubeCon EU 2026 in Amsterdam, I sat down with Mitch Connors, a principal software engineer at Microsoft and Istio maintainer who sits on the project’s Technical Oversight Committee. Connors joined Microsoft two years ago to work on service mesh operations and recently transitioned into a PM role to ship the Azure Kubernetes Application Network, a fully managed service built on Istio’s ambient mode.
“Success for me looks like most people not knowing what a service mesh is, even though they’re using one.”
In this episode, we talk about why Microsoft built a fully managed service mesh that deliberately avoids the words “service mesh,” how Istio’s ambient mode makes that possible, and what AI workloads demand from the network layer.
Making service mesh disappear
Istio’s sidecar model worked well, but created day-two headaches, Connors says. “You run ‘helm upgrade Istio,’ none of your proxies have upgraded,” Connors tells The New Stack. “You need to go and recreate all your applications, allow rollouts to start all over again in order to take advantage of that upgrade.”
Ambient mode, which became generally available about two years, moved encryption into a lightweight per-node Rust proxy and shifted Layer 7 features into waypoint proxies that upgrade independently. No more restarting applications. But Connors says it’s still not boring enough: about 85% of ambient installations aren’t keeping up with CVE patches.
That convinced him that there’s a need for a better service mesh product. At KubeCon, Microsoft launched Azure Kubernetes Application Network, built on Istio ambient, with mutual transportation layer security (mTLS) by default across all clusters.
The name of this new project deliberately doesn’t include ‘service mesh,’ Connors says: “We’ve met a lot of customers who say, ‘I don’t need service mesh. I just need a proxy that gives me mTLS.’” So the product meets them there. “And maybe once you’ve added all those features together, you’ll realize you’re on a service mesh,” he says. “But it’s a very different experience than someone who adopted service mesh three years ago.”
AI traffic needs a different kind of network
Today’s AI workloads also put different demands on service meshes, Conners noted. Traditional HTTP routing assumes each request costs roughly the same to serve. “That’s not even remotely true with an LLM, where one request might be the perpetual ‘hi’ message being sent to our LLM, and another request might be someone asking Copilot to explain service mesh for them, and that’s going to take a little bit.”
The Istio project has spent years making its APIs stable and boring, but AI protocols like MCP have been around for barely a year. The project’s answer for this new era is a two-speed approach, partnering with Agent Gateway, a Linux Foundation project designed for agentic traffic by engineers with deep Istio experience.
“If you want cutting-edge A2A, MCP and all of the other alphabet soup of AI, you can use Agent Gateway APIs, and it’s just a clear signal that these are going to be an alpha experience,” Connors says. “We can’t promise you that they’re going to be around in the same shape in two years, but you’ve expressed interest in using the bleeding edge of technology, and so we’re going to support you in that as well.”
For inference routing, the Gateway API’s inference extension runs a small LLM as a token estimator to score request complexity up front. On the back end, token usage in the response payload feeds rate limits that Istio distributes across the cluster. App Net ships with the inference extension today; Agent Gateway integration will follow upstream.
Connors also sees a role for the mesh in AI governance. Platform teams offer approved LLM endpoints, but can’t enforce that users stick to them. “We want to actually inspect the body of the request and say, this is an LLM request. It needs to be going to an approved LLM service.”
Multi-cluster and GPU scarcity
Ambient multi-cluster support has been the biggest upstream development in the past year — and a prerequisite for App Net. Without a consistent root of trust across clusters, “any traffic that goes between clusters effectively just bypasses all of the network safety that you’ve set up,” Connors notes.
AI intensifies this. GPU capacity is often available but not in the right region. The mesh lets teams move GPU-heavy workloads to where capacity exists while maintaining service-to-service communication.
Sixty percent of Kubernetes clusters still don’t run any service mesh, something Connors is quite aware of. “We earned the reputation for being very complicated,” he tells The New Stack. “We were very complicated. And I don’t blame a user who tried service mesh five years ago from saying, ‘I don’t want anything with the word Istio put on my cluster at any point in time.’ Trust is hard to earn back, but we’re working to do that. And we’re also working towards an audience that maybe doesn’t have the cloud-native vocabulary to express their needs as a service mesh, but understands them much more from a tactile business perspective.”
The bet with App Net is that making the mesh invisible and managed brings those holdouts in for mTLS alone — just as AI starts demanding more from the network layer than anyone expected.
YOUTUBE.COM/THENEWSTACK
Tech moves fast, don’t miss an episode. Subscribe to our YouTube
channel to stream all our podcasts, interviews, demos, and more.
SUBSCRIBE
Group
Created with Sketch.
Before joining The New Stack as its senior editor for AI, Frederic was the enterprise editor at TechCrunch, where he covered everything from the rise of the cloud and the earliest days of Kubernetes to the advent of quantum computing….
Read more from Frederic Lardinois
