Self-Managed Envoy Gateway on AKS and NGINX Migration

This guide combines installation and Application Routing NGINX migration into one self-managed Envoy Gateway runbook. It is distinct from the AKS-managed, Istio-based Application Routing implementation. The source uses Envoy Gateway v1.7.0 and an illustrative v1.8.0 upgrade. It cites March 2026 upstream Ingress NGINX retirement and Microsoft support for the NGINX Application Routing add-on through November 2026. Verify lifecycle and release information before choosing a migration deadline.

1. Architecture, ownership, and trade-offs

Helm installs the control plane, which reconciles configuration but does not serve application traffic. Creating a Gateway provisions an Envoy data-plane Deployment, a dedicated LoadBalancer Service, and an HPA when configured. The resource chain is EnvoyProxy → GatewayClass → Gateway → HTTPRoute.

• Benefits: Gateway API's separation of responsibilities, Envoy's established proxy implementation, independently scalable tenant gateways, and public, private, or Private Link Service deployment patterns.
• Costs: the operator owns Helm upgrades, security patches, CRDs, monitoring and incident response. New concepts include EnvoyProxy, GatewayClass, Gateway, HTTPRoute and policy CRDs.
• Compatibility is not guaranteed merely by choosing Gateway API. Check the implementation/version feature matrix, especially policy extensions and converted annotations.
• The source describes an interim move to the managed NGINX add-on and a longer-term Envoy strategy. That is a planning pattern, not an assertion that self-managed Envoy is the only Microsoft-supported destination.

2. Create an isolated AKS lab

RESOURCE_GROUP="YOUR_RESOURCE_GROUP"
CLUSTER_NAME="YOUR_CLUSTER"
LOCATION="eastus"
VM_SKU="Standard_D2a_v4"
NODE_COUNT=2
az group create --name "$RESOURCE_GROUP" --location "$LOCATION"
az aks create --resource-group "$RESOURCE_GROUP" --name "$CLUSTER_NAME" \
  --location "$LOCATION" --node-count "$NODE_COUNT" --node-vm-size "$VM_SKU" \
  --network-plugin azure --network-plugin-mode overlay \
  --generate-ssh-keys --enable-managed-identity
az aks get-credentials --resource-group "$RESOURCE_GROUP" --name "$CLUSTER_NAME" --overwrite-existing
kubectl get nodes
kubectl cluster-info

The source baseline is two Standard_D2a_v4 nodes (2 vCPU, 8 GB RAM), a system pool without a taint, Azure CNI Overlay and managed identity. No Kubernetes version is pinned, so the service chooses an available default; production should select and validate a supported version deliberately. Importing credentials with overwrite-existing changes the local kubeconfig entry.

3. Install and verify the control plane

EG_VERSION="v1.7.0"
helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version "$EG_VERSION" -n envoy-gateway-system --create-namespace \
  --set podDisruptionBudget.minAvailable=1 \
  --set deployment.replicas=2 \
  --set deployment.envoyGateway.resources.requests.cpu=125m \
  --set deployment.envoyGateway.resources.requests.memory=128Mi \
  --set deployment.envoyGateway.resources.limits.cpu=500m \
  --set deployment.envoyGateway.resources.limits.memory=512Mi
kubectl wait deployment envoy-gateway -n envoy-gateway-system \
  --for=condition=Available=True --timeout=5m
kubectl get pods -n envoy-gateway-system

The example uses two fixed control-plane replicas with PDB minAvailable=1, CPU request/limit 125m/500m and memory request/limit 128Mi/512Mi. It does not configure control-plane HPA because application traffic is handled elsewhere. This is the source's sizing baseline, not a universal capacity recommendation.

Upgrade CRDs before the controller

Helm does not automatically upgrade chart CRDs. Back up existing resources, read release and schema migration notes, and update CRDs before upgrading the controller. Never delete a CRD simply because one of its served API versions was deprecated: deleting the CRD deletes all instances.

# Illustrative upgrade; confirm the target release exists and supports your cluster.
helm pull oci://docker.io/envoyproxy/gateway-helm --version v1.8.0 --untar
kubectl apply -f gateway-helm/crds/
kubectl get crd | grep -E "gateway|envoy"
helm upgrade eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.8.0 -n envoy-gateway-system \
  --set podDisruptionBudget.minAvailable=1 \
  --set deployment.replicas=2 \
  --set deployment.envoyGateway.resources.requests.cpu=125m \
  --set deployment.envoyGateway.resources.requests.memory=128Mi \
  --set deployment.envoyGateway.resources.limits.cpu=500m \
  --set deployment.envoyGateway.resources.limits.memory=512Mi
kubectl wait deployment envoy-gateway -n envoy-gateway-system \
  --for=condition=Available=True --timeout=5m
kubectl get pods -n envoy-gateway-system
# Remove only the chart directory downloaded for this operation, if no longer needed.
rm -r gateway-helm

kubectl apply does not remove obsolete CRDs. Inventory deprecated resources and follow the release-specific migration process. The source mentions GRPCRoute/ReferenceGrant v1alpha2 removals around v1.2.0; treat this as a reminder to inspect served/storage versions, not as a safe blanket CRD-deletion command. The upgrade above explicitly preserves two replicas, correcting an omission in the source's otherwise equivalent Helm options.

4. Configure an internal Gateway with draining and autoscaling

Apply these three resources in order. The EnvoyProxy lives in the infrastructure namespace; the Gateway lives in default. For this deployment model the generated data-plane Pods and Services are in envoy-gateway-system.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: internal-proxy
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: LoadBalancer
        annotations:
          service.beta.kubernetes.io/azure-load-balancer-internal: "true"
      envoyDeployment:
        replicas: 2
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                spec:
                  terminationGracePeriodSeconds: 300
      envoyHpa:
        minReplicas: 2
        maxReplicas: 10
        metrics:
        - type: Resource
          resource:
            name: cpu
            target:
              type: Utilization
              averageUtilization: 80
  shutdown:
    drainTimeout: 120s
    minDrainDuration: 5s
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg-private
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: internal-proxy
    namespace: envoy-gateway-system
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg-private
  namespace: default
spec:
  gatewayClassName: eg-private
  listeners:
  - name: http
    protocol: HTTP
    port: 80

kubectl get envoyproxy -n envoy-gateway-system
kubectl get gatewayclass
kubectl get gateway
kubectl get svc -n envoy-gateway-system \
  -l gateway.envoyproxy.io/owning-gateway-name=eg-private

Why all three shutdown settings matter

After SIGTERM, Envoy waits at least the minimum drain interval and up to the drain timeout for requests to finish. Kubernetes' grace period must exceed the drain timeout so that the proxy is not killed first. Without suitable draining, rolling updates can terminate active requests and surface 502s. Long-running uploads and WebSockets need workload-specific settings; no timeout guarantees zero loss.

The built-in envoyHpa produces an independent HPA per Gateway with min 2, max 10 and CPU utilization target 80%; a separate HPA manifest is unnecessary. Platform teams can offer several EnvoyProxy/GatewayClass chains: an API pattern with 30s drain/60s grace, a streaming pattern with 120s/300s, and an upload pattern with 180s/600s. Application teams select the class rather than changing shared infrastructure.

5. Deploy a test application and route

apiVersion: v1
kind: ServiceAccount
metadata:
  name: aks-helloworld
  namespace: default
---
apiVersion: v1
kind: Service
metadata:
  name: aks-helloworld
  namespace: default
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector: {app: aks-helloworld}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: aks-helloworld
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels: {app: aks-helloworld}
  template:
    metadata:
      labels: {app: aks-helloworld}
    spec:
      serviceAccountName: aks-helloworld
      containers:
      - name: aks-helloworld
        image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
        ports:
        - containerPort: 80
        env:
        - name: TITLE
          value: "Envoy Gateway on AKS"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: aks-helloworld
  namespace: default
spec:
  parentRefs:
  - name: eg-private
  rules:
  - matches:
    - path: {type: PathPrefix, value: /}
    backendRefs:
    - name: aks-helloworld
      port: 80

kubectl wait --timeout=60s --for=condition=available deployment/aks-helloworld
kubectl get pods -l app=aks-helloworld
GATEWAY_IP=$(kubectl get svc -n envoy-gateway-system \
  -l gateway.envoyproxy.io/owning-gateway-name=eg-private \
  -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')
kubectl run curl-test --rm -it --restart=Never --image=curlimages/curl -- curl -s "http://$GATEWAY_IP/"

# Alternative for local browser testing: discover the Service rather than reconstructing its generated name.
SERVICE_NAME=$(kubectl get svc -n envoy-gateway-system \
  -l gateway.envoyproxy.io/owning-gateway-name=eg-private \
  -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward -n envoy-gateway-system "svc/$SERVICE_NAME" 8080:80
# Browse http://localhost:8080 while the forward is active.

An internal LoadBalancer is reachable from connected private networks, not directly from the Internet. Use a same-VNet VM, a VPN-connected client, a VM reached through Bastion, or the test Pod. Traffic flows from the private client through Azure Internal LoadBalancer, Envoy data plane, HTTPRoute, ClusterIP Service and application Pod.

kubectl get gateway
kubectl get httproute
kubectl get pods,svc -n envoy-gateway-system

6. Prove isolation with a second namespace

Create namespace team-b. Duplicate the preceding ServiceAccount, Service and Deployment with namespace team-b, name and app label aks-helloworld-b, matching serviceAccountName and selectors, and TITLE Team-B App on Envoy Gateway. Keep two replicas, the same image and port 80. These exact substitutions reproduce the source's second full application block without changing its behavior.

kubectl create namespace team-b
# Apply the second application's resources with the substitutions above.
kubectl wait --timeout=60s --for=condition=available deployment/aks-helloworld-b -n team-b

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg-private-team-b
  namespace: team-b
spec:
  gatewayClassName: eg-private
  listeners:
  - name: http
    protocol: HTTP
    port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: aks-helloworld-b
  namespace: team-b
spec:
  parentRefs:
  - name: eg-private-team-b
  rules:
  - matches:
    - path: {type: PathPrefix, value: /}
    backendRefs:
    - name: aks-helloworld-b
      port: 80

kubectl get pods -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-namespace
kubectl get gateway -A
kubectl get svc -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-name
kubectl get hpa -n envoy-gateway-system
GATEWAY_IP_B=$(kubectl get svc -n envoy-gateway-system \
  -l gateway.envoyproxy.io/owning-gateway-name=eg-private-team-b \
  -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}')
kubectl run curl-test-b --rm -it --restart=Never \
  --image=curlimages/curl -- curl -s "http://$GATEWAY_IP_B/"

Expected: the original two data-plane Pods remain, two team-b data-plane Pods appear, and the two control-plane replicas remain unchanged. There are two private LoadBalancer addresses and two independently scaling HPAs, each min 2/max 10. Sharing a GatewayClass shares a template, not the actual proxy instances.

7. Independent versus shared gateways

For a shared Gateway, label only approved namespaces and allow them through the listener. Routes in those namespaces must include the Gateway namespace in parentRefs. This is an attachment permission; network policies still control packet flow.

listeners:
- name: http
  protocol: HTTP
  port: 80
  allowedRoutes:
    namespaces:
      from: Selector
      selector:
        matchLabels:
          shared-gateway: "true"

8. Put Application Gateway in front of the private proxy

A public Application Gateway can terminate TLS for illustrative domain api.contoso.com, forward HTTP to the Envoy private LB address, and let HTTPRoute select backends by Host and path. DNS then points to Application Gateway's public IP; Envoy itself matches the HTTP Host header, not DNS ownership.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-route
  namespace: default
spec:
  hostnames: [api.contoso.com]
  parentRefs:
  - name: eg-private
  rules:
  - matches:
    - path: {type: PathPrefix, value: /api/users}
    backendRefs:
    - name: user-service
      port: 80
  - matches:
    - path: {type: PathPrefix, value: /api/orders}
    backendRefs:
    - name: order-service
      port: 80

Unexpected Host rewriting breaks hostname matching. Verify both client requests and probes. HTTP between Application Gateway and Envoy is the source's offload pattern, not end-to-end encryption; choose backend TLS if your security requirements require it.

9. Map NGINX features explicitly

TLS: choose the termination point

For direct exposure, reuse the Key Vault → SecretProviderClass/CSI → Kubernetes TLS Secret pipeline and reference the Secret from the Gateway's namespace. For Application Gateway offload, manage the certificate on Application Gateway (optionally backed by Key Vault) and retain an HTTP-only Envoy listener. Do not deploy an unused CSI pipeline merely because the previous NGINX used one.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg-private
  namespace: default
spec:
  gatewayClassName: eg-private
  listeners:
  - name: http
    protocol: HTTP
    port: 80
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: keyvault-api-contoso-com

The Secret name is illustrative; it must already exist and contain a matching certificate/private key. The source does not supply a complete Envoy CSI or identity setup. A cross-namespace certificate reference additionally needs a supported ReferenceGrant.

Rewrite /api to the backend root

The original NGINX pattern used /api(/|$)(.*) and rewrite-target: /$2. The standard prefix rewrite below removes /api without copying NGINX regex syntax.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-rewrite
  namespace: default
spec:
  parentRefs:
  - name: eg-private
  hostnames: [api.contoso.com]
  rules:
  - matches:
    - path: {type: PathPrefix, value: /api}
    filters:
    - type: URLRewrite
      urlRewrite:
        path:
          type: ReplacePrefixMatch
          replacePrefixMatch: /
    backendRefs:
    - name: api-service
      port: 80

HTTP to HTTPS redirect

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-redirect
  namespace: default
spec:
  parentRefs:
  - name: eg-private
    sectionName: http
  rules:
  - filters:
    - type: RequestRedirect
      requestRedirect:
        scheme: https
        statusCode: 301

Attach application HTTPS routes to sectionName: https and ensure the HTTPS listener and certificate work before enabling redirects. Avoid applying this direct-Envoy pattern blindly behind an HTTP-offloading frontend, where redirection can loop.

Timeout policy

The source maps NGINX read/send timeouts of 120 seconds and connect timeout of 10 seconds to the following policy. This is not a one-to-one equivalence: a connection idle timeout and overall request timeout do not configure a ten-second upstream connect timeout.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: api-timeout
  namespace: default
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: api-route
  timeout:
    http:
      connectionIdleTimeout: 120s
      requestTimeout: 120s

Local rate limiting

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: api-ratelimit
  namespace: default
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: api-route
  rateLimit:
    type: Local
    local:
      rules:
      - limit:
          requests: 10
          unit: Second

The NGINX source used 10 requests/second with burst multiplier 5. This local policy does not preserve that burst or imply a cluster-wide aggregate limit; validate per-proxy behavior under scaling and decide whether global limiting is required. The timeout and rate-limit examples are alternatives illustrating individual features; combine compatible fields into a deliberate policy rather than assuming multiple same-target policies merge.

Source-IP authorization

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: ip-whitelist
  namespace: default
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: eg-private
  authorization:
    defaultAction: Deny
    rules:
    - action: Allow
      principal:
        clientCIDRs:
        - "10.0.0.0/8"
        - "172.16.0.0/12"

These are illustrative RFC1918 ranges, not an approved production allowlist. When another proxy fronts Envoy, establish which IP is considered the client and how forwarded headers are trusted before relying on this policy. CORS is mapped to SecurityPolicy in the source, but no full CORS policy is provided there; inspect the installed schema and test preflight, credentials, origins and response headers rather than assuming the NGINX annotations carry over.

10. Replace managed DNS automation deliberately

The NGINX add-on supplied external-dns integration through attached Azure DNS zones. Self-managed Envoy does not. The source's illustrative Helm command is:

# Configuration fragment only: configure the chart repository, compatible chart version,
# Azure authentication, zone/domain scoping and ownership before running.
helm install external-dns bitnami/external-dns \
  --set provider=azure \
  --set azure.resourceGroup="YOUR_DNS_ZONE_RESOURCE_GROUP" \
  --set policy=sync \
  --set 'sources[0]=gateway-httproute' \
  --set 'sources[1]=gateway-grpcroute'

This is not a complete external-dns installation. Chart values differ across versions; the source omits identity and DNS permissions. The sync policy can delete records. Scope domains and zones, configure ownership, and ensure two controllers do not reconcile the same records during migration.

If Application Gateway is the public entry point, DNS may instead be managed manually or directly in Azure DNS as api.contoso.com → Application Gateway public IP. Keep the Host unchanged on the backend leg; no external-dns instance is required solely to make Envoy match that hostname.

11. Migration validation and safe cutover

1. Inventory Ingress hosts, paths, TLS references, rewrites, redirects, timeouts, limits, CORS, DNS ownership and source-IP controls.
2. Use ingress2gateway to assist conversion, but manually review every generated resource and unsupported annotation.
3. Deploy the new controller and parallel private Gateway without changing the existing entry point. Verify Gateway and HTTPRoute conditions, backend reachability, probes and certificate trust.
4. Exercise host/path routing, slash boundaries, redirects, large requests, long-lived connections, timeouts, rate limits and source-IP authorization. Validate rolling-update draining and independent HPA behavior.
5. Plan DNS or Application Gateway backend changes with a rollback to the old entry point. Retain the old controller and resources until production behavior is validated; migrate before the applicable NGINX support deadline.
6. Transfer DNS/certificate responsibility explicitly, then retire unused resources only after confirming they no longer serve traffic.

References and source limits

• Installing Envoy Gateway on AKS: three deployment patterns
• Envoy Gateway release notes
• AKS Application Routing
• NGINX configuration
• NGINX DNS and TLS integration
• Ingress NGINX retirement update

The source fully demonstrates the private-LB pattern, not public LB or Private Link Service manifests. It does not supply backend Services for the illustrative user/order/API routes, a complete external-dns identity configuration, or production performance tests. Version-pinned examples are retained as documented baselines, not represented as today's latest releases.

Resources

No standalone companion files are distributed for this topic. Use the in-page examples and review the editorial notes below for source availability.

Editorial notes

This page consolidates the following source documents into an English technical guide:

• Installing Envoy Gateway on AKS
• Migrating Application Routing NGINX to Envoy Gateway

• Environment-specific cluster and resource-group names are replaced with variables; private addresses and source links are not published.
• The second hello-world application's repeated manifest is consolidated into exact name, namespace, label, ServiceAccount and title substitutions; its Gateway, route and validation remain explicit.
• No standalone resource files exist in the source folder; inline examples have not been misrepresented as source downloads.

No original credentials or private repository links are included. Do not put populated configuration files or copied production outputs back into this public site.