AKS Application Routing Gateway API: Migration, Routing, and Key Vault TLS

This consolidated runbook covers managed Application Routing Gateway API installation, an internal NGINX baseline and its conversion artifacts, shared HTTPRoute patterns, Gateway operations, and two generations of Key Vault integration. It is not the separately installed Envoy Gateway Helm product. All private environment identifiers have been removed; example.com and contoso domains are illustrative.

1. Read the chronology before choosing a procedure

The original GA/manual documents say there is no turnkey Key Vault integration. The updated document explicitly supersedes that statement. Conversely, this TLS update deliberately does not cover ClusterExternalDNS/ExternalDNS automation. Do not interpret an old limitation as current, or imply that DNS automation has been configured by following this TLS-only article.

The source's GA reference is April 28, 2026, with June checks showing the rollout finished in East US and Korea Central. Those are observations, not a live rollout report. No Azure operations were executed for this publication.

2. Check prerequisites and conflicting installations

• Azure CLI 2.86.0 or later for the GA path; a preview extension is not required.
• The target region's resource-provider rollout must include v20260428 or later. Check the AKS Release Tracker.
• Use AKS Managed Gateway API installation. Self-managed Gateway API CRDs and Experimental-channel CRDs are not the source's supported configuration.
• Application Routing Gateway API and the Istio service mesh add-on cannot be enabled together in the described configuration.
• Inventory residual Istio CRDs and an existing istio GatewayClass before enabling this implementation. Residual mesh resources can conflict with control-plane startup.

export RESOURCE_GROUP="YOUR_RESOURCE_GROUP"
export CLUSTER="YOUR_CLUSTER"
export LOCATION="YOUR_AZURE_REGION"
az --version | head -3
az upgrade
az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" --query serviceMeshProfile
kubectl get crd | grep istio.io
kubectl get gatewayclass istio
kubectl get crd gateways.gateway.networking.k8s.io -o yaml | grep channel

The source includes deleting every Istio CRD and the istio GatewayClass as conflict cleanup. CRD deletion also deletes all associated custom resources, including VirtualService and DestinationRule objects. Back up and migrate live workloads first; disable the mesh through its supported process and delete only confirmed-unused resources. Do not run a blanket CRD deletion on a shared production cluster.

3. Enable and verify the managed implementation

# New disposable cluster: enable the prerequisite and implementation together.
az aks create -g "$RESOURCE_GROUP" -n "$CLUSTER" -l "$LOCATION" \
  --enable-gateway-api --enable-app-routing-istio \
  --node-count 2 --node-vm-size Standard_D2a_v4 --generate-ssh-keys

# OR an existing cluster, in this order:
az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" --enable-gateway-api
az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" --enable-app-routing-istio
# Equivalent combined update:
# az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" --enable-gateway-api --enable-app-routing-istio
az aks get-credentials -g "$RESOURCE_GROUP" -n "$CLUSTER" --overwrite-existing
kubectl get crd | grep gateway.networking.k8s.io
kubectl get gatewayclass
kubectl get pods -n aks-istio-system
kubectl get validatingwebhookconfiguration
kubectl get cm -n aks-istio-system istio-gateway-class-defaults

Expect GatewayClass approuting-istio, two Running istiod Pods in the source baseline, and the azure-service-mesh-ccp-validating-webhook. The CRD inventory includes GatewayClass, Gateway, HTTPRoute, GRPCRoute and ReferenceGrant. If app-routing-istio is enabled first, the source observed a successful CLI command but no GatewayClass or defaults ConfigMap until managed Gateway API was also enabled.

kubectl get crd gateways.gateway.networking.k8s.io -o jsonpath='{.metadata.annotations}'
kubectl get deploy -n aks-istio-system istiod \
  -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
helm list -A | grep -E 'azure-service-mesh-istio-discovery|aks-managed-overlay'

If the Deployment is revision-named, discover it with kubectl get deploy -n aks-istio-system -l app=istiod rather than assuming the literal name istiod. The source compares the image with GA release tags asm-1-27 1.27.9-2, asm-1-28 1.28.6-1 and asm-1-29 1.29.2-1, and an add-on chart build such as v20260520-addon-260521-1. These checks help distinguish an old CLI Preview warning from the deployed build, but a date or tag alone is not a substitute for the applicable support and rollout documentation.

4. Reproduce the historical internal NGINX baseline

The historical environment used Azure CNI with Cilium, a three-node Standard_D2as_v5 pool and an already enabled Application Routing add-on. A public default controller existed; no Ingress or Gateway API resources initially existed. Actual pool names, Pod CIDR and addresses are intentionally omitted. Create an additional internal controller rather than disrupting the public one:

apiVersion: approuting.kubernetes.azure.com/v1alpha1
kind: NginxIngressController
metadata:
  name: nginx-internal
spec:
  ingressClassName: nginx-internal
  controllerNamePrefix: nginx-internal
  loadBalancerAnnotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"

kubectl apply -f downloads/istio-gateway-api/current_env/internal-nginx-controller.yaml
kubectl apply -f downloads/istio-gateway-api/current_env/sample-app.yaml
kubectl get ingressclass
kubectl get svc,pods -n app-routing-system
kubectl get pods,svc -n demo

The supplied sample-app.yaml creates namespace demo, two echo Pods using ealen/echo-server:latest, and Service echo on port 80. The image tag is historical and mutable; pin a reviewed version for production. The lab's generated internal Service was nginx-internal-0, with two controller Pods Running; a metrics Service exposed 10254. The source's test command inconsistently queried nginx-internal. Discover the actual generated Service and use that name.

For a minimal HTTP baseline, apply this Ingress rather than the annotation-heavy conversion fixture:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo-ingress
  namespace: demo
spec:
  ingressClassName: nginx-internal
  rules:
  - host: echo.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: echo
            port: {number: 80}

export NGINX_SERVICE="YOUR_DISCOVERED_NGINX_SERVICE"
INTERNAL_IP=$(kubectl get svc -n app-routing-system "$NGINX_SERVICE" \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
kubectl run curl-test --rm -it --restart=Never --image=curlimages/curl -- \
  curl -H "Host: echo.example.com" "http://$INTERNAL_IP/"
kubectl get ingress -n demo

Test from the cluster or a connected VNet. The historical echo response confirmed the requested Host, GET /, HTTP protocol and forwarded headers, demonstrating Internal LB → NGINX → echo. Internal and public controllers can coexist. Removing the default public NginxIngressController is optional and only safe after checking all dependent traffic.

5. Understand the six source YAML downloads

All six files are sanitized copies of actual source resources, not invented extractions. The target Gateway has been renamed consistently to demo-istio-gw, and the test hostname is consistently echo.example.com. Historical classes, conversion semantics and status stubs remain visible with warnings. Modern examples below use spec.infrastructure.annotations; verify propagation before trusting a metadata annotation in a legacy file.

A converter does not prove semantic equivalence. In particular, 20m0s is not the source's 120-second timeout, regex matching is implementation-specific, the class nginx is not approuting-istio, and full-path rewriting differs from prefix stripping. Review unsupported annotations and verify accepted schema, traffic, TLS, client-IP behavior, timeouts, buffers and error handling before cutover.

6. Test an HTTP Gateway before TLS

export ISTIO_RELEASE=release-1.27
kubectl apply -f "https://raw.githubusercontent.com/istio/istio/$ISTIO_RELEASE/samples/httpbin/httpbin.yaml"

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: httpbin-gateway
  namespace: default
spec:
  gatewayClassName: approuting-istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces: {from: Same}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin
  namespace: default
spec:
  parentRefs:
  - name: httpbin-gateway
  hostnames: [httpbin.example.com]
  rules:
  - matches:
    - path: {type: PathPrefix, value: /get}
    backendRefs:
    - name: httpbin
      port: 8000

kubectl get deployment,svc,hpa,pdb -l 'gateway.networking.k8s.io/gateway-name=httpbin-gateway'
kubectl wait --for=condition=programmed gateway httpbin-gateway --timeout=300s
INGRESS_HOST=$(kubectl get gateway httpbin-gateway -o jsonpath='{.status.addresses[0].value}')
curl -s -I -H "Host: httpbin.example.com" "http://$INGRESS_HOST/get"

The default is a public LoadBalancer. Expected source baseline: Deployment and Service named httpbin-gateway-approuting-istio, two replicas, HPA min 2/max 5/CPU 80%, PDB minAvailable 1, and HTTP 200. Generated names follow <gateway-name>-<gatewayClassName>; keep DNS-compatible names under the resource limit of 63 characters. The source notes the gateway.istio.io/name-override annotation for supported naming overrides.

7. Internal shared Gateway and namespace boundaries

A Gateway provisions its own data plane and LB. Many per-namespace Gateways therefore mean many IPs and proxy Deployments. A shared Gateway in gateway-system lets service teams keep their HTTPRoutes in separate namespaces while sharing one entry point. The following static address and subnet are placeholders, not source environment values; replace them before applying.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shared-gateway
  namespace: gateway-system
spec:
  infrastructure:
    annotations:
      service.beta.kubernetes.io/azure-load-balancer-internal: "true"
      service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "YOUR_INGRESS_SUBNET"
      service.beta.kubernetes.io/azure-load-balancer-ipv4: "YOUR_PRIVATE_LB_IP"
  gatewayClassName: approuting-istio
  listeners:
  - name: http
    hostname: "*.contoso.io"
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: Selector
        selector:
          matchLabels:
            shared-gateway: "true"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: catalog-route
  namespace: ns-catalog
spec:
  parentRefs:
  - name: shared-gateway
    namespace: gateway-system
  hostnames: [catalog.contoso.io]
  rules:
  - backendRefs:
    - name: catalog-service
      port: 8080

Create the namespaces and backend Services first; label allowed route namespaces with kubectl label namespace ns-catalog shared-gateway=true. The source also shows from: All, which permits every namespace; Selector is a narrower alternative. A hostname may be omitted to accept all hosts or set to a supported wildcard such as *.contoso.io; a bare * is not a valid hostname value. A wildcard normally excludes the apex.

HTTPRoute → Gateway attachment requires listener allowedRoutes, not ReferenceGrant. A route referring to a Service in another namespace needs a ReferenceGrant in the target namespace. A Gateway referring to a Secret elsewhere similarly needs explicit target-namespace permission. Backend Services in a route's own namespace need no cross-namespace grant. None of these API permissions replaces NetworkPolicy.

8. Customize generated resources through supported ConfigMaps

Do not hand-edit generated Deployment, Service, HPA or PDB resources and expect changes to persist. The supported customization path uses YAML strings under ConfigMap data keys deployment, service, horizontalPodAutoscaler and podDisruptionBudget.

kubectl edit cm istio-gateway-class-defaults -n aks-istio-system

data:
  deployment: |
    metadata:
      labels:
        owner: platform-team
    spec:
      minReadySeconds: 15
  horizontalPodAutoscaler: |
    spec:
      minReplicas: 3
      maxReplicas: 6
  podDisruptionBudget: |
    spec:
      minAvailable: 1

Unsupported fields are rejected by the managed webhook. This is a summary, not a replacement for the current official allowlist. A PDB that prevents necessary eviction can cause UpgradeFailed/PodDrainFailure. Set replicas, distribution and disruption requirements together.

apiVersion: v1
kind: ConfigMap
metadata:
  name: httpbin-gw-options
  namespace: default
data:
  horizontalPodAutoscaler: |
    spec:
      minReplicas: 2
      maxReplicas: 4
  deployment: |
    spec:
      template:
        spec:
          nodeSelector:
            workload: ingress
          tolerations:
          - key: ingress-only
            operator: Exists
            effect: NoSchedule
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: httpbin-gateway
  namespace: default
spec:
  gatewayClassName: approuting-istio
  infrastructure:
    parametersRef:
      group: ""
      kind: ConfigMap
      name: httpbin-gw-options
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces: {from: Same}

kubectl get hpa httpbin-gateway-approuting-istio
# Expect MINPODS 2 and MAXPODS 4.

Use infrastructure annotations for supported LB behavior, ConfigMaps for operational Pod/Service/HPA/PDB settings, and HTTPRoute for application routing. Ensure suitable labeled/tainted nodes exist before adding node placement requirements.

9. Access logs and original client IP

The source reports that a Preview-era missing access-log issue was resolved in its GA test: Envoy JSON access logs appeared on stdout without extra configuration, on Istio 1.29.2-1. This observation refers to that build, not every environment.

kubectl get pods -n demo -l 'gateway.networking.k8s.io/gateway-name=demo-istio-gw'
kubectl logs -n demo "YOUR_GATEWAY_POD" --tail=50

The original successful example had HTTP 200, 10 ms backend/total duration, zero received bytes and 1946 sent bytes. Its actual addresses, request ID and environment names are omitted. Collect stdout through Container Insights/Log Analytics, Fluent Bit/Loki or existing logging infrastructure; parse JSON in KQL/LogQL. Check the customization allowlist before attempting format changes.

With externalTrafficPolicy=Cluster, the observed client may be a node after SNAT. Set Local using a class-level or Gateway-level ConfigMap when source-IP preservation is required:

apiVersion: v1
kind: ConfigMap
metadata:
  name: gateway-local-options
  namespace: demo
data:
  service: |
    spec:
      externalTrafficPolicy: Local

Reference this ConfigMap from the Gateway's infrastructure.parametersRef using group empty, kind ConfigMap and name gateway-local-options, or merge its data.service block into the existing options ConfigMap.

kubectl get svc -n demo -l 'gateway.networking.k8s.io/gateway-name=demo-istio-gw' \
  -o jsonpath='{.items[0].spec.externalTrafficPolicy}'

ContainerLogV2
| where PodNamespace == "demo"
| where ContainerName has "istio-proxy" or PodName has "demo-istio-gw"
| extend log = parse_json(LogMessage)
| project TimeGenerated,
          x_forwarded_for = tostring(log.x_forwarded_for),
          downstream = tostring(log.downstream_remote_address),
          method = tostring(log.method),
          path = tostring(log.path),
          status = toint(log.response_code)
| order by TimeGenerated desc

Local routes only to local gateway endpoints. Confirm LB probes remove nodes without a ready gateway Pod, provide sufficient replicas, use topology spreading when appropriate, and test distribution. Uneven per-node Pod counts can produce uneven traffic. Behind another proxy, forwarded-header trust and the distinction between original client and immediate peer still matter.

10. Historical multi-port Local health-probe issue

The source observed intermittent connection refused, approximately 50% success, with a multi-port generated Service (for example 15021+80 or 80+443) and externalTrafficPolicy=Local. One port's LB probe excluded nodes without Pods while another appeared healthy on every node. It attributed this to the generated Service/probe wiring around a single healthCheckNodePort and reported product-team investigation without an ETA. This is a build-specific incident report, not a general Kubernetes rule that every multi-port Local Service is broken.

The reported workaround was to add per-port probe annotations via the customization ConfigMap, not by editing the generated Service. The original reproduction showed empty values and reported success for port 80 merely from the keys being present:

# Historical reproduction only: these empty values are NOT a validated production probe configuration.
data:
  service: |
    metadata:
      annotations:
        service.beta.kubernetes.io/port_80_health-probe_port:
        service.beta.kubernetes.io/port_80_health-probe_protocol:
        service.beta.kubernetes.io/port_80_health-probe_request-path:
        service.beta.kubernetes.io/port_443_health-probe_port:
        service.beta.kubernetes.io/port_443_health-probe_protocol:
        service.beta.kubernetes.io/port_443_health-probe_request-path:
    spec:
      externalTrafficPolicy: Local

Do not deploy null annotation values as a generic fix. Determine valid string values for the actual node-level probe port, protocol and path from the Service and supported Azure LB documentation, then test every frontend rule and node. The source did not establish a working 443 configuration: SSL connection refused remained unresolved. No fix ETA or validated 443 recipe is available in these documents.

export GATEWAY_NAMESPACE="YOUR_GATEWAY_NAMESPACE"
export GATEWAY_NAME="YOUR_GATEWAY_NAME"
export NODE_RESOURCE_GROUP="YOUR_NODE_RESOURCE_GROUP"
kubectl get svc -n "$GATEWAY_NAMESPACE" \
  -l "gateway.networking.k8s.io/gateway-name=$GATEWAY_NAME" -o yaml | grep -A2 health-probe
kubectl get svc -n "$GATEWAY_NAMESPACE" \
  -l "gateway.networking.k8s.io/gateway-name=$GATEWAY_NAME" -o yaml \
  | grep -A30 -E 'ports:|healthCheckNodePort|externalTrafficPolicy'
LB=$(az network lb list -g "$NODE_RESOURCE_GROUP" --query '[0].name' -o tsv)
az network lb probe list -g "$NODE_RESOURCE_GROUP" --lb-name "$LB" -o table
az network lb rule list -g "$NODE_RESOURCE_GROUP" --lb-name "$LB" \
  --query '[].{rule:name,feport:frontendPort,beport:backendPort,probe:probe.id}' -o table

If there are multiple LBs, identify the one owning the Gateway frontend rather than blindly using index zero. For 443-only failures, verify an HTTPS 443 listener, correct certificateRefs and Secret namespace, successful CSI/operator synchronization, the Service's 443 port and endpoints, Envoy listener/TLS logs, and the Azure 443 rule-to-probe/backend mapping. Intermittent per-node failure and total listener/TLS failure need different diagnoses.

11. Cilium policies around the Gateway

A centralized NGINX controller previously required opening one namespace. Per-namespace Gateways move proxy endpoints into their own namespaces, so existing allow rules may no longer cover ingress. Standard NetworkPolicy works with Cilium, but the following uses CiliumNetworkPolicy to express entities. Verify the actual gateway Pod label before applying; the source uses istio.io/gateway-name while generated-resource inspection also uses gateway.networking.k8s.io/gateway-name.

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-ingress-to-gateway
  namespace: demo
spec:
  endpointSelector:
    matchLabels:
      istio.io/gateway-name: demo-istio-gw
  ingress:
  - fromEntities: [world, cluster, health, remote-node]
    toPorts:
    - ports:
      - {port: "80", protocol: TCP}
      - {port: "443", protocol: TCP}
      - {port: "15021", protocol: TCP}

Ports 80 and 443 carry application traffic; 15021 is the Istio status/readiness endpoint. The broad example grants more than namespace isolation. For a narrower model, the source separates public traffic, health traffic and approved namespaces. The following makes the local-node health allowance explicit:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: restricted-ingress-to-gateway
  namespace: demo
spec:
  endpointSelector:
    matchLabels:
      istio.io/gateway-name: demo-istio-gw
  ingress:
  - fromEntities: [world]
    toPorts:
    - ports:
      - {port: "80", protocol: TCP}
      - {port: "443", protocol: TCP}
  - fromEntities: [host, remote-node, health]
    toPorts:
    - ports:
      - {port: "15021", protocol: TCP}
  - fromEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: frontend-allowed
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: another-allowed
    toPorts:
    - ports:
      - {port: "80", protocol: TCP}
      - {port: "443", protocol: TCP}

Use the broad and narrow examples as alternatives. Cilium allow policies are additive; leaving the broad rule installed defeats the narrow rule. With externalTrafficPolicy=Cluster, external traffic may appear as remote-node after SNAT. Closing remote-node on 80/443 can therefore break external access. Validate Local and its health probes first, then tighten sources. Confirm actual probe source identities and port translation instead of assuming this template matches every AKS network path.

For scalable namespace membership, label approved namespaces, but use Cilium's namespace-label representation, not an unqualified Pod label. The source shows an unqualified access-to-demo-gw selector; inspect endpoint identities and use the correctly qualified key for your version:

kubectl label ns frontend-allowed access-to-demo-gw=true
kubectl label ns another-allowed access-to-demo-gw=true

fromEndpoints:
- matchLabels:
    k8s:io.cilium.k8s.namespace.labels.access-to-demo-gw: "true"

kubectl exec -n kube-system ds/cilium -- cilium monitor --type drop | grep "YOUR_GATEWAY_POD_IP"
kubectl exec -n kube-system ds/cilium -- cilium endpoint list | grep demo

Run diagnostics on the node hosting the relevant endpoint; a DaemonSet exec may select a different node. The source leaves Gateway-to-backend cross-namespace policy validation as future work. Explicitly allow required backend ingress, gateway egress, DNS, health and control-plane communication with aks-istio-system where policies isolate them. A policy isolates selected directions on selected Pods, not every Pod in a namespace automatically.

12. Preferred TLS path: Application Routing operator

The operator watches listeners of GatewayClass approuting-istio with two TLS options: kubernetes.azure.com/tls-cert-keyvault-uri and kubernetes.azure.com/tls-cert-service-account. It creates an SPC named kv-gw-cert-<gateway>-<listener>, coordinates CSI synchronization to a same-named kubernetes.io/tls Secret in the Gateway namespace, and patches certificateRefs. CSI still fetches the material; the operator automates the chain. No administrator-maintained dummy Deployment is needed in this path.

• Enable both the Application Routing operator and the Application Routing Istio Gateway implementation, plus managed Gateway API.
• Enable OIDC issuer and Microsoft Entra Workload Identity.
• Enable Azure Key Vault provider for Secrets Store CSI Driver.
• Have permissions to create role assignments, UAMI and federated identity credentials, such as appropriately scoped Owner or RBAC Administrator plus Managed Identity Contributor.
• Existing NGINX users can keep NGINX running while enabling the Gateway implementation; OIDC/Workload Identity may still need enabling.

az aks approuting enable -g "$RESOURCE_GROUP" -n "$CLUSTER" --enable-kv
az aks approuting gateway istio enable -g "$RESOURCE_GROUP" -n "$CLUSTER"
az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" \
  --enable-oidc-issuer --enable-workload-identity
# Managed Gateway API must already be enabled as in section 3.
az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" \
  --query 'addonProfiles.azureKeyvaultSecretsProvider.enabled'
kubectl get pods -n kube-system -l app=secrets-store-csi-driver
kubectl get pods -n kube-system -l app=secrets-store-provider-azure

The source also expresses these as create/update flags --enable-app-routing and --enable-app-routing-istio. Both components are required. If the CSI add-on is already enabled, --enable-kv may be omitted; redundant enablement is described as idempotent. Legacy --attach-kv and --attach-zones configure the old NGINX add-on identity, not the UAMI/Workload Identity model below.

13. Create a vault and a development certificate

export KV_NAME="YOUR_KEY_VAULT_NAME"
export UAMI_NAME="YOUR_MANAGED_IDENTITY_NAME"
az keyvault create --name "$KV_NAME" --resource-group "$RESOURCE_GROUP" \
  --location "$LOCATION" --enable-rbac-authorization true --enable-purge-protection true
az keyvault show --name "$KV_NAME" --resource-group "$RESOURCE_GROUP" \
  --query 'properties.{rbac:enableRbacAuthorization,uri:vaultUri}'

Choose a globally unique valid vault name. Purge protection is recommended for production and prevents early permanent deletion during retention. The certificate creator needs Key Vault Certificates Officer or an appropriate administrator role; deployment ownership alone does not automatically grant data-plane access.

cat > cert-policy.json <<'EOF'
{
  "issuerParameters": {"name": "Self"},
  "keyProperties": {"exportable": true, "keyType": "RSA", "keySize": 2048, "reuseKey": false},
  "secretProperties": {"contentType": "application/x-pkcs12"},
  "x509CertificateProperties": {
    "subject": "CN=*.example.com",
    "subjectAlternativeNames": {"dnsNames": ["*.example.com", "example.com"]},
    "validityInMonths": 12,
    "keyUsage": ["digitalSignature", "keyEncipherment"]
  }
}
EOF
az keyvault certificate create --vault-name "$KV_NAME" \
  --name approuting-demo-cert --policy @cert-policy.json
export CERT_URI=$(az keyvault certificate show --vault-name "$KV_NAME" \
  --name approuting-demo-cert --query id -o tsv | sed 's|/[^/]*$||')

This self-signed certificate is for testing only. Import a trusted CA-issued certificate with az keyvault certificate import for production. Use an unversioned certificates URI, structurally https://YOUR_KEY_VAULT_NAME.vault.azure.net/certificates/YOUR_CERTIFICATE_NAME, so the integration can follow new versions.

14. Bind a ServiceAccount to the vault identity

az identity create --resource-group "$RESOURCE_GROUP" --name "$UAMI_NAME" --location "$LOCATION"
export UAMI_CLIENT_ID=$(az identity show -g "$RESOURCE_GROUP" -n "$UAMI_NAME" --query clientId -o tsv)
export UAMI_PRINCIPAL_ID=$(az identity show -g "$RESOURCE_GROUP" -n "$UAMI_NAME" --query principalId -o tsv)
az role assignment create --assignee-object-id "$UAMI_PRINCIPAL_ID" \
  --assignee-principal-type ServicePrincipal --role "Key Vault Secrets User" \
  --scope "$(az keyvault show --name "$KV_NAME" --query id -o tsv)"
export OIDC_ISSUER=$(az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" --query oidcIssuerProfile.issuerUrl -o tsv)
export SA_NAME=approuting-demo-sa
for ns in app-a app-b; do
  kubectl create namespace "$ns"
  az identity federated-credential create --identity-name "$UAMI_NAME" \
    --resource-group "$RESOURCE_GROUP" --name "approuting-demo-fic-$ns" \
    --issuer "$OIDC_ISSUER" --subject "system:serviceaccount:$ns:$SA_NAME" \
    --audiences "api://AzureADTokenExchange"
  kubectl apply -n "$ns" -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: $SA_NAME
  annotations:
    azure.workload.identity/client-id: $UAMI_CLIENT_ID
  labels:
    azure.workload.identity/use: "true"
EOF
done

The TLS sync identity needs Key Vault Secrets User because private key-bearing certificate material is retrieved through the vault's secret surface. A federated credential is required for each namespace/ServiceAccount pair, with exact issuer, subject and audience. Allow time for identity/RBAC propagation; the source observed a typical one-to-three-minute wait.

The source places azure.workload.identity/use=true on the ServiceAccount and describes it as token-injection configuration. Ordinary Workload Identity mutation is triggered on the consuming Pod, not solely by a ServiceAccount label. The operator manages the consuming resources in this integration: inspect the generated workload/token wiring and current official requirements if federation fails. Do not assume the SA label alone injects a token into arbitrary application Pods.

Shared Gateway identity scope

If the Gateway is only in gateway-system and HTTPRoutes are in several service namespaces, create the ServiceAccount, FIC and TLS configuration only for gateway-system. The operator-generated SPC and Secret also live there. Service namespaces that only contain HTTPRoutes do not need duplicate certificate identities. Set allowedRoutes to All or a restricted Selector and use cross-namespace parentRefs. No ReferenceGrant is needed for this route-to-Gateway attachment or a same-namespace Secret.

15. Enable operator-managed HTTPS listeners

for ns in app-a app-b; do
  kubectl apply -n "$ns" -f https://raw.githubusercontent.com/istio/istio/release-1.27/samples/httpbin/httpbin.yaml
done
for pair in "app-a:a" "app-b:b"; do
  ns=${pair%%:*}
  sub=${pair##*:}
  fqdn=${sub}.example.com
  kubectl apply -n "$ns" -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: ${sub}-gateway
  labels:
    app: approuting-demo
    zone: ${sub}
spec:
  gatewayClassName: approuting-istio
  listeners:
  - name: https
    hostname: $fqdn
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      options:
        kubernetes.azure.com/tls-cert-keyvault-uri: $CERT_URI
        kubernetes.azure.com/tls-cert-service-account: $SA_NAME
    allowedRoutes:
      namespaces: {from: Same}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: ${sub}-route
spec:
  parentRefs:
  - name: ${sub}-gateway
  hostnames: ["$fqdn"]
  rules:
  - matches:
    - path: {type: PathPrefix, value: /get}
    backendRefs:
    - name: httpbin
      port: 8000
EOF
done
kubectl wait -n app-a --for=condition=programmed gateway a-gateway --timeout=300s
kubectl wait -n app-b --for=condition=programmed gateway b-gateway --timeout=300s
kubectl get secretproviderclass,secret -n app-a
kubectl get secretproviderclass,secret -n app-b

Expected in app-a: SPC and TLS Secret kv-gw-cert-a-gateway-https, with two Secret data entries. app-b uses the corresponding b-gateway name. Do not manually compete with the operator over certificateRefs.

GATEWAY_IP=$(kubectl get -n app-a gateway a-gateway -o jsonpath='{.status.addresses[0].value}')
# Self-signed lab only: -k bypasses server certificate verification.
curl -k -I --resolve "a.example.com:443:${GATEWAY_IP}" "https://a.example.com/get"
# For trusted certificates, omit -k or use --cacert YOUR_CA_CHAIN_FILE.

A successful source test expects HTTP/2 200 with the synchronized certificate. --resolve bypasses DNS delegation for the test while preserving TLS SNI and Host.

16. Rotate and troubleshoot certificates

az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" \
  --enable-secret-rotation --rotation-poll-interval 2m
az keyvault certificate create --vault-name "$KV_NAME" \
  --name approuting-demo-cert --policy @cert-policy.json
az keyvault certificate show --vault-name "$KV_NAME" --name approuting-demo-cert \
  --query x509ThumbprintHex -o tsv
kubectl -n app-a get secret kv-gw-cert-a-gateway-https -o jsonpath='{.data.tls\.crt}' \
  | base64 -d | openssl x509 -noout -fingerprint -sha1

Use an unversioned URI and enable rotation. The source expects the Secret and Envoy configuration to follow the next poll. Normalize colon separators/case when comparing thumbprints, and also verify the certificate actually served over a new TLS connection. A Secret fingerprint alone does not prove the live listener updated. Avoid overly short intervals that risk vault throttling.

17. Historical fallback: manual CSI synchronization

Use this separate path only when the operator integration is unavailable or a deliberately centralized certificate-store design requires it. In the original manual model, declaring an SPC does not itself start synchronization. Envoy consumes a Kubernetes Secret through control-plane/API configuration, not by mounting that SPC. A continuously mounted CSI volume supplies the synchronization trigger. The updated operator path removes the need for an administrator-created mount holder.

Enable CSI and authorize its identity

export AKV_NAME="$KV_NAME"
export APP_NS=demo
az aks enable-addons --addons azure-keyvault-secrets-provider \
  --resource-group "$RESOURCE_GROUP" --name "$CLUSTER"
CLIENT_ID=$(az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" \
  --query 'addonProfiles.azureKeyvaultSecretsProvider.identity.clientId' -o tsv | tr -d '\r')
OBJECT_ID=$(az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" \
  --query 'addonProfiles.azureKeyvaultSecretsProvider.identity.objectId' -o tsv | tr -d '\r')
TENANT_ID=$(az keyvault show -g "$RESOURCE_GROUP" -n "$AKV_NAME" --query properties.tenantId -o tsv | tr -d '\r')
AKV_SCOPE=$(az keyvault show -g "$RESOURCE_GROUP" -n "$AKV_NAME" --query id -o tsv | tr -d '\r')
az role assignment create --role "Key Vault Secrets User" \
  --assignee-object-id "$OBJECT_ID" --assignee-principal-type ServicePrincipal --scope "$AKV_SCOPE"
# Optional certificate access: validate role availability and data actions for your operation.
az role assignment create --role "Key Vault Certificate User" \
  --assignee-object-id "$OBJECT_ID" --assignee-principal-type ServicePrincipal --scope "$AKV_SCOPE"

This identity is the CSI add-on's automatically created user-assigned identity, not the operator path's explicit UAMI/FIC. If the vault instead uses the legacy access-policy model, grant secret get/list and certificate get/list through az keyvault set-policy --name "$AKV_NAME" --object-id "$OBJECT_ID" --secret-permissions get list --certificate-permissions get list. Do not mix RBAC and access-policy assumptions.

# The person importing certificates/secrets also needs appropriate data-plane roles.
ME_OBJECT_ID=$(az ad signed-in-user show --query id -o tsv)
az role assignment create --role "Key Vault Certificates Officer" \
  --assignee-object-id "$ME_OBJECT_ID" --assignee-principal-type User --scope "$AKV_SCOPE"
az role assignment create --role "Key Vault Secrets Officer" \
  --assignee-object-id "$ME_OBJECT_ID" --assignee-principal-type User --scope "$AKV_SCOPE"
kubectl create namespace "$APP_NS" --dry-run=client -o yaml | kubectl apply -f -

Development certificate and two storage choices

The manual source creates a 365-day RSA-2048 test root and server certificate with OpenSSL. Its server certificate lacks a SAN; the following explicitly adds one so hostname verification can work. Private keys are local test material: never commit them or publish their contents.

mkdir -p httpbin_certs
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
  -subj '/O=example Inc./CN=example.com' \
  -keyout httpbin_certs/example.com.key -out httpbin_certs/example.com.crt
openssl req -out httpbin_certs/httpbin.example.com.csr -newkey rsa:2048 -nodes \
  -keyout httpbin_certs/httpbin.example.com.key \
  -subj '/CN=httpbin.example.com/O=httpbin organization'
printf 'subjectAltName=DNS:httpbin.example.com\n' > httpbin_certs/server.ext
openssl x509 -req -sha256 -days 365 -CA httpbin_certs/example.com.crt \
  -CAkey httpbin_certs/example.com.key -set_serial 0 \
  -in httpbin_certs/httpbin.example.com.csr -out httpbin_certs/httpbin.example.com.crt \
  -extfile httpbin_certs/server.ext

# A: store separate PEM key and certificate secrets.
az keyvault secret set --vault-name "$AKV_NAME" --name test-httpbin-key --file httpbin_certs/httpbin.example.com.key
az keyvault secret set --vault-name "$AKV_NAME" --name test-httpbin-crt --file httpbin_certs/httpbin.example.com.crt

# B: combine into a PFX and import a certificate object.
# OpenSSL prompts for an export password; do not put real passwords into a published script.
openssl pkcs12 -export -inkey httpbin_certs/httpbin.example.com.key \
  -in httpbin_certs/httpbin.example.com.crt -out httpbin_certs/httpbin.example.com.pfx
az keyvault certificate import --vault-name "$AKV_NAME" \
  --name test-httpbin-cert-pfx --file httpbin_certs/httpbin.example.com.pfx \
  --password "$YOUR_PFX_PASSWORD"

The source's PFX export/import block contains a pre-existing masked/broken line. Its hidden content is not reconstructed. The two explicit commands above express the intended operations with a caller-supplied password variable. Certificate-object storage enables Key Vault certificate metadata, expiration monitoring and rotation; choose a provider-supported private-key encoding.

SPC for two separate PEM secrets

Render the shell variables in this YAML before applying; kubectl does not expand variables in a file. The mounted object aliases and secretObjects objectName values must agree.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: httpbin-credential-spc
  namespace: demo
spec:
  provider: azure
  secretObjects:
  - secretName: httpbin-credential
    type: kubernetes.io/tls
    data:
    - objectName: test-httpbin-key
      key: tls.key
    - objectName: test-httpbin-crt
      key: tls.crt
  parameters:
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "$CLIENT_ID"
    keyvaultName: "$AKV_NAME"
    cloudName: ""
    objects: |
      array:
        - |
          objectName: test-httpbin-key
          objectType: secret
          objectAlias: test-httpbin-key
        - |
          objectName: test-httpbin-crt
          objectType: secret
          objectAlias: test-httpbin-crt
    tenantId: "$TENANT_ID"

Certificate-object variant from the source

The manual document replaces the objects array with the following while keeping the same Secret alias mapping:

objects: |
  array:
    - |
      objectName: test-httpbin-cert-pfx
      objectType: secret
      objectAlias: test-httpbin-key
    - |
      objectName: test-httpbin-cert-pfx
      objectType: cert
      objectAlias: test-httpbin-crt

This alias pattern is preserved as source guidance, not proof that a raw PFX secret is automatically converted into a valid tls.key. objectAlias renames mounted content; certificate/private-key formats depend on the provider and object format. Validate that tls.crt contains the PEM certificate and tls.key a matching PEM private key with the installed provider before use. Do not assume every objectType=cert operation uses the secret API: certificate retrieval and private-key-bearing secret retrieval have different permission/format semantics. The source's blanket claim is overbroad.

Omit objectVersion to follow the latest material; setting it pins that object version and blocks automatic rotation to newer versions.

18. Keep the manual sync mount alive

apiVersion: apps/v1
kind: Deployment
metadata:
  name: secrets-store-sync-httpbin
  namespace: demo
  labels: {app: secrets-store-sync-httpbin}
spec:
  replicas: 2
  selector:
    matchLabels: {app: secrets-store-sync-httpbin}
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  template:
    metadata:
      labels: {app: secrets-store-sync-httpbin}
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels: {app: secrets-store-sync-httpbin}
              topologyKey: kubernetes.io/hostname
      containers:
      - name: pause
        image: mcr.microsoft.com/oss/kubernetes/pause:3.6
        resources:
          requests: {cpu: "10m", memory: "16Mi"}
          limits: {cpu: "50m", memory: "32Mi"}
        volumeMounts:
        - name: secrets-store01-inline
          mountPath: /mnt/secrets-store
          readOnly: true
      volumes:
      - name: secrets-store01-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: httpbin-credential-spc

kubectl -n "$APP_NS" get deploy secrets-store-sync-httpbin
kubectl -n "$APP_NS" get pod -l app=secrets-store-sync-httpbin -o wide
kubectl -n "$APP_NS" describe secret httpbin-credential

A Deployment recreates evicted Pods; a bare Pod does not. Two replicas with preferred anti-affinity and maxUnavailable=0 improve resilience but do not guarantee separate-node placement or survival of every failure. Check actual placement and disruption handling. This is a low-resource mount holder, not a traffic-serving app.

The source claims a synchronized Secret necessarily remains after all consumers stop. Do not rely on that: CSI synchronized Secret lifetime may be tied to consuming Pods and garbage collection. Even if a Secret temporarily remains, rotation stops without a mount. Maintain the mount holder throughout the manual integration's lifetime.

Manual HTTPS listener and route

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: httpbin-gateway
  namespace: demo
spec:
  gatewayClassName: approuting-istio
  listeners:
  - name: https
    hostname: httpbin.example.com
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      certificateRefs:
      - name: httpbin-credential
    allowedRoutes:
      namespaces:
        from: Selector
        selector:
          matchLabels:
            kubernetes.io/metadata.name: demo
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin
  namespace: demo
spec:
  parentRefs:
  - name: httpbin-gateway
  hostnames: [httpbin.example.com]
  rules:
  - matches:
    - path: {type: PathPrefix, value: /status}
    - path: {type: PathPrefix, value: /delay}
    backendRefs:
    - name: httpbin
      port: 8000

kubectl apply -n "$APP_NS" -f https://raw.githubusercontent.com/istio/istio/release-1.27/samples/httpbin/httpbin.yaml
kubectl wait -n "$APP_NS" --for=condition=programmed gateway httpbin-gateway --timeout=300s
INGRESS_HOST=$(kubectl get -n "$APP_NS" gateway httpbin-gateway -o jsonpath='{.status.addresses[0].value}')
SECURE_INGRESS_PORT=$(kubectl get -n "$APP_NS" gateway httpbin-gateway -o jsonpath='{.spec.listeners[?(@.name=="https")].port}')
curl -v --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
  --cacert httpbin_certs/example.com.crt \
  "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"

The expected response is HTTP/2 418, the httpbin test status, not a routing failure. certificateRefs.name must equal secretObjects.secretName. Enable the same two-minute rotation option described earlier and upload a new certificate version. For separate key/crt secrets, update a matching pair; uploading only a new certificate with a different key breaks TLS. Compare the vault thumbprint, Secret certificate fingerprint and served TLS certificate. Logs help diagnose handshakes but alone are not fingerprint proof.

19. Central certificate store and tenant isolation

In the manual per-Gateway pattern, placing SPC and sync Pods in every namespace duplicates maintenance. An alternative is cert-store containing one SPC, one two-replica sync Deployment and one or more TLS Secrets, with explicit ReferenceGrants allowing other namespaces' Gateways to reference those Secrets. This is different from a single shared Gateway: here multiple Gateways reuse centralized certificates.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: gw-can-read-tls-secret
  namespace: cert-store
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: Gateway
    namespace: tenant-a
  - group: gateway.networking.k8s.io
    kind: Gateway
    namespace: tenant-b
  to:
  - group: ""
    kind: Secret
    name: httpbin-credential

# Listener fragment on a Gateway in tenant-a or tenant-b:
tls:
  mode: Terminate
  certificateRefs:
  - kind: Secret
    namespace: cert-store
    name: httpbin-credential

Without the target namespace's permission, the source reports ResolvedRefs=False with RefNotPermitted and the affected listener not being programmed. Other same-namespace references/listeners need not fail. Verify actual listener conditions rather than treating the entire Pod as broken.

kubectl describe gateway "YOUR_GATEWAY_NAME" -n "YOUR_APP_NAMESPACE"
kubectl get gateway "YOUR_GATEWAY_NAME" -n "YOUR_APP_NAMESPACE" \
  -o jsonpath='{.status.listeners[*].conditions[?(@.type=="ResolvedRefs")]}'

One vault, several certificates

Both SPC objects and secretObjects are arrays. To reproduce the source's two-certificate example, create SPC all-tls-spc in cert-store with the same identity/vault/tenant fields as above; use the following object aliases and Secret mappings. The certificate-object format caveat in section 17 still applies.

# spec.parameters.objects fragment:
objects: |
  array:
    - |
      objectName: tenant-a-cert-pfx
      objectType: secret
      objectAlias: tenant-a-key
    - |
      objectName: tenant-a-cert-pfx
      objectType: cert
      objectAlias: tenant-a-crt
    - |
      objectName: tenant-b-cert-pfx
      objectType: secret
      objectAlias: tenant-b-key
    - |
      objectName: tenant-b-cert-pfx
      objectType: cert
      objectAlias: tenant-b-crt
# Corresponding spec.secretObjects:
secretObjects:
- secretName: tls-tenant-a
  type: kubernetes.io/tls
  data:
  - {objectName: tenant-a-crt, key: tls.crt}
  - {objectName: tenant-a-key, key: tls.key}
- secretName: tls-tenant-b
  type: kubernetes.io/tls
  data:
  - {objectName: tenant-b-crt, key: tls.crt}
  - {objectName: tenant-b-key, key: tls.key}

Move the sync Deployment into cert-store and change its secretProviderClass to all-tls-spc. One mounted volume then covers that SPC's objects. Each Gateway references its intended TLS Secret in cert-store.

ReferenceGrant permissions are the Cartesian product of from[] and to[]. Putting tenant-a and tenant-b plus both TLS Secrets into one grant lets each tenant reference both Secrets. For tenant isolation, make one grant per authorized namespace/Secret pair. Namespace grants also authorize that namespace's Gateways generally, not a single named Gateway.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: tenant-a-to-own-certificate
  namespace: cert-store
spec:
  from:
  - {group: gateway.networking.k8s.io, kind: Gateway, namespace: tenant-a}
  to:
  - {group: "", kind: Secret, name: tls-tenant-a}
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: tenant-b-to-own-certificate
  namespace: cert-store
spec:
  from:
  - {group: gateway.networking.k8s.io, kind: Gateway, namespace: tenant-b}
  to:
  - {group: "", kind: Secret, name: tls-tenant-b}

If all tenants are intentionally allowed to use every certificate, one broad grant is simpler but less isolated. Add certificate objects, Secret mappings and a per-pair grant for each new isolated tenant. Split SPCs when certificates belong to different vaults (keyvaultName is per SPC), when large object counts create polling/throttling pressure, or when ownership/change cadence warrants a smaller failure impact.

Multiple SPCs can still share one sync Deployment: mount one CSI volume per SPC, with corresponding volumeMounts. The source's volume-only sketch is:

volumes:
- name: cert-a
  csi:
    driver: secrets-store.csi.k8s.io
    readOnly: true
    volumeAttributes: {secretProviderClass: spc-cert-a}
- name: cert-b
  csi:
    driver: secrets-store.csi.k8s.io
    readOnly: true
    volumeAttributes: {secretProviderClass: spc-cert-b}

For manual troubleshooting, check missing mount holders, CSI identity permissions, alias/Secret name mismatches, pinned versions, rotation settings, replica placement and ReferenceGrants. Consider Key Vault expiry alerts through Event Grid to a webhook or Logic App. If Key Vault need not be the source of truth, cert-manager with Azure DNS or a supported Gateway HTTPRoute solver is a separate alternative, not configured here.

20. HTTPRoute patterns for the shared Gateway

All patterns below use the standard Gateway API, not VirtualService or EnvoyFilter. Create application namespaces and referenced Services first. The common route envelope is shown once; subsequent rules are fragments to place under the same envelope with the indicated route name, namespace, hostname and listener. They are separate examples, not one set to apply together; overlapping routes require deliberate conflict resolution.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: YOUR_ROUTE_NAME
  namespace: YOUR_ROUTE_NAMESPACE
spec:
  parentRefs:
  - name: shared-gateway
    namespace: gateway-system
    sectionName: https
  hostnames: [api.contoso.io]
  rules: []

Replace YOUR_ placeholders with valid lowercase Kubernetes names. The https examples require a matching HTTPS listener and certificate on shared-gateway; the HTTP-only Gateway in section 7 does not provide that listener. A Service port is the Service's port, not necessarily the container port.

Path routing plus fallback

For console-route in ns-console, hostname console.contoso.io: /orders/list reaches orders-service:8080 as /list, and every other path reaches web-frontend:80.

rules:
- matches:
  - path: {type: PathPrefix, value: /orders}
  filters:
  - type: URLRewrite
    urlRewrite:
      path: {type: ReplacePrefixMatch, replacePrefixMatch: /}
  backendRefs:
  - {name: orders-service, port: 8080}
- matches:
  - path: {type: PathPrefix, value: /}
  backendRefs:
  - {name: web-frontend, port: 80}

The catch-all is a path fallback, not interception of a backend 5xx response. Longer prefixes take precedence regardless of list order; put the fallback last for readability.

Request timeout

For reports-timeout in ns-reports, reports.contoso.io on listener https:

rules:
- matches:
  - path: {type: PathPrefix, value: /reports}
  backendRefs:
  - {name: reports-service, port: 8080}
  timeouts:
    request: 800ms

The request timeout bounds gateway processing of the request/response; exceeding the supported timeout can produce 504. It is not simply time to the first backend byte. backendRequest can separately constrain an individual backend attempt and must fit within request when both are set. Use supported duration values such as 300ms, 2s or 1m. Choose longer limits for batch/report APIs and shorter ones for latency-sensitive calls.

Request and response header modification

For header-modify-demo in ns-demo, echo.contoso.io → echo-service:8080:

rules:
- matches:
  - path: {type: PathPrefix, value: /}
  filters:
  - type: RequestHeaderModifier
    requestHeaderModifier:
      add:
      - {name: x-forwarded-by, value: contoso-gateway}
  - type: ResponseHeaderModifier
    responseHeaderModifier:
      add:
      - {name: x-served-by, value: approuting-istio}
      set:
      - {name: cache-control, value: no-store}
  backendRefs:
  - {name: echo-service, port: 8080}

add appends, set replaces, and remove deletes named headers. The same mechanism can set security headers such as x-frame-options. A client-supplied identification header is not a trustworthy security boundary merely because a proxy also adds one.

Redirect HTTP to HTTPS

For redirect-http-to-https in ns-demo, hostnames [*.contoso.io], change parentRefs.sectionName to http:

rules:
- filters:
  - type: RequestRedirect
    requestRedirect:
      scheme: https
      statusCode: 301

The Gateway returns a permanent redirect without forwarding to a backend. Configure and validate HTTPS first.

Move an old domain

For redirect-legacy-domain, hostname shop-old.contoso.io on https:

rules:
- filters:
  - type: RequestRedirect
    requestRedirect:
      hostname: shop.contoso.io
      statusCode: 301

The path is retained while the client moves to the new hostname. Both the old hostname's TLS certificate and listener match must work for an HTTPS redirect.

Redirect API version /v1 to /v2

For redirect-api-version, api.contoso.io:

rules:
- matches:
  - path: {type: PathPrefix, value: /v1}
  filters:
  - type: RequestRedirect
    requestRedirect:
      path: {type: ReplacePrefixMatch, replacePrefixMatch: /v2}
      statusCode: 301
- matches:
  - path: {type: PathPrefix, value: /}
  backendRefs:
  - {name: api-backend, port: 8080}

The client sees a changed URL. Use URLRewrite instead if the version translation must remain internal.

Strip an API prefix without redirecting

For rewrite-strip-api-prefix, api.contoso.io, /api/users becomes /users at api-backend:8080:

rules:
- matches:
  - path: {type: PathPrefix, value: /api}
  filters:
  - type: URLRewrite
    urlRewrite:
      path: {type: ReplacePrefixMatch, replacePrefixMatch: /}
  backendRefs:
  - {name: api-backend, port: 8080}

Route microservices and remove the version prefix

For rewrite-microservice-routing, app.contoso.io:

rules:
- matches:
  - path: {type: PathPrefix, value: /v1/users}
  filters:
  - type: URLRewrite
    urlRewrite:
      path: {type: ReplacePrefixMatch, replacePrefixMatch: /users}
  backendRefs:
  - {name: users-service, port: 8080}
- matches:
  - path: {type: PathPrefix, value: /v1/orders}
  filters:
  - type: URLRewrite
    urlRewrite:
      path: {type: ReplacePrefixMatch, replacePrefixMatch: /orders}
  backendRefs:
  - {name: orders-service, port: 8080}

Rewrite the backend Host

For rewrite-host-header in ns-demo, incoming legacy.contoso.io, preserve the external URL while sending a service-local Host:

rules:
- filters:
  - type: URLRewrite
    urlRewrite:
      hostname: legacy-app.ns-demo.svc.cluster.local
  backendRefs:
  - {name: legacy-app, port: 8080}

This accommodates legacy backends requiring a specific Host. The service DNS name is an illustrative in-cluster name, not an exposed private hostname.

Replace one complete health path

For rewrite-health-endpoint, api.contoso.io, exact /healthz → /actuator/health:

rules:
- matches:
  - path: {type: Exact, value: /healthz}
  filters:
  - type: URLRewrite
    urlRewrite:
      path: {type: ReplaceFullPath, replaceFullPath: /actuator/health}
  backendRefs:
  - {name: api-backend, port: 8080}

Matching and filter rules

• Exact path is more specific than PathPrefix; a longer prefix wins over a shorter one. Method/header/query matching adds specificity under Gateway API's precedence rules; ties and cross-route conflicts have additional standard rules, so YAML order is not the sole decision mechanism.
• PathPrefix respects path-element boundaries, not arbitrary regex syntax. RegularExpression support is implementation-specific.
• RequestRedirect changes what the client sees; URLRewrite changes the upstream request internally. Do not combine incompatible redirect/rewrite filters within one rule.
• Hostname matching, listener sectionName, allowedRoutes, reference permissions and successful backend resolution must all align.
• Response-status-based custom error pages and a global NGINX-style default backend are not supplied by these standard route examples. A catch-all path does not intercept backend errors.

21. Source-era support limits and managed-control-plane bug

For sticky sessions, the source recommends application-level state/cookie handling, shared state storage or another deliberately chosen supported architecture. Directly patching an unsupported sessionPersistence field is not a solution. The mesh's consistentHash DestinationRule cannot be borrowed into a configuration that forbids those CRDs. Future upstream implementation still needs release adoption and AKS rollout; no delivery date is supplied.

istiod HPA CPU requests missing

The source reports the managed istiod Deployment missing resources.requests.cpu, causing repeated FailedGetResourceMetric and FailedComputeMetricsReplicas events. Without a CPU request denominator the HPA cannot calculate utilization, so it remains at its minimum rather than scaling out. This is the control-plane HPA; the source's Gateway data-plane HPA worked normally.

kubectl get hpa -n aks-istio-system
kubectl describe hpa -n aks-istio-system "YOUR_ISTIOD_HPA_NAME"
kubectl get deploy -n aks-istio-system -l app=istiod \
  -o jsonpath='{.items[*].spec.template.spec.containers[*].resources}'

The source says direct patches were reverted by the add-on reconciler and supplied no customer-side workaround. Open an Azure Support case with the exact build and evidence, track a managed fix, and distinguish known event noise from new failures. Normal data-plane scaling does not prove unlimited control-plane capacity.

22. Migration and operational acceptance checklist

1. Confirm CLI version, regional rollout, managed Standard CRDs, no conflicting mesh and an Accepted approuting-istio class.
2. Establish the NGINX baseline and retain it while testing a parallel Gateway. Do not reuse an in-use static LB IP or change DNS prematurely.
3. Review all conversion artifacts, mismatched classes, regex/rewrite behavior, timeouts and unsupported legacy annotations.
4. Verify one Gateway creates the expected Deployment, Service, HPA and PDB; choose private/public exposure and namespace topology deliberately.
5. Use the operator TLS path where available, or explicitly own manual CSI mounts, rotation and certificate grants. Test trust, SNI, expiry, rotation and a new handshake.
6. Validate NetworkPolicy on external-to-Gateway and Gateway-to-backend paths, client-IP preservation and every LB probe/port, including 443.
7. Test route precedence, headers, redirects, request sizes, long-running traffic, error handling, session state and disruption budgets.
8. Collect access logs and distinguish control-plane HPA issues from data-plane errors.
9. Plan DNS or fronting Application Gateway cutover with a reversal to the old backend. Preserve the original Host if routes depend on it, and avoid competing DNS reconcilers.
10. Retire the old ingress only after traffic and operational acceptance. The source refers to a separate migration guide that is absent from the inventoried repository; no missing detailed cutover script has been fabricated.

23. Cleanup and retention cautions

Run cleanup only for confirmed test resources, after checking the active cluster and namespace. Remove dependent routes and gateways before certificate infrastructure. The original manual cleanup incorrectly deletes a Pod after recommending a Deployment; the corrected resource kind is shown.

kubectl delete -n "$APP_NS" gateway httpbin-gateway
kubectl delete -n "$APP_NS" httproute httpbin
kubectl delete -n "$APP_NS" deployment secrets-store-sync-httpbin
kubectl delete -n "$APP_NS" secretproviderclass httpbin-credential-spc
kubectl delete -n "$APP_NS" secret httpbin-credential
kubectl delete -n "$APP_NS" -f https://raw.githubusercontent.com/istio/istio/release-1.27/samples/httpbin/httpbin.yaml

# Only when no remaining Gateway workload needs the implementation:
az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" --disable-app-routing-istio
# Removing managed Gateway API CRDs affects every dependent Gateway/Route:
# az aks update -g "$RESOURCE_GROUP" -n "$CLUSTER" --disable-gateway-api

# Only for a disposable vault no longer used by any workload:
# az keyvault delete --name "$AKV_NAME" --resource-group "$RESOURCE_GROUP"
# az keyvault purge --name "$AKV_NAME" --location "$LOCATION"

Purge protection prevents permanent deletion before retention expires; do not disable safeguards or assume purge succeeds immediately. Operator-created resources and UAMI/FIC/role assignments should be reviewed against remaining consumers rather than deleted indiscriminately. The operator update does not supply a complete identity cleanup recipe.

References and publication boundary

• Application Routing Gateway API
• Managed Gateway API installation
• Gateway customization allowlist
• Annotation customization
• Current operator DNS/TLS integration documentation
• Historical TLS documentation link
• PDB drain failure troubleshooting
• Envoy response flags
• Public access-log issue
• Upstream session persistence issue
• Historical CSI synchronization issue
• cert-manager Gateway usage

This article preserves the source's measured observations, limitations and incomplete investigations while labeling outdated claims and unsafe assumptions. Public references are retained for verification; no private repository links, screenshots, credential values or original environment mappings are published.

Resources

Source-derived, parameterized resources. Review every placeholder, permission, dependency, and deployment effect before use. These files are not a one-click deployment.

• current_env/converted-gateway.yaml

  Sanitized historical ingress2gateway 0.5.0 output; requires class, infrastructure and feature review.

• current_env/gwapi.yaml

  Sanitized historical ingress2gateway 1.0.0 output retaining regex, full-path rewrite and timeout limitations.

• current_env/internal-ingress.yaml

  Legacy NGINX annotation inventory with explicit controller-class and TLS configuration warnings.

• current_env/internal-nginx-controller.yaml

  Application Routing NGINX controller using an internal Azure LoadBalancer.

• current_env/sample-app.yaml

  Echo application namespace, two-replica Deployment and port-80 Service.

• tobe_env/gateway.yaml

  Sanitized target HTTP-only approuting-istio Gateway and echo route; not full NGINX feature parity.

Editorial notes

This page consolidates the following source documents into an English technical guide:

• HTTPRoute Configuration Guide for Application Routing Gateway API
• Application Routing Gateway API GA Deployment Guide
• Application Routing Gateway API: Operator-Managed Azure Key Vault TLS Integration
• Application Routing Gateway API: Manual Azure Key Vault TLS Integration
• Configuring an Internal NGINX Ingress with Application Routing
• Conversion Output from ingress2gateway 0.5.0
• Conversion Output from ingress2gateway 1.0.0
• Legacy NGINX Ingress Annotation Inventory
• Internal NGINX Controller Manifest
• Echo Sample Application Manifest
• Target Istio Gateway and HTTPRoute Manifest

• Repeated setup and TLS material is consolidated; the operator revision is clearly preferred and the earlier manual chain remains documented as a fallback.
• Observed cloud identifiers, private hostnames, node/Pod names, network addresses and request IDs are removed or replaced without publishing an original-value mapping.
• The referenced standalone migration guide is absent from the repository inventory; no contents or files are invented.
• The masked/broken PFX command is not reconstructed; safe explicit export/import operations are described with an input password variable.
• The reported empty health-probe annotations are preserved only as a historical reproduction, not a validated deployment fix; port 443 and product-fix ETA remain unresolved.
• Common HTTPRoute envelopes and repeated certificate object patterns are consolidated into labeled fragments with all substantive rule behavior retained.

No original credentials or private repository links are included. Do not put populated configuration files or copied production outputs back into this public site.