Migrating AKS from No Network Policy to Cilium

This article translates a validation report recorded on May 21, 2026 in East US 2. The lab used a two-node Azure CNI Overlay cluster with Kubernetes 1.34.7 and Standard_B2s VMs. All environment identifiers and observed addresses have been removed. Timings below are historical observations from that one lab, not a promise of production availability.

1. Choose a policy engine and understand the migration risk

The source recommends Cilium to make standard NetworkPolicy enforcement effective while retaining Overlay, and cites Azure NPM Linux retirement on September 30, 2028, plus a Windows retirement window of September 2026. These dates and platform support are a source snapshot; verify current Microsoft lifecycle guidance. Upstream Cilium capabilities do not automatically mean every feature is available in every AKS managed configuration.

This is not a zero-downtime switch. Nodes are replaced sequentially, Pods are evicted, and the source states that an in-place rollback from Cilium to the Azure dataplane is unavailable. Test a representative staging cluster, back up configuration, and plan recovery through another cluster if necessary.

2. Preflight production readiness

1. Inventory custom node iptables rules with iptables -L -n. eBPF datapath behavior may bypass assumptions made by those rules; port required controls to supported policies and verify traffic.
2. Inspect externalTrafficPolicy: Local on NodePort/LoadBalancer Services and test source-IP preservation after kube-proxy replacement.
3. Check nodeProvisioningProfile. The report says dataplane change is blocked while Node Auto-Provisioning is enabled; plan disable, transition, and re-enable using supported procedures.
4. Allow node headroom for cilium-agent. The report suggests approximately 100m CPU and 200Mi memory per node as an operational estimate, not a guaranteed request or limit.
5. Review PDBs, application replicas, surge capacity, drain behavior, readiness, and pod distribution. Two or more replicas plus a feasible disruption budget are safer than the single-replica lab.
6. Schedule the actual update in an approved maintenance window. Merely configuring an AKS maintenance policy should not be assumed to defer a manually initiated command.
7. Prepare policy and flow observability. Hubble availability must be checked; the relay query below detects a relay but does not install or enable it.

export RESOURCE_GROUP="YOUR_RESOURCE_GROUP"
export CLUSTER="YOUR_CLUSTER"
az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" --query nodeProvisioningProfile
kubectl get svc -A -o yaml | grep externalTrafficPolicy
kubectl top nodes
kubectl get pdb -A
kubectl get pods -n kube-system -l k8s-app=hubble-relay

3. Build a no-policy baseline

The following is a disposable lab, not an instruction to recreate a production cluster. The initial profile should report networkDataplane=azure, networkPlugin=azure, networkPluginMode=overlay, and networkPolicy=none.

az aks create --resource-group "$RESOURCE_GROUP" --name "$CLUSTER" \
  --location eastus2 --node-count 2 --node-vm-size Standard_B2s \
  --network-plugin azure --network-plugin-mode overlay --network-policy none \
  --enable-managed-identity --generate-ssh-keys
az aks get-credentials -g "$RESOURCE_GROUP" -n "$CLUSTER"
kubectl create namespace netpol-test
az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" --query networkProfile
kubectl get nodes

Deploy the two-tier application

Save the backend Deployment and Service as backend.yaml, and the frontend as frontend.yaml. These are inline examples, not additional source downloads. The namespace creation above supplies a step omitted from the report's deployment block.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend
  namespace: netpol-test
spec:
  replicas: 1
  selector:
    matchLabels: {app: backend}
  template:
    metadata:
      labels: {app: backend}
    spec:
      containers:
      - name: backend
        image: nginx:1.25-alpine
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: backend
  namespace: netpol-test
spec:
  selector: {app: backend}
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: netpol-test
spec:
  replicas: 1
  selector:
    matchLabels: {app: frontend}
  template:
    metadata:
      labels: {app: frontend}
    spec:
      containers:
      - name: frontend
        image: curlimages/curl:8.8.0
        command: ["/bin/sh", "-c", "sleep 36000"]

kubectl -n netpol-test exec deploy/frontend -- \
  sh -c "curl -s -o /dev/null -w '%{http_code}' http://backend"
# Recorded outcome: exit code 0, HTTP 200.

With no policy enforcement, same-namespace traffic succeeded. Image versions are retained from the test and should be reviewed for security and availability before reuse.

4. Transition and monitor availability

az aks update --resource-group "$RESOURCE_GROUP" --name "$CLUSTER" \
  --network-dataplane cilium --network-policy cilium
az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER" \
  --query '{dataplane:networkProfile.networkDataplane,policy:networkProfile.networkPolicy,state:provisioningState}'
kubectl get nodes
kubectl -n netpol-test get pods -o wide
kubectl -n netpol-test exec deploy/frontend -- \
  sh -c "curl -s -o /dev/null -w '%{http_code}' http://backend"

From a second terminal, sample node readiness and frontend-to-backend requests approximately every ten seconds while the update runs. The report used a Kubernetes exec-based probe. It recorded one EXEC_FAIL, which may include a Pod eviction or exec failure and is not proof of an exactly eleven-second application outage.

The observed replacement sequence was maxSurge: add a new node, make the old node unschedulable and drain it, delete the old node, then repeat. It was not simultaneous reimaging. Final networkDataplane and networkPolicy were both cilium; both nodes and workloads were Ready/Running, and the post-update request returned exit code 0 and HTTP 200.

5. Move the backend to a separate namespace

Create a backend-only copy of the YAML above with metadata.namespace: backend-ns on both resources. The -n flag does not override a conflicting namespace written in a manifest. Moving the backend is deliberately disruptive in this test.

kubectl create namespace backend-ns
kubectl -n netpol-test delete deployment backend
kubectl -n netpol-test delete service backend
# backend.yaml must now contain only the backend resources with namespace backend-ns.
kubectl -n backend-ns apply -f backend.yaml
kubectl -n netpol-test exec deploy/frontend -- \
  sh -c "curl -s -o /dev/null -w '%{http_code}' http://backend.backend-ns.svc.cluster.local"
# Recorded: exit code 0, HTTP 200.

Cilium installation alone does not default-deny all traffic. Cross-namespace traffic remained allowed before selecting workloads with policies.

6. Deny egress except DNS

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-egress
  namespace: netpol-test
spec:
  podSelector: {}
  policyTypes: [Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
  namespace: netpol-test
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          k8s-app: kube-dns
    ports:
    - protocol: UDP
      port: 53
    - protocol: TCP
      port: 53

The namespaceSelector and podSelector are in the same peer entry: both must match. UDP and TCP DNS are allowed. Policies are additive; the DNS allow policy adds permitted egress to the otherwise isolated Pods.

kubectl -n netpol-test exec deploy/frontend -- \
  sh -c "curl -sS --connect-timeout 5 -m 8 -o /dev/null -w '%{http_code}' http://backend.backend-ns.svc.cluster.local"
# Recorded: curl exit code 28; connection timed out after about 5002 ms.

7. Allow only frontend to reach backend

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
  namespace: netpol-test
spec:
  podSelector:
    matchLabels: {app: frontend}
  policyTypes: [Egress]
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: backend-ns
      podSelector:
        matchLabels: {app: backend}
    ports:
    - protocol: TCP
      port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: other
  namespace: netpol-test
spec:
  replicas: 1
  selector:
    matchLabels: {app: other}
  template:
    metadata:
      labels: {app: other}
    spec:
      containers:
      - name: other
        image: curlimages/curl:8.8.0
        command: ["/bin/sh", "-c", "sleep 36000"]

for APP in frontend other; do
  kubectl -n netpol-test exec "deploy/$APP" -- \
    sh -c "curl -sS --connect-timeout 5 -m 8 -o /dev/null -w '%{http_code}' http://backend.backend-ns.svc.cluster.local"
done

8. Cilium-specific operational examples

Standard NetworkPolicy covers L3/L4 addresses and ports. CiliumNetworkPolicy adds Cilium-specific selectors and, where supported, FQDN and L7 controls such as allowing only particular HTTP methods or paths. The following examples reproduce each policy pattern in the source. The private CIDRs below are illustrative RFC1918 ranges, not addresses from the lab. Replace them with approved subnets or endpoints.

Application Gateway subnet to frontend

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-ingress-from-appgw
  namespace: demo1
spec:
  endpointSelector:
    matchLabels: {app: frontend}
  ingress:
  - fromCIDR:
    - 10.100.2.0/24 # Illustrative Application Gateway subnet, not a real environment value.
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP

Cross-namespace egress to a service tier

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-egress-to-demo2
  namespace: demo1
spec:
  endpointSelector:
    matchLabels: {app: frontend}
  egress:
  - toEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: demo2
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP

CoreDNS access

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-dns
  namespace: demo1
spec:
  endpointSelector: {}
  egress:
  - toEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: kube-system
        k8s-app: kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: UDP
      - port: "53"
        protocol: TCP

External API by FQDN

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-egress-google-api
  namespace: demo1
spec:
  endpointSelector:
    matchLabels: {app: backend-api}
  egress:
  - toFQDNs:
    - matchPattern: "*.googleapis.com"
    - matchPattern: "*.google.com"
    toPorts:
    - ports:
      - port: "443"
        protocol: TCP

The source's DNS example only permits port 53; it does not demonstrate the DNS proxy/observation rules used to populate FQDN policy state. Do not assume the FQDN block works merely by allowing DNS transport. Check AKS/Cilium support, DNS visibility, and required DNS L7 rules, then verify resolved addresses and dropped flows. Wildcards also do not necessarily match the bare apex domain.

MySQL private endpoint

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-egress-mysql-pe
  namespace: demo1
spec:
  endpointSelector:
    matchLabels: {app: backend-api}
  egress:
  - toCIDR:
    - 10.200.1.5/32 # Illustrative private endpoint only; replace before use.
    toPorts:
    - ports:
      - port: "3306"
        protocol: TCP

9. Progressive enforcement and troubleshooting

1. Complete the dataplane transition and confirm provisioningState=Succeeded.
2. Verify existing services before applying restrictive policies.
3. Apply ingress allow policies to explicitly selected workloads. Ingress becomes isolated for that selected direction; a separate deny-all ingress policy is not always necessary.
4. Harden egress in stages: deny-all, DNS, namespace/label-selected services, supported FQDN-based external APIs, and IP-based databases.
5. Check both ends of each connection. An egress exception does not override an independent backend ingress deny policy.
6. Use Hubble or Cilium drop monitoring to find missing permissions and refine gradually. Do not overstate a policy's reach: enforcement is determined by selected endpoints and directions, not by the mere existence of a policy somewhere in the cluster.

10. Remove only the disposable lab

# Destructive: verify the current subscription, resource group and cluster first.
az aks delete -g "$RESOURCE_GROUP" -n "$CLUSTER" --yes
az aks show -g "$RESOURCE_GROUP" -n "$CLUSTER"

The source's cleanup ran from 05:09:43Z to 05:14:48Z on the validation date. The subsequent show command returned ResourceNotFound (exit code 3), confirming deletion. Cluster deletion is not a rollback strategy for a production workload.

References and limitations

• AKS network policies
• Azure CNI powered by Cilium
• Update Azure CNI
• NPM to Cilium migration
• Cilium policy documentation

The report verifies the six lab outcomes: initial same-namespace success, successful transition with one failed probe, cross-namespace success without policy, egress blocking with DNS retained, frontend-specific allowance, and other-workload blocking. It does not establish production availability, Windows behavior, or measured L7/FQDN/Hubble operation. No cluster commands were executed for this publication.

Resources

No standalone companion files are distributed for this topic. Use the in-page examples and review the editorial notes below for source availability.

Editorial notes

This page consolidates the following source documents into an English technical guide:

• AKS Network Policy Migration Validation Report: None to Cilium

• Observed node and Pod names, resource identifiers, and real network addresses are omitted; test timestamps, versions, measured durations and outcomes are preserved.
• No standalone YAML file exists in this folder; all original policy patterns remain inline rather than inventing resource downloads.
• Claims about upstream feature availability and retirement dates are retained as source-era guidance with explicit verification caveats.

No original credentials or private repository links are included. Do not put populated configuration files or copied production outputs back into this public site.