Azure Kubernetes Service Cost Optimization: Right-Sizing Node Pools and Autoscaling Policies

Executive Snapshot

Every figure in this article is a planning range rather than a quote. VM prices, control-plane tier pricing, spot discounts, and Log Analytics ingestion rates vary by region, series, and agreement, and Microsoft revises them. Verify current pricing for your exact SKUs and regions in the Azure Pricing Calculator before you build a business case on any number here.

---

TL;DR

• AKS bills you for nodes, not for Kubernetes. The control plane is a small line item; the agent pool VMs, their disks, load balancers, egress, and log ingestion are the bill.
• Requests, not usage, decide how many nodes you run. The scheduler packs against requests, so a fleet requesting three times what it uses will run roughly three times the nodes it needs.
• Profile workloads before you profile nodes. Get requests honest first, then pick node SKUs whose CPU-to-memory ratio matches the aggregate shape of those requests.
• A cluster autoscaler at defaults is conservative on the way down. Scale-down thresholds, unneeded time, and — most of all — pods that cannot be evicted are why underutilised nodes stay alive for days.
• Spot node pools carry the biggest discount and the only real availability risk. Taints and tolerations keep the wrong workloads off them; graceful eviction handling keeps the right ones honest.
• Make cost visible per namespace before you negotiate with teams. The AKS cost analysis add-on or an OpenCost/Kubecost deployment turns "the cluster costs too much" into a chargeback conversation.
• Commitments go under node pools, not around them. A reservation or savings plan discounts the VMs your pools run; size it to the autoscaler's floor, never its peak.

---

Introduction

AKS cost problems rarely look like cost problems. They look like a cluster that has thirty nodes when the graphs say the workloads use eight, an autoscaler that adds capacity readily and gives it back reluctantly, and a monthly invoice where nobody can name which team owns which slice.

The mechanics behind that are boring and consistent. Kubernetes schedules on requests. Requests are set by developers, usually once, usually generously, usually never revisited. The cluster autoscaler adds a node when a pod is unschedulable and removes one only when it is confident every pod on that node can move somewhere else — a condition your Pod Disruption Budgets, local storage, and system pods quietly veto. Meanwhile the node pool itself may be an E-series memory-optimised SKU running CPU-bound services, so you are buying RAM that no pod ever asks for.

This guide walks the sequence that actually reduces the number: understand where the money goes, right-size the workloads, then the node pools, then the autoscaler, then layer spot and commitments on top. It closes with the waste patterns ranked by how much they usually cost, because the order you attack them in matters more than the individual techniques.

---

Where AKS Money Actually Goes

Start by separating the bill into its real components, because optimisation effort aimed at the wrong one is wasted.

Agent pool virtual machines. This is almost always the dominant cost and the entire subject of the rest of this guide. You pay standard VM rates for every node in every node pool, running or idle, whether pods are scheduled on it or not.

The control plane. AKS offers a Free tier (no financially backed SLA), a Standard tier billed per cluster-hour with an SLA, and a Premium tier that adds long-term support at a higher hourly rate. The Standard tier is commonly cited at roughly $0.10 per cluster-hour — verify the current pricing tiers, because the tier names and rates have changed since AKS launched. Either way, a single cluster's control plane is normally a rounding error next to its nodes. Consolidating five clusters into one to save control-plane fees is usually the wrong trade if it costs you blast-radius isolation.

Storage. Every node has an OS disk. Every PersistentVolumeClaim backed by Azure Disk is a managed disk billed on provisioned size, not consumed size — a 1 TiB Premium SSD holding 40 GiB of data bills as 1 TiB. Deleted workloads with a Retain reclaim policy leave those disks behind, still billing.

Networking. Standard Load Balancer rules, public IPs, NAT Gateway, and cross-zone or egress data transfer. Egress is the one that surprises people — a chatty service pulling images or telemetry out of the region can outgrow its own compute cost.

Observability. Container Insights sends container logs and metrics to Log Analytics, billed on ingestion and retention. On a busy cluster with verbose applications this regularly lands in the top three line items, and it is the easiest one to fix because most of the volume is debug-level stdout nobody reads.

A useful first pass: open Cost Management, filter to the node resource group (MC_<rg>_<cluster>_<region> by default), and group by meter category. If storage or Log Analytics is more than a fifth of the total, fix that before you touch node pools.

---

Step 1: Profile the Workload Before the Node Pool

The single highest-leverage number in AKS cost work is the ratio between what pods request and what they use. Requests reserve capacity on a node whether or not the container consumes it, so an over-requested fleet buys nodes to hold reservations.

Start with the cluster-level view. kubectl describe node reports allocated requests as a percentage of allocatable capacity — that is your bin-packing efficiency:

# Requested vs allocatable, per node — the packing view
kubectl describe nodes | grep -A 6 "Allocated resources"

# Actual live consumption, per node and per pod (needs the metrics server,
# which AKS installs by default)
kubectl top nodes
kubectl top pods --all-namespaces --sort-by=cpu

If kubectl describe says 85% of CPU is requested while kubectl top says 12% is used, you have found your problem and it is not the autoscaler.

For a defensible per-workload number you need history, not a snapshot. If Container Insights is enabled, query the requested-versus-used gap over a representative fortnight:

// Average CPU actually used vs requested, per container, last 14 days.
// Ratios far below 1 are over-requested workloads.
let used = Perf
| where TimeGenerated > ago(14d)
| where ObjectName == "K8SContainer" and CounterName == "cpuUsageNanoCores"
| summarize UsedMilliCores = avg(CounterValue) / 1000000 by InstanceName;
let requested = Perf
| where TimeGenerated > ago(14d)
| where ObjectName == "K8SContainer" and CounterName == "cpuRequestNanoCores"
| summarize RequestedMilliCores = avg(CounterValue) / 1000000 by InstanceName;
used
| join kind=inner requested on InstanceName
| extend Efficiency = round(UsedMilliCores / RequestedMilliCores, 2)
| where RequestedMilliCores > 0
| project InstanceName, UsedMilliCores = round(UsedMilliCores, 1),
          RequestedMilliCores = round(RequestedMilliCores, 1), Efficiency
| order by Efficiency asc

Counter names and table shapes differ between classic Container Insights and Azure Monitor managed Prometheus. If this query returns nothing, check which collection mode your cluster uses and adapt the source table rather than assuming the data is missing.

Let the Vertical Pod Autoscaler do the arithmetic. AKS ships VPA as a managed add-on. Run it in recommendation-only mode — updateMode: "Off" — so it observes and advises without restarting pods. This is the safe way to get per-workload numbers you can defend in a pull request:

# Enable the managed VPA add-on (part of the AKS workload autoscaler)
az aks update \
  --resource-group rg-prod-aks \
  --name aks-prod-weu \
  --enable-vpa

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api-vpa
  namespace: payments
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  updatePolicy:
    updateMode: "Off"          # recommend only — never evicts a pod
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: 25m
          memory: 64Mi
        maxAllowed:
          cpu: "2"
          memory: 4Gi

Read the recommendation with kubectl describe vpa api-vpa -n payments. It reports a target, plus lowerBound and upperBound. Move requests toward target, keep memory limits close to requests (memory is not compressible — exceeding a limit is an OOMKill, not throttling), and leave CPU limits generous or unset so bursty work can use idle cycles.

Two rules that prevent self-inflicted incidents. Never run VPA in Auto mode on the same Deployment as a CPU-based HPA — they fight over the same signal. And apply a LimitRange per namespace so that new workloads land with a sane default instead of inheriting whatever a copied YAML file contained.

---

Step 2: Node Pool Architecture

Only once requests reflect reality does node SKU selection become meaningful — before that you are sizing hardware to fictional demand.

System and user pools are different jobs

Every AKS cluster has a system node pool hosting CoreDNS, metrics-server, and the rest of kube-system. Keep application workloads off it. A system pool should be small, stable, and boring; user pools should be where all the scaling drama happens.

# A lean system pool: taint it so only critical add-ons schedule there
az aks nodepool add \
  --resource-group rg-prod-aks --cluster-name aks-prod-weu \
  --name systempool --mode System \
  --node-vm-size Standard_D4s_v5 \
  --node-count 3 --zones 1 2 3 \
  --node-taints CriticalAddonsOnly=true:NoSchedule

Three nodes across zones is the usual production floor for a system pool; a single-node system pool is a control-plane-adjacent outage waiting to happen. A user pool, by contrast, can scale to zero when idle — the system pool cannot.

Match the SKU family to the workload shape

Azure VM families differ mainly in their vCPU-to-memory ratio. Pick the family whose ratio matches the aggregate of your (now honest) requests:

If your workloads request 100 vCPU and 200 GiB in aggregate, an E-series pool means you buy roughly 800 GiB of RAM to get those 100 cores. That is a permanent overpay no autoscaler can rescue. Arm64 pools are worth a genuine look when your images build multi-arch — the price-performance gap is real, but confirm every dependency, sidecar, and agent has an arm64 build before committing a production pool.

Split pools by workload class, not by team

flowchart TD
    A[Workload] --> B{Interruptible}
    B -->|Yes| C[Spot user pool]
    B -->|No| D{Needs special hardware}
    D -->|GPU or memory heavy| E[Dedicated user pool]
    D -->|No| F{Steady 24x7 baseline}
    F -->|Yes| G[On-demand baseline pool]
    F -->|No| H[Autoscaling burst pool]
    C --> I[Taints plus tolerations]
    E --> I
    G --> I
    H --> I

Three or four pools separated by workload class is usually the sweet spot: a system pool, an on-demand baseline pool, an autoscaling burst pool, and a spot pool. More pools than that fragments capacity and makes bin-packing worse, not better. Pools split per team are a chargeback solution to a scheduling problem — use labels and namespace cost reporting instead.

Two node-level settings with direct cost impact:

• Ephemeral OS disks. When the chosen VM size has enough local cache, AKS defaults to ephemeral OS disks, which removes a managed disk charge per node and speeds up node provisioning. Verify with az aks nodepool show --query osDiskType; if it reads Managed on a size that supports ephemeral, you are paying for disks you did not need.
• --max-pods. The per-node pod cap interacts with your IP plan and your density. Setting it too low leaves nodes half-empty with capacity to spare; it is fixed at pool creation, so getting it wrong means building a replacement pool.

---

Step 3: Tune the Cluster Autoscaler

Enable the autoscaler per user pool, with an honest minimum. The minimum is what you will run at 3 a.m. — and, not coincidentally, what you should size a reservation against.

az aks nodepool update \
  --resource-group rg-prod-aks --cluster-name aks-prod-weu \
  --name userpool \
  --enable-cluster-autoscaler --min-count 2 --max-count 20

The defaults are deliberately cautious about removing nodes. The tunables that matter live in the cluster-wide autoscaler profile:

az aks update \
  --resource-group rg-prod-aks --name aks-prod-weu \
  --cluster-autoscaler-profile \
    scale-down-utilization-threshold=0.65 \
    scale-down-unneeded-time=5m \
    scale-down-delay-after-add=10m \
    expander=least-waste \
    max-graceful-termination-sec=300

Change one or two settings, then watch for a week. Aggressive scale-down that triggers repeated scale-up costs you node provisioning time, image pulls, and cold starts — churn is a real cost even when it is not on the invoice.

The underutilised-but-cannot-evict trap

This is where most AKS clusters lose money, and it is invisible unless you go looking. The autoscaler will not remove a node it cannot fully drain. The usual vetoes:

1. Pod Disruption Budgets that permit no disruption. A PDB with minAvailable equal to the replica count, or maxUnavailable: 0, means the autoscaler can never evict that pod. It respects the PDB — correctly — and abandons the node. This is the same mechanism that blocks node drains during upgrades, which is why PDB hygiene pays twice.
2. Pods using local storage. With skip-nodes-with-local-storage=true, any pod with an emptyDir or hostPath volume pins its node forever. Annotate pods whose scratch data is genuinely disposable.
3. Bare pods. Anything not managed by a controller (Deployment, StatefulSet, Job) will not be recreated elsewhere, so the autoscaler leaves it alone.
4. kube-system pods without a PDB, and DaemonSet pods, which do not block removal but do consume the requests that keep utilisation above your threshold.
5. Restrictive affinity or nodeSelector that leaves the pod nowhere else to go.

Write PDBs that protect availability without forbidding movement, and annotate disposable scratch volumes:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-pdb
  namespace: payments
spec:
  # With 5 replicas this allows one to move at a time.
  # minAvailable: 5 with 5 replicas would block scale-down entirely.
  minAvailable: 4
  selector:
    matchLabels:
      app: api
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: batch-worker
  namespace: payments
spec:
  template:
    metadata:
      annotations:
        # Scratch space in emptyDir is disposable — let the node go
        cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: worker
          image: registry.example.com/batch-worker:1.4.2

When scale-down stalls, the autoscaler says why. Read its status rather than guessing:

kubectl get configmap cluster-autoscaler-status -n kube-system -o yaml
kubectl -n kube-system logs -l app=cluster-autoscaler --tail=200 | grep -i "scale.down\|unremovable"

---

Step 4: Spot Node Pools for Interruptible Work

Spot node pools run on Azure surplus capacity at a large discount — commonly cited in the 60-90% range, varying continuously by size and region, so verify current pricing for your target SKU. The trade is that Azure can reclaim the node with 30 seconds of notice, and there is no SLA.

az aks nodepool add \
  --resource-group rg-prod-aks --cluster-name aks-prod-weu \
  --name spotpool --priority Spot \
  --eviction-policy Delete --spot-max-price -1 \
  --node-vm-size Standard_D8s_v5 \
  --enable-cluster-autoscaler --min-count 0 --max-count 30 \
  --no-wait

--spot-max-price -1 means "pay up to the pay-as-you-go rate", which eliminates price-based eviction and leaves only capacity eviction. Spot pools cannot be system pools, and --min-count 0 lets the pool disappear entirely when there is no interruptible work.

AKS taints spot nodes automatically with kubernetes.azure.com/scalesetpriority=spot:NoSchedule, so nothing lands there by accident. Opt in explicitly:

spec:
  tolerations:
    - key: "kubernetes.azure.com/scalesetpriority"
      operator: "Equal"
      value: "spot"
      effect: "NoSchedule"
  affinity:
    nodeAffinity:
      # Prefer spot, but stay schedulable if the spot pool is empty
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          preference:
            matchExpressions:
              - key: kubernetes.azure.com/scalesetpriority
                operator: In
                values: ["spot"]

Use preferred affinity for work that should run somewhere even when spot capacity is gone, and required only when you would rather the job wait than pay full price.

Handle eviction as the normal case, not the exception. AKS node auto-drain reacts to Azure Scheduled Events — including the Preempt event that precedes a spot eviction — by cordoning and draining the node, but 30 seconds is a short window and delivery is best-effort. Design so that abrupt loss is survivable:

• Keep terminationGracePeriodSeconds well inside the window for spot workloads; a 600-second grace period is fiction on a spot node.
• Implement a SIGTERM handler that stops accepting work, finishes or abandons the current item, and exits — do not wait for a clean shutdown you may not get.
• Make work items idempotent and driven by a queue with visibility timeouts, so an unacknowledged item is redelivered rather than lost.
• Spread across two or three VM sizes in separate spot pools; a capacity crunch is usually size-specific.
• Keep PDBs on spot workloads permissive. A strict PDB does not prevent an Azure eviction — it only slows the graceful drain that precedes it.

Good spot candidates: CI runners, batch ETL, media processing, ML training with checkpointing, and ephemeral preview environments. Bad candidates: anything holding state on local disk, singleton controllers, and ingress.

---

Step 5: Make Cost Visible Per Namespace

Cluster-level cost cannot be negotiated with anyone. Namespace-level cost can.

The AKS cost analysis add-on surfaces cost broken down by namespace, controller, and idle capacity directly in Cost Management. It requires the cluster to be on the Standard or Premium tier:

az aks update \
  --resource-group rg-prod-aks --name aks-prod-weu \
  --enable-cost-analysis

# Confirm it is on
az aks show -g rg-prod-aks -n aks-prod-weu \
  --query "metricsProfile.costAnalysis.enabled"

Data takes a day or so to appear, and the add-on requires the cluster's tier to remain eligible — downgrading to Free turns it off. For chargeback with more granular allocation rules, or for multi-cloud estates, OpenCost (the CNCF project) and Kubecost (its commercial superset) deploy into the cluster and expose per-namespace allocation including idle cost. Whichever you pick, the number that changes behaviour is idle cost per namespace — the capacity a team's requests reserved and did not use.

While you are here, cap observability spend. Container Insights cost is driven by ingested log volume, so configure collection deliberately: exclude noisy namespaces from stdout collection in the container-azm-ms-agentconfig ConfigMap, and evaluate the Basic logs tier for ContainerLogV2 where you only need occasional queries rather than alerting.

---

Step 6: Commitments Under Node Pools

Reservations and savings plans discount the VMs your node pools run. Nothing about AKS changes how they apply, but two details matter:

• Reservations bind to a VM series and region. A three-year reservation on Standard_D8s_v5 in West Europe stops applying the day you rebuild that pool on E-series or move region. Buy them for baseline pools whose SKU has been stable for a year, not for burst pools.
• Savings plans follow the fleet. Because an Azure savings plan commits to hourly spend rather than a SKU, it survives node pool rebuilds, SKU migrations, and region moves. For clusters still evolving, that flexibility is usually worth the slightly smaller discount.

Size the commitment against the autoscaler's floor — the sum of --min-count across on-demand pools plus the system pool — not the average and certainly not the peak. And exclude spot pools from the calculation entirely: spot usage is never eligible for reservation or savings plan discounts, so including it produces a commitment with nothing to attach to.

The full comparison of the three commitment models, including exchange and cancellation rules, is covered in the companion article linked at the end.

---

Step 7: Stop Paying for Dev Clusters Overnight

A non-production cluster running 24/7 for a team that works 40 hours a week is roughly a 75% overpay. AKS supports stopping the whole cluster, which deallocates the nodes and stops control-plane billing:

# Stop the entire cluster (nodes deallocated; disks still bill)
az aks stop --resource-group rg-dev-aks --name aks-dev-weu

# Start it again
az aks start --resource-group rg-dev-aks --name aks-dev-weu

# Or stop a single pool on a cluster that must stay reachable
az aks nodepool stop \
  --resource-group rg-dev-aks --cluster-name aks-dev-weu --name userpool

Drive both from a scheduled job — a GitHub Actions cron workflow, an Azure Automation runbook, or a Logic App. Two cautions: managed disks continue to bill while stopped, and a stopped cluster still ages, so keep an eye on node image and Kubernetes version support windows for clusters that sleep for weeks.

---

The Waste Patterns, Ranked

Roughly in order of how much they cost the average cluster:

1. Requests set far above real usage. The root cause of most over-provisioned clusters. Fix with VPA recommendations before anything else.
2. Scale-down blocked by PDBs, local storage, or bare pods. The autoscaler is working; something is vetoing it. Read cluster-autoscaler-status before blaming the tuning.
3. Non-production clusters running around the clock. Pure calendar arithmetic, and the easiest win to automate.
4. A node SKU family that does not match the workload shape. Paying for memory nobody requests, permanently.
5. No spot pool for obviously interruptible work. CI runners and nightly batch on on-demand nodes is the most common version.
6. Unbounded Container Insights ingestion. Debug logs from a chatty service, retained for months, queried never.
7. Orphaned disks, snapshots, public IPs, and load balancer rules left behind by deleted workloads with Retain reclaim policies.
8. Oversized system pools. Six large nodes hosting CoreDNS and metrics-server.
9. No commitment on a stable baseline that has run unchanged for a year at list price.
10. Too many small clusters, each carrying its own system pool and control plane, where namespaces with proper isolation would do.

---

Troubleshooting

Nodes sit at 20% utilisation and never scale down

Check the autoscaler status ConfigMap for unremovable nodes and the stated reason. In order of likelihood: a PDB that permits no disruption, a pod with an emptyDir volume under skip-nodes-with-local-storage=true, a bare pod with no controller, or a kube-system pod without a PDB.

The autoscaler adds nodes even though kubectl top shows the cluster is idle

Scheduling is driven by requests, not usage. A pod requesting 4 CPU and using 50m still needs 4 CPU of allocatable space. Right-size the requests; the node count will follow.

Scale-down happens, then scale-up immediately follows, repeatedly

Scale-down is too aggressive for the workload's arrival pattern. Raise scale-down-unneeded-time and scale-down-delay-after-add, and consider lowering scale-down-utilization-threshold back toward the default. Node churn costs provisioning time and image pulls even when it does not obviously cost money.

Spot nodes never get any pods

The spot taint is applied automatically; the workload needs a matching toleration. Confirm the taint with kubectl describe node <spot-node> | grep Taints and check that your Deployment tolerates kubernetes.azure.com/scalesetpriority=spot:NoSchedule. If the toleration is present and pods still will not schedule, check whether a required node affinity is pointing at a label the pool does not carry.

A spot pool exists but has zero nodes and will not scale up

That is a capacity signal, not a configuration error. Azure has no surplus of that size in that region right now. Add a second spot pool on a different VM size, or a different zone, and use preferred affinity so work falls back to on-demand rather than queueing indefinitely.

The cost analysis add-on shows no data

Confirm the cluster is on the Standard or Premium tier — the add-on is unavailable on Free — verify metricsProfile.costAnalysis.enabled is true, and allow up to a day for the first data to surface in Cost Management.

Pods start getting OOMKilled after a right-sizing pass

Memory is not compressible: exceeding a memory limit terminates the container. Raise memory requests and limits back toward the VPA upperBound rather than target for workloads with spiky allocation, and treat CPU (which throttles) and memory (which kills) as different risk categories.

---

Key Takeaways

• The AKS bill is a node bill. Control-plane tier choices matter for SLA and support, not for cost control.
• Requests drive node count. Profile the workload with VPA recommendations before you change a single node pool setting.
• Node SKU family should match the aggregate CPU-to-memory ratio of your requests; the wrong family is a permanent overpay.
• Separate system and user pools, keep the pool count low, and split by workload class rather than by team.
• Autoscaler defaults favour stability over savings. Tune scale-down-utilization-threshold, scale-down-unneeded-time, and the expander — then verify against the status ConfigMap.
• Most stalled scale-downs are eviction vetoes: restrictive PDBs, local storage, and bare pods. That is a workload fix, not an autoscaler fix.
• Spot pools carry the deepest discount and require idempotent, checkpointing workloads with short grace periods and real SIGTERM handling.
• Commitments discount the VMs beneath your pools. Size them to the autoscaler floor, exclude spot, and prefer savings plans while the fleet is still changing shape.

---

Next Steps

1. Split the bill. In Cost Management, filter to the cluster's node resource group and group by meter category. Confirm nodes really are the dominant cost before optimising them.
2. Measure packing efficiency. Run the kubectl describe nodes and kubectl top nodes pair. The gap between requested and used is your headline opportunity.
3. Turn on VPA in recommendation mode for your ten largest Deployments and collect two weeks of recommendations before changing anything.
4. Enable the AKS cost analysis add-on (or deploy OpenCost) and publish per-namespace idle cost to the teams that own those namespaces.
5. Audit eviction blockers. List every PDB in the cluster, flag any where minAvailable equals the replica count, and annotate disposable emptyDir workloads as safe to evict.
6. Pilot one spot pool with a CI runner or nightly batch job, with --spot-max-price -1, --min-count 0, and a tested SIGTERM path. Measure the real eviction rate for a fortnight before expanding.
7. Schedule stop/start for every non-production cluster, and set a calendar reminder to confirm the schedule still fires a month later.

---

Related Articles

• Reserved Instances vs Savings Plans vs Spot VMs: An Azure Cost Optimization Comparison — the commitment models that discount the VMs under your node pools
• AKS Production Hardening: A 2026 Security and Reliability Checklist — the reliability guardrails that should survive every right-sizing pass
• Azure Cost Anomaly Detection: Setting Up Budgets, Alerts, and Automated Shutdowns — catch a runaway node pool before the invoice does
• Kubernetes Ingress Controllers Compared: nginx vs Traefik vs Azure Application Gateway — the load balancer choices that shape the networking line of your bill