Kubernetes Autoscaling in Practice: HPA, VPA, and Cluster Autoscaler

Three scaling loops, not one feature

Kubernetes does not have an autoscaler. It has three, they act on different objects at different speeds, and most autoscaling incidents come from treating them as one thing — or letting two of them fight over the same workload. Before tuning anything, hold the mental model clearly:

• Horizontal Pod Autoscaler (HPA) changes the number of pod replicas for a workload, reacting to observed load in seconds to minutes.
• Vertical Pod Autoscaler (VPA) changes the CPU and memory requests and limits on each pod, right-sizing a single replica rather than adding more.
• Cluster Autoscaler (or Karpenter) changes the number of nodes so there is somewhere for new pods to land, reacting to unschedulable pods.

HPA scales the app, VPA sizes the app, and the node autoscaler makes room for the app. All three depend on one thing being correct: accurate resource requests. Get requests wrong and every loop above them makes bad decisions with confidence.

HPA: scaling out the replicas

The Horizontal Pod Autoscaler is the workhorse and the one most teams reach for first. It watches a metric, compares it to a target, and adjusts replica count to close the gap. The default metric is CPU utilization as a percentage of the pod's CPU request — which is exactly why requests matter. If a pod requests 250m and HPA targets 60% CPU, it scales when average usage crosses 150m, not when the node is busy.

Key practicalities:

• You need a metrics source. CPU and memory targets require metrics-server running in the cluster. Without it, kubectl get hpa shows <unknown> and the controller does nothing.
• CPU is a poor proxy for many workloads. Queue depth, requests per second, or p95 latency often describe load far better. Custom and external metrics let HPA scale on those, and KEDA extends this to event sources like queue length, letting a consumer scale — including to zero — off the backlog it is actually draining.
• Tune the behavior, not just the target. Aggressive scale-up with slow, stabilized scale-down is usually right: add capacity fast when load arrives, remove it cautiously so a brief dip does not trigger thrashing.

HPA is the correct default for stateless, horizontally scalable services behind a load balancer. It does nothing useful for a workload that cannot run more than one copy.

VPA: right-sizing each pod

The Vertical Pod Autoscaler solves a different problem: not how many replicas, but how big each one should be. It observes actual consumption over time and recommends — or automatically applies — better requests and limits. This is the antidote to the two most common cluster-economics failures: every pod requesting far more than it uses (wasted, unschedulable capacity) or requesting too little (OOMKilled pods and CPU throttling).

Two things to understand before enabling it:

• Applying a change means restarting the pod. Kubernetes cannot resize a running pod's requests in place in most setups, so VPA in Auto mode evicts and recreates pods to apply new sizing. Respect a PodDisruptionBudget and use it only where restarts are safe.
• VPA and HPA conflict on the same metric. If both act on CPU or memory for one workload, they contradict each other — HPA adds replicas to lower per-pod CPU while VPA shrinks per-pod CPU requests, and they oscillate. Run them together only when HPA scales on a custom metric (like requests per second) and VPA handles memory. Many teams instead run VPA in recommendation-only mode and feed its numbers into their manifests.

Cluster Autoscaler and Karpenter: scaling the nodes

Pods only run if a node has room. When HPA or a deployment creates pods that cannot be scheduled, the node autoscaler adds capacity; when nodes sit underutilized, it removes them. Two approaches dominate:

• Cluster Autoscaler works through node groups (managed instance groups / node pools). You predefine the instance types, and it grows and shrinks those groups to fit pending pods. Predictable, but you own the shape of the groups.
• Karpenter provisions nodes just-in-time from a broad set of instance types, picking the cheapest shape that fits the pending pods and consolidating workloads onto fewer nodes as demand falls. It reacts faster and bin-packs better, at the cost of a newer, more dynamic model to reason about.

Whichever you use, the interaction with your compute strategy matters. Diversify across instance types so the autoscaler is not stranded when one t3.large pool runs dry, and place scaling-hungry workloads on infrastructure sized for them — often the dense high-performance servers that give the scheduler the most room to pack pods.

Making the three loops cooperate

The loops are independent controllers with no shared brain, so cooperation is something you design, not something you get for free.

• Set honest requests first. Every loop keys off them. Measure real usage, then set requests near the steady-state working set — not the peak, not a round guess. This single input does more for autoscaling quality than any tuning parameter.
• Give the node autoscaler slack. If pods scale up but nodes take minutes to join, users feel the gap. Over-provisioning with low-priority "pause" pods reserves warm headroom that real workloads preempt instantly, trading a little cost for fast scale-out.
• Protect availability during scale-down. A PodDisruptionBudget stops the cluster autoscaler and VPA from draining so many replicas at once that the service dips below its floor.
• Do not double-scale one signal. Pick one loop per metric per workload. Overlapping controllers on the same signal is the root cause of most autoscaling oscillation.

Common failure modes

• Missing metrics-server. HPA silently does nothing. Check kubectl get hpa for <unknown> targets.
• No limits, no requests. Pods with unset requests are invisible to the scheduler's math; the autoscalers cannot reason about capacity they cannot see.
• Scaling on CPU when the bottleneck is elsewhere. A memory-bound or I/O-bound service will OOMKill while CPU sits low and HPA stays idle. Scale on the metric that actually saturates.
• Nodes that cannot shrink. A single pod without a disruption budget, or local storage, can pin an expensive node open indefinitely. Audit what blocks consolidation.
• Cold-start stampedes. Scaling from too few replicas means new pods take traffic before they are warm. Set sensible minimums and readiness probes.

Where to start

Begin with observability, not automation. Turn on metrics-server, measure a week of real CPU and memory per workload, and correct your requests — most clusters are badly mis-sized, and fixing that alone recovers meaningful capacity. Then add HPA to stateless services on a metric that reflects real load, enable the node autoscaler with a little warm headroom, and use VPA in recommendation mode to keep sizing honest over time. Treat the three loops as one system with one shared input, and validate behavior against a load test before production traffic does it for you.

Autoscaling that works quietly is the product of deliberate design across scheduling, cloud infrastructure, and the metrics that drive it. If you want a review of how your clusters scale under real load, or a Kubernetes platform built to right-size itself, talk to our team.