Kubernetes Cost Management: Taming the Cluster Bill

Why the cluster bill grows faster than the cluster

Kubernetes is a scheduler, and it schedules against what you ask for, not what you use. That single fact explains most runaway cluster bills. A team sets a pod's CPU request to 1 core "to be safe," the container actually uses 80 millicores, and the scheduler still reserves the full core for nobody. Multiply that across hundreds of pods and dozens of deployments and you pay for two or three times the compute you consume — while dashboards show idle CPU, making the waste feel like a paradox instead of a line item.

The cloud provider does not help here. It bills you for the nodes in your cluster; whether those nodes run at 20 or 80 percent utilization is a function of how well your workloads are sized and packed. Kubernetes cost management is therefore not a pricing negotiation but an engineering discipline that closes the gap between requested, used, and paid-for capacity. This is the practice we run for clients on cloud infrastructure, and it reliably takes 30 to 50 percent off a neglected cluster without touching application code.

The three multipliers that inflate the bill

Nearly every over-sized cluster is some combination of three compounding problems that multiply — which is why the total waste is usually worse than any one of them looks in isolation.

• Over-requested resources. Requests set far above real usage. This is the root cause, because requests are what the scheduler reserves and what effectively determines how many nodes you need.
• Low bin-packing. Even with reasonable requests, pods scheduled across too many nodes leave every node half-empty. The cluster autoscaler will not remove a node while a single pod sits on it, so fragmentation quietly pins capacity.
• Idle nodes. Nodes kept warm for a peak that rarely comes, node pools that never scale down, and non-production clusters running 24/7 for a workload used 40 hours a week.

Figure: the bar you pay for is requests, not usage — the gap between the two is the first place to look for savings.

The order matters. Fix requests first, because right-sized requests make bin-packing tractable and let the autoscaler actually remove nodes. Chase idle nodes before fixing requests and they simply reappear next week.

Right-size requests and limits

Requests and limits do different jobs, and conflating them is the most common mistake. The request is a reservation used for scheduling — it is the number that costs you money. The limit is a ceiling the container is not allowed to cross. Set them deliberately:

1. Measure real usage over a representative window. Pull at least two weeks of per-container CPU and memory, including your real peak — a month-end batch, a morning login surge, a marketing spike. Averages hide the spikes that matter, so work from the distribution.
2. Set CPU requests near the p95 of real usage. CPU is compressible; a container that briefly wants more just gets throttled, it does not die. Sizing the request around the 95th percentile of observed usage recovers the reserved-but-idle headroom while still covering normal peaks.
3. Set memory requests and limits close together, near peak. Memory is not compressible. A container that exceeds its memory limit is OOM-killed, not slowed. Size memory to the working-set peak plus a modest margin, and keep the request and limit near each other so scheduling reflects reality.
4. Be careful with CPU limits. A tight CPU limit throttles latency-sensitive services even when the node has spare capacity. Many teams get more predictable performance by setting CPU requests accurately and leaving CPU limits generous or unset, while keeping memory limits firm.

The payoff is direct: lowering an over-stated request frees reservation on the node, which lets more pods pack onto fewer nodes, which lets the cluster autoscaler delete the emptied nodes. One well-tuned request cascades into fewer billed instances.

Autoscale at both layers

Kubernetes has three autoscalers, and they solve different problems. Cost management uses all three together.

• Horizontal Pod Autoscaler (HPA) adds and removes pod replicas based on a signal. Use it for stateless web and API tiers, and scale on a leading indicator such as request rate or queue depth where you can — CPU lags real load and scales up only after users feel the slowdown. Set a replica floor that survives a node failure.
• Vertical Pod Autoscaler (VPA) adjusts a pod's requests toward observed usage. In recommendation mode it is a low-risk source of right-sizing data; in auto mode it enforces sane requests continuously. Do not run VPA and HPA on the same CPU metric — they fight each other.
• Cluster Autoscaler / node auto-provisioning adds and removes nodes so pending pods can schedule and empty nodes go away. Modern consolidating provisioners go further, actively repacking pods onto fewer, cheaper nodes and terminating the rest — attacking bin-packing directly.

The combination is the goal. VPA (or a rightsizing tool) keeps requests honest, HPA matches replica count to demand, and the cluster autoscaler collapses the node count to fit. Any one alone leaves money on the table.

Node strategy: spot, families, and packing

Once workloads are sized and packed, the nodes underneath them are the next lever.

• Run interruption-tolerant work on spot. Batch jobs, CI runners, stateless web tiers, and dev environments are ideal for spot or preemptible nodes, which discount 60 to 90 percent off on-demand rates. Use taints and tolerations so only fault-tolerant workloads land there, keep stateful services on on-demand, and spread across instance types so one capacity reclaim does not drain a tier.
• Pick the right instance families and generations. Match node shape to workload shape — compute-optimized for CPU-bound services, memory-optimized for caches and in-memory stores — and move to current-generation or ARM-based instances, which usually deliver better price-performance for a like-for-like migration once you validate your images build and run on the architecture.
• Prefer fewer, larger nodes for packing, within reason. Larger nodes pack more efficiently and dilute per-node overhead, but oversized nodes make scale-down coarse and enlarge the blast radius of a node failure. Size the pool so a typical pod is a small fraction of a node.
• Commit only the steady baseline. Cover the always-on floor of the cluster with savings plans or committed-use discounts, and keep the variable top layer on-demand and spot. Never commit before right-sizing, or you lock in the waste.

See cost by namespace and team

You cannot govern what nobody owns. The reason clusters re-inflate a quarter after every cleanup is that the people creating cost never see it. Close that loop with allocation:

• Attribute spend to namespaces, labels, and teams. Split node cost across the pods on each node by their requests, then roll it up by namespace, workload, and a mandatory owner label. Tools such as OpenCost (the CNCF project) turn raw node bills into per-team showback.
• Report requested versus used per namespace. The efficiency ratio — used divided by requested — is the single most actionable number in the cluster. A namespace running at 20 percent efficiency is your next win; one at 70 percent is already healthy.
• Alert on anomalies, not just month-end. A cost spike caught in 48 hours is a quick fix; found six weeks later it is a write-off.

This visibility rides on the same telemetry as everything else, which is why continuous infrastructure monitoring is a prerequisite for cost work — it supplies the usage percentiles and the before-and-after baseline that prove a change was safe.

Make it continuous, and govern it

The largest mistake is treating cluster right-sizing as a one-time cleanup. Deployments ship, traffic shifts, and a workload you tuned in March is wrong by September. Build a standing loop: measure usage, recommend request changes against explicit headroom targets, apply them in lower environments first, verify latency and error rates held, and repeat monthly. Enforce it with admission policy so pods without sensible requests, limits, or an owner label are flagged at creation.

Governance is also where cost and security converge. The same weak visibility that hides idle nodes hides exposed workloads, over-broad service accounts, and misconfigured network policy — which is why we pair cluster cost work with cloud security posture management, governing the estate for spend and exposure at once.

Bring the bill back to reality

A well-run Kubernetes cluster should bill for roughly what it uses. Getting there is not exotic: right-size requests to real usage, let the autoscalers do their jobs, push tolerant work onto spot, pack onto fewer nodes, and give every team a clear view of what it spends. Run as a continuous loop rather than a quarterly fire drill, it routinely cuts a neglected cluster bill by a third to a half with no user-visible impact.

If your cluster costs have outrun the workloads inside it, talk to our cloud team. We will baseline your requests-versus-usage gap, model the right-sizing and node moves, and prove the headroom before anything changes.