Kubernetes Autoscaling Explained: HPA, VPA, and Cluster Autoscaler

Three different autoscalers, three different axes of scale, and one very common mistake: assuming they solve the same problem.

Kubernetes autoscaling is one of those topics where the vocabulary is deceptively simple and the actual behavior is deceptively complex. Teams adopt “autoscaling” as a checkbox — turn it on, feel safe — without understanding that Kubernetes actually ships three distinct autoscalers, each operating on a different axis, each with its own failure modes, and each capable of fighting the other two if configured carelessly.

Understanding the difference is not academic. Misconfigured autoscaling is one of the most common root causes of both “why did we get paged at 3 AM” incidents and “why is our cloud bill twice what it should be” finance conversations.

The Three Axes of Scale

• Horizontal Pod Autoscaler (HPA): scales the number of pod replicas for a workload, based on observed metrics like CPU utilization, memory, or custom metrics.
• Vertical Pod Autoscaler (VPA): scales the resource requests and limits (CPU and memory) assigned to each pod, adjusting how big each individual pod is rather than how many there are.
• Cluster Autoscaler (CA): scales the number of nodes in the underlying cluster, adding capacity when pods can’t be scheduled due to insufficient resources, and removing nodes when they sit underutilized.

These operate at genuinely different layers. HPA and VPA both live at the pod layer but conflict with each other if you’re not careful. Cluster Autoscaler lives one layer below both of them, reacting to whatever HPA and VPA have already decided. Getting all three to cooperate rather than fight is the actual skill here.

Horizontal Pod Autoscaler: The Workhorse

HPA is the autoscaler most teams reach for first, and for good reason — it directly addresses the most common scaling need: “we’re getting more traffic, spin up more copies of this stateless service to handle it.”

HPA works by polling a metrics source (typically the Kubernetes Metrics Server, or a custom/external metrics adapter for things like queue depth or requests-per-second) on a regular interval, comparing the observed value against a target, and calculating a desired replica count using a straightforward ratio:

desiredReplicas = ceil(currentReplicas * (currentMetricValue / desiredMetricValue))

If you’ve set a target of 50% CPU utilization and your pods are currently averaging 80%, HPA will scale you up by roughly 60%. This ratio-based approach means HPA reacts proportionally rather than jumping straight to some arbitrary number, which keeps scaling behavior smooth and predictable.

The metric choice matters more than the threshold. CPU-based scaling is the default and the easiest to set up, but it’s a poor proxy for load in many real-world services. A service that’s I/O-bound (waiting on database calls, waiting on a downstream API) can be under severe load — long queues, growing latency, unhappy users — while its CPU utilization stays low, because the pod is mostly idle, waiting. For these workloads, custom metrics like requests-per-second, queue depth, or even a direct latency SLI are far more honest signals of real load than CPU ever will be.

Scaling down is where most incidents happen, not scaling up. HPA’s default behavior includes a stabilization window specifically to prevent “flapping” — rapidly scaling up and down in response to noisy metrics. Without this window (or with it set too aggressively short), a brief dip in traffic can trigger a scale-down, immediately followed by a traffic spike that requires scaling back up — except now you’re paying the cold-start cost of new pods (image pull, application startup, JVM warmup, connection pool initialization) exactly when you can least afford the latency. The fix is almost always a longer scale-down stabilization window than scale-up, since the cost of scaling down too eagerly is much higher than the cost of scaling up a beat too early.

Vertical Pod Autoscaler: The Underused Sibling

VPA solves a different problem entirely: “I don’t know how much CPU and memory this pod actually needs, and I’m afraid of setting the request too low (causing throttling and OOM kills) or too high (wasting money on unused reserved capacity).”

VPA watches actual resource usage over time and recommends — or, in “Auto” mode, directly applies — updated resource requests. This matters more than it sounds like it should, because Kubernetes resource requests are simultaneously a scheduling hint, a billing unit, and a throttling threshold. Set a CPU request too low and the pod gets scheduled onto a busy node and throttled hard the moment it needs more than it asked for. Set memory too low and the pod gets OOM-killed the moment usage briefly spikes above the limit. Set either too high “just to be safe,” and you’re paying for reserved capacity that mostly sits idle, and you’re wasting schedulable space on every node in the cluster.

The uncomfortable operational detail: applying a new resource request to a running pod currently requires restarting the pod in most VPA implementations (in-place resource resize is landing in newer Kubernetes versions but is not yet universally available). This means VPA in “Auto” mode will periodically evict and recreate your pods to apply new sizing — fine for a fleet of ten interchangeable stateless replicas, potentially disruptive for a singleton or a stateful workload with a slow, expensive startup sequence.

VPA and HPA conflict when configured on the same metric. If HPA is scaling replica count based on CPU utilization, and VPA is simultaneously adjusting the CPU request of each replica, the two systems are effectively fighting over the same signal — VPA changes the denominator that HPA’s percentage is calculated against, and HPA’s replica count changes the aggregate load VPA sees per pod. The standard mitigation is to run VPA in “recommendation-only” mode alongside CPU/memory-based HPA (letting a human review and apply VPA’s suggestions periodically), or to use VPA in Auto mode only on workloads where HPA scales on a completely independent, unrelated metric like queue depth.

Cluster Autoscaler: The Layer Beneath

Cluster Autoscaler doesn’t care about individual pod metrics at all. It watches for one specific condition: pods that are unschedulable because no node in the cluster has enough free capacity to fit them. When it sees this, it adds nodes (usually by increasing the desired size of a cloud provider’s node group or instance group). When it sees nodes that are significantly underutilized and whose pods could be safely rescheduled elsewhere, it drains and removes them.

This creates an important dependency chain that’s easy to overlook: HPA scaling up doesn’t guarantee your pods actually run — it just creates more pod specs for the scheduler to place. If the cluster has no spare node capacity, those new pods sit in a Pending state until Cluster Autoscaler notices, provisions a new node, and waits for that node to become Ready — a process that, depending on your cloud provider and instance type, can take anywhere from 30 seconds to several minutes. If your traffic spike happens faster than that provisioning time, your HPA-driven scale-up is, in effect, running behind the actual demand curve, and users feel the gap as elevated latency or errors during exactly the window autoscaling was supposed to protect against.

The standard mitigation is maintaining some amount of buffer capacity — either a small number of always-on “slack” nodes, or low-priority placeholder pods (sometimes called “pause pods”) that reserve node capacity and get preempted the instant a real workload needs to schedule. This trades a small, predictable, always-on cost for eliminating the multi-minute cold-start gap during genuine traffic spikes.

Node Pool Segmentation Reduces Provisioning Latency

A practical technique that pairs well with Cluster Autoscaler’s buffer capacity approach is segmenting node pools by workload type rather than running every pod on a single, homogeneous pool. Latency-sensitive, customer-facing workloads can live on a pool with a small standing buffer and faster-provisioning instance types, while batch and background jobs live on a separate, more elastic pool that’s allowed to scale from a much smaller baseline, since a few minutes of provisioning delay for a batch job rarely matters the way it does for a checkout request.

This segmentation also limits blast radius: a runaway workload that suddenly demands far more resources than expected consumes capacity from its own pool rather than starving node availability for unrelated, more critical services sharing the same undifferentiated pool. The cost of this approach is added operational complexity, since you’re now managing scaling policies, buffer sizes, and node taints across multiple pools instead of one, but for any cluster running a meaningful mix of latency-sensitive and best-effort workloads, that complexity usually pays for itself the first time a background job’s demand spike would otherwise have starved a customer-facing service of schedulable capacity.

Pod Disruption Budgets: The Safety Net Everyone Forgets

Cluster Autoscaler’s node removal, and general cluster maintenance operations, work by draining nodes — evicting all pods running on them so they can be rescheduled elsewhere before the node is terminated. Without a Pod Disruption Budget (PDB), there’s nothing stopping the autoscaler (or a cluster upgrade, or a manual node drain) from evicting every single replica of a service simultaneously, causing a brief but complete outage of that service even though the overall cluster capacity was fine.

A PDB expresses a constraint like “at least 2 replicas of this deployment must remain available during any voluntary disruption,” and the eviction API respects it, spacing out evictions so the constraint holds. Every production workload with more than one replica should have a PDB. It costs nothing to define and it is the single most common gap between “we configured autoscaling” and “our service survived the next cluster upgrade without a blip.”

KEDA: Event-Driven Autoscaling for the Cases HPA Wasn’t Built For

Standard HPA works well for request-driven workloads, where load correlates reasonably with CPU, memory, or a request-rate metric scraped continuously from a running pod. It’s a poor fit for event-driven workloads, such as a queue consumer that should scale based on how many messages are waiting, including scaling to and from zero when the queue is empty, which vanilla HPA cannot do at all since it requires at least one running pod to measure a metric from in the first place.

KEDA (Kubernetes Event-Driven Autoscaling) fills this gap by acting as a metrics adapter that can scale based on external event sources, such as queue depth in Kafka, RabbitMQ, or SQS, the number of pending jobs in a database table, or nearly any custom metric exposed by an external system, and, critically, by being able to scale a deployment down to zero replicas when there’s genuinely no work to do, then scale back up the instant new events arrive. This makes it possible to run genuinely bursty, intermittent workloads, like nightly batch processing or sporadic webhook consumers, without paying for idle replicas sitting around waiting for the next burst.

The operational tradeoff is the same cold-start latency problem discussed earlier, but more pronounced: scaling from zero means the very first event in a new burst has to wait for a pod to schedule, pull its image, and initialize before it’s processed, which is a real cost for latency-sensitive queues even though it’s usually an acceptable cost for the batch and asynchronous workloads KEDA is most often applied to.

Making the Three Work Together

A coherent autoscaling strategy for a typical production workload looks roughly like this:

1. HPA scales replica count based on the metric that best represents actual load for that specific workload — CPU for compute-bound services, custom metrics (queue depth, RPS, latency) for I/O-bound ones.
2. VPA, in recommendation-only mode, runs continuously in the background, and its suggestions get reviewed and applied during planned maintenance windows rather than live, avoiding unplanned pod restarts and avoiding direct conflict with HPA’s CPU-based decisions.
3. Cluster Autoscaler handles node-level capacity, backed by a small buffer of reserved or low-priority placeholder capacity to absorb the provisioning lag during sudden spikes.
4. Pod Disruption Budgets are defined for every multi-replica workload, so that Cluster Autoscaler’s node draining, and any other voluntary disruption, never take down more capacity than the service can tolerate losing at once.

None of these four pieces is optional if you actually want autoscaling to behave the way people assume it does by default. Turning on HPA alone gives you exactly one axis of the picture — and in a cluster with no spare node capacity and no PDBs, it can create the appearance of resilience while leaving the two most common causes of scaling-related incidents completely unaddressed.