How does the Horizontal Pod Autoscaler work, and how does it relate to the Cluster Autoscaler?

Question

IT Interview Practice · Accepted Answer

The **Horizontal Pod Autoscaler (HPA)** adds or removes **Pod replicas** based on observed metrics (CPU, memory, or custom). The **Cluster Autoscaler (CA)** adds or removes **nodes** when Pods can't be scheduled or nodes are underused. They operate at different layers and work together.

## Two autoscalers, two layers

```text
HPA  → scales PODS  (more/fewer replicas of a Deployment)     — reacts to LOAD
CA   → scales NODES (grows/shrinks the cluster's machines)     — reacts to PENDING Pods
VPA  → (bonus) adjusts a Pod's requests/limits                 — reacts to right-sizing

Typical flow: load rises → HPA adds Pods → Pods can't fit → CA adds a node →
              Pods schedule. Load drops → HPA removes Pods → node idle → CA removes it.
```

## How the HPA control loop works

```text
Every ~15s the HPA controller:
  1. reads current metric (e.g. avg CPU across Pods) from metrics-server / custom API
  2. computes:  desiredReplicas = ceil(currentReplicas * currentMetric / targetMetric)
  3. scales the Deployment toward that (bounded by minReplicas/maxReplicas)
Example: 4 Pods at 80% CPU, target 50% → 4 * 80/50 = 6.4 → 7 replicas.
Stabilization windows dampen flapping (esp. on scale-down).
```

## An HPA manifest

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60      # keep avg CPU ~60% of the REQUEST
  behavior:                           # optional: tune scale up/down speed
    scaleDown:
      stabilizationWindowSeconds: 300
```

```bash
kubectl get hpa                       # current/target metric, replica count
kubectl top pods                      # requires metrics-server
```

## Important dependencies & caveats

```text
• HPA on CPU/memory REQUIRES metrics-server installed; utilization is relative to REQUESTS
  → if requests are unset/wrong, HPA math is meaningless. Set requests correctly.
• Custom/external metrics (RPS, queue depth) need an adapter (Prometheus Adapter, KEDA).
• HPA and manual `replicas` conflict — let the HPA own the replica count.
• Cluster Autoscaler needs correct requests too: it looks at Pending Pods' REQUESTS to
  decide node scaling. Node scale-up takes minutes (provisioning), so keep headroom.
```

## Why it matters

Autoscaling is what lets Kubernetes match capacity to demand automatically, and understanding the two layers is senior-level knowledge that separates cost-efficient, resilient clusters from over- or under-provisioned ones. The key mental model is that **HPA scales Pods** in response to load (adjusting replica count from observed CPU, memory, or custom metrics) while the **Cluster Autoscaler scales nodes** in response to scheduling pressure (adding machines when Pods are Pending, removing them when nodes are idle) — they operate at different layers and cooperate: HPA can only add Pods if there is room, and the CA provides that room. Understanding the HPA's control loop — the `desiredReplicas = ceil(currentReplicas * currentMetric / targetMetric)` formula, the periodic reconciliation, and stabilization windows that prevent flapping — lets you configure and debug it rather than treat it as magic. The caveats are where senior judgment shows: HPA on CPU/memory **requires metrics-server** and computes utilization **relative to requests**, so incorrect or missing requests make its math meaningless; custom metrics (requests-per-second, queue depth) need an adapter like Prometheus Adapter or KEDA; and node scale-up takes minutes, so headroom matters. Because right-sizing capacity directly affects reliability under traffic spikes and cloud spend during quiet periods, and because both autoscalers depend on correctly set resource requests, mastering HPA and the Cluster Autoscaler is fundamental to operating production Kubernetes efficiently and reliably.

Two autoscalers, two layers

How the HPA control loop works

An HPA manifest

Important dependencies & caveats

Why it matters

Related questions