Articles
Kubernetes Autoscaling Interview Questions: HPA Metrics, Pending Pods, and Oscillation
Prepare for Kubernetes autoscaling interviews with HPA metric reasoning, Pending Pod diagnosis, readiness, oscillation, and safe scale-down evidence.
Kubernetes autoscaling interview questions rarely end with the HPA formula. An autoscaler can request more replicas while users continue waiting because the new Pods are Pending, failing startup, or unable to reach a saturated database. Your answer should follow the entire path from a trustworthy metric to useful serving capacity.
For each scaling decision, explain what changed in desired replicas, scheduled Pods, ready endpoints, and the user-facing result.
Evidence boundary: Kubernetes documentation supplies the controller and scheduling behavior below. The checkout workload, arithmetic, and incident timelines are original interview exercises, not measured cluster results. Candidate-reported question records provide related practice without guaranteeing the same questions or infrastructure in your interview.
Explain what HPA controls and what it does not
Official behavior: HorizontalPodAutoscaler adjusts the desired scale of a workload from observed metrics. Its basic calculation compares the current metric with the target, while handling missing measurements, readiness, bounds, and configured scaling behavior. It does not itself provision nodes or make an unhealthy application ready. Kubernetes: horizontal Pod autoscaling.
Start with an original checkout service running four replicas. Traffic increases and HPA recommends six. That recommendation is one event in a longer chain: the workload controller creates Pods, the scheduler places them, containers start, readiness succeeds, and traffic reaches the new capacity.
The first interview distinction is therefore between desired and available capacity. “HPA scaled successfully” may mean the desired replica count changed. It does not establish that six healthy instances are serving requests.
Ask which symptom the interviewer is describing. A desired count stuck at four calls for different evidence from a desired count of six with two Pending Pods. A ready count of six with unchanged latency points toward yet another investigation.
Name the denominator before interpreting CPU utilization
Official resource model: Kubernetes distinguishes resource requests from limits. Requests participate in scheduling decisions; CPU limits constrain execution through throttling. They are not interchangeable measures of allocated or consumed capacity. Kubernetes: resource management.
For a CPU utilization target, explain utilization relative to the relevant CPU request. A container using 250 millicores with a 500-millicore request has 50% utilization on that basis. Calling it “25% CPU” without naming a denominator leaves the claim ambiguous.
In the four-replica exercise, suppose valid measurements show average utilization of 75% against a 50% target. The basic calculation is ceil(4 × 75 / 50) = 6. This is the starting recommendation under simplified assumptions, not an unconditional prediction of the controller's final action.
Now keep actual CPU use unchanged and double the request. Reported request-relative utilization falls. The application has not become twice as efficient; the denominator changed. That is why a resource-request edit can alter scaling behavior even without a traffic change.
Official walkthrough: Kubernetes demonstrates configuring resource requests and an HPA target together, and inspecting HPA status while applying load. Use that relationship when explaining a configuration, rather than treating the target percentage as an isolated knob. Kubernetes: HPA walkthrough.
If utilization is unknown, verify the metric pipeline and required request configuration before assuming zero demand. Missing data and a measured zero are different states.
Trace the failed transition with a capacity ledger
This original ledger turns “autoscaling is broken” into a testable claim:
| Observed state | Likely investigation boundary | Evidence to inspect |
|---|---|---|
| Demand rises, metric remains flat or unknown | Signal selection or collection | Metric definition, timestamp, adapter health, requests |
| Metric warrants growth, desired replicas stay fixed | HPA recommendation or constraints | Status conditions, maximum replicas, scaling behavior |
| Desired count grows, new Pods remain Pending | Scheduling or capacity | Pod events, requests, affinity, taints, volume constraints |
| Pods run but stay unready | Application startup or dependencies | Probe results, startup logs, downstream reachability |
| Ready count grows, throughput does not improve | Bottleneck or traffic distribution | Per-Pod load, queues, database limits, connection reuse |
| Load falls and rises with repeated replica churn | Feedback timing or competing changes | Metric, recommendation, readiness, and rollout timelines |
The ledger does not assign a root cause from a single status field. It identifies where to collect the next discriminating observation.
For example, a Pending Pod with a volume-zone constraint will not necessarily become schedulable after adding arbitrary nodes. A running Pod whose readiness check fails because a dependency is unavailable will not become useful just because HPA creates ten more copies.
Record the timing of each transition. A system that eventually reaches six ready replicas after several minutes may be functioning as configured while still reacting too slowly for the application's latency objective.
Pending Pods are a scheduling question
Official guidance: Kubernetes recommends examining Pod descriptions and events when diagnosing Pending Pods, including whether resources and scheduling constraints prevent placement. Kubernetes: debug Pods.
For the checkout exercise, inspect the exact new Pods rather than only aggregate node CPU. Nodes can look idle while the scheduler cannot satisfy declared requests, a required zone, an accelerator requirement, or a placement rule. Current utilization does not fully describe schedulability.
Official distinction: node autoscaling changes the available node capacity and must consider Pod scheduling requirements and infrastructure constraints. HPA and node autoscaling operate at different layers. Kubernetes: node autoscaling.
A useful follow-up asks whether a suitable node can actually be provisioned. The node pool may have reached a configured maximum, cloud capacity may be unavailable, or no offered node shape may satisfy the Pod. Raising the workload's maximum replicas does not resolve those constraints.
Avoid proposing a lower CPU request solely to force placement. That might change both scheduling and the HPA signal while leaving the workload underprovisioned. If requests are wrong, correct them using observed application needs and then reassess the scaling target.
Keep the distinction between admission and scheduling as well. A quota can prevent additional Pods from being created at all; that is not the same as existing Pods waiting for placement. Inspect controller events alongside Pod events when expected new objects are missing.
More replicas help only when they relieve the bottleneck
A checkout worker blocked on database connections may consume little CPU while requests accumulate. A CPU-based signal can therefore underrepresent demand. Conversely, adding workers can increase pressure on the database and make latency worse.
Before selecting another metric, define what it represents. Queue depth, oldest-item age, request rate, and in-flight work describe different aspects of demand. Explain whether the metric is a total workload value or a per-Pod value, and how changing replica count should affect it.
An original queue example makes the distinction clear: 1,000 short jobs and 1,000 long jobs have equal depth but very different processing demand. Oldest-item age can reveal delay, while processing rate helps estimate whether additional workers are draining the backlog.
Do not claim a queue metric automatically solves scaling. It still needs reliable collection, a meaningful target, a workload that benefits from more workers, and downstream capacity. A stale backlog metric can keep replicas high after work has drained or delay growth during a new surge.
For multiple metrics, avoid averaging unrelated units. Each signal has its own scaling interpretation. Describe how the configured controller combines recommendations and what happens when one metric source is unavailable before making a prediction about scale-down.
Diagnose oscillation with a timeline
Consider an original repeating pattern: traffic rises, replicas increase, average load per Pod falls, replicas are removed, and the remaining Pods become overloaded. That sequence suggests a feedback loop, but it does not identify the only cause. Bursty traffic, delayed metrics, startup overhead, and competing configuration changes can produce similar graphs.
Align four series on one timeline: observed demand, HPA recommendations, ready replicas, and user latency. Include deployment events and resource-request changes. If recommendations are stable but ready replicas oscillate, investigate restarts or readiness rather than tuning HPA first.
Official controls: HPA supports separate scale-up and scale-down behavior. Stabilization can retain recent higher scale recommendations during downscaling, and rate policies constrain replica changes. These controls address different aspects of responsiveness. Kubernetes: configurable HPA behavior.
For the exercise, a downscale stabilization window can prevent immediately removing capacity after a brief quiet interval. A removal-rate limit can make a sustained scale-down more gradual. Neither creates capacity faster when node provisioning and application startup dominate recovery time.
Explain the trade-off: extra headroom costs resources, while aggressive removal risks another overload and cold start. Choose values based on the workload's burst pattern and recovery time, not because a copied manifest uses a familiar number.
Also check for competing writers. A deployment pipeline repeatedly applying a fixed replica count can interfere with the intended scaling workflow. Establish which component owns scale and avoid interpreting configuration churn as purely a metrics problem.
Readiness and termination complete the scaling story
Official probe behavior: readiness indicates whether a container is ready to accept traffic, while startup and liveness probes serve different purposes. Misusing these checks can remove healthy capacity or expose an instance before it is useful. Kubernetes: probes.
For checkout, a ready instance should satisfy the application's serving contract. A process that has opened a port but is still warming essential state may not yet provide useful capacity. On the other hand, tying readiness to a fragile optional dependency can remove too many endpoints simultaneously.
Scale-down also needs a business-level check. If an instance is processing a payment or holding a long-lived connection, removal must be coordinated with draining and the application's retry or idempotency design. A smaller replica count is not evidence that in-flight work completed safely.
Official lifecycle mechanism: Kubernetes provides termination hooks within the Pod termination process. Hook execution and graceful shutdown need to fit the configured termination budget. Kubernetes: container lifecycle hooks.
In an interview, propose a controlled test that increases demand, observes the full path to ready capacity, removes demand, and checks in-flight outcomes during downscale. Capture latency, failures, queue age, and completed work alongside replica counts.
Report the outcome with its limits
A strong original incident summary is: “HPA recommended six replicas, but two new Pods could not fit the constrained node pool. After suitable capacity became available, they passed readiness and checkout latency recovered. We will measure provisioning delay and retain enough baseline capacity for the expected burst.”
That statement separates observed recovery from a prevention proposal. It does not claim HPA alone fixed the incident or that every future burst is covered.
For verification, repeat the same workload and configuration where possible. If you change the application, requests, target metric, and node pool together, record each change and avoid assigning the improvement to one knob without additional evidence.