Articles
Kubernetes Scheduling Interview Questions: Affinity, Taints, and Topology Spread
Practice Kubernetes scheduling interviews with a Pending Pod, node affinity, taints, topology spread, and a worked placement calculation.
A Pod is Pending even though several nodes show free CPU and memory. The YAML includes required node affinity, a topology spread constraint, and the node pool has taints. Which rule is blocking placement? A strong Kubernetes scheduling interview answer treats these as separate filters and preferences, then checks the eligible node set. It does not conclude that the scheduler is broken because aggregate cluster capacity is available.
The platform claims in this guide come from Kubernetes documentation. The node inventory and manifests are original interview exercises, not reported questions from an employer. For broader cluster reasoning, Compare cloud compute; design Kubernetes security and CI/CD is a related practice prompt.
The three controls are not interchangeable
Node affinity expresses which node labels a Pod requires or prefers. requiredDuringSchedulingIgnoredDuringExecution is a hard placement requirement at scheduling time; preferredDuringSchedulingIgnoredDuringExecution affects scoring but does not make other nodes ineligible by itself. “Ignored during execution” means changing a matching node label after placement does not automatically evict the Pod. The assign Pods to nodes guide documents the syntax and the AND/OR behavior of terms.
Taints and tolerations operate from the node side. A NoSchedule taint prevents a Pod without a matching toleration from being newly scheduled there. A toleration permits consideration of that node; it does not attract the Pod or guarantee placement. NoExecute can also evict existing Pods without appropriate tolerations, while PreferNoSchedule is a soft preference. See the taints and tolerations guide.
Topology spread constraints control how Pods matching a selector are distributed across topology domains such as zones or hostnames. maxSkew and whenUnsatisfiable determine whether imbalance is a hard scheduling obstacle (DoNotSchedule) or a preference (ScheduleAnyway). Eligibility policies and available topology labels matter; spread is not a synonym for “one Pod per zone.” The official topology spread guide defines the accounting.
These controls combine. A Pod must first be eligible for a node under hard affinity and taints, and it must satisfy any hard topology spread constraint in the relevant eligible domains. A preference cannot rescue a violated hard constraint. This ordering is a useful mental model; the scheduler's implementation has multiple plugins and phases, so avoid pretending one displayed event reveals a unique execution order.
Interview question 1: What would you inspect for a Pending Pod?
Start with kubectl describe pod <name> -n <namespace> and read the FailedScheduling events. Check the exact admitted Pod spec, not only the original manifest: admission can add tolerations or scheduling fields. List node labels, taints, allocatable and requested resources, and zone/hostname labels. Confirm whether the Pod uses a nodeSelector, node affinity, pod affinity or anti-affinity, a custom scheduler, topology spread, volume node affinity, or another constraint. The event can summarize several simultaneous blockers; do not stop after the first phrase.
Narrow the candidate set manually. Which nodes match required labels? Of those, which are tolerated? Which have sufficient requested-resource headroom and compatible volumes? Of those, which would keep the topology spread rule within its allowed skew? This intersection gives you a falsifiable explanation. If no node remains, modifying an unrelated CPU limit will not help.
Also inspect existing Pods counted by a topology spread selector. A selector that matches the wrong workload or misses the intended Pods changes the distribution calculation. If a zone label is absent on eligible nodes, do not count that node as a valid domain for a zone-based constraint. Check documentation for the precise policy defaults used by the cluster version rather than relying on a diagram alone.
A concrete scheduling exercise
Assume three zones: A, B, and C. Each has two nodes and enough spare CPU for one new API Pod. The new Pod requires label workload=latency, which only the nodes in A and B have. Both A nodes carry taint dedicated=api:NoSchedule; B has no taint. Existing API Pods matched by the spread selector are distributed A:2, B:2, C:0. The new Pod has no toleration and uses zone topology spread with maxSkew: 1 and whenUnsatisfiable: DoNotSchedule.
The key is not just to compute “2, 2, 0.” First determine which topology domains are eligible under the constraint's node-affinity and taint inclusion rules in the relevant Kubernetes version. With the default policy described in current documentation, node affinity is honored and taints are ignored for inclusion in domain accounting unless configured otherwise. C is excluded by required affinity; A remains a domain for skew accounting even though this Pod cannot land on an A node without toleration. A placement in B may raise B to three against A's two, which is within maxSkew: 1; it may therefore be feasible. The point is to show the calculation under declared policies, not to assume all three physical zones are counted.
| Check | A | B | C |
|---|---|---|---|
Required workload=latency label | Yes | Yes | No |
dedicated=api:NoSchedule tolerated? | No | Not needed | Not applicable |
| Existing matching API Pods | 2 | 2 | 0 |
| Candidate placement before other constraints | Blocked by taint | Possible | Blocked by affinity |
This is an original reasoning exercise. A real result depends on the full admitted spec, the cluster's defaults, matching Pods, resource fit, and any minDomains setting. If the interviewer supplies a different nodeAffinityPolicy or nodeTaintsPolicy, redo the domain calculation. The best answer states the assumption before giving a binary yes/no.
Interview question 2: Can a toleration target a node pool?
Not by itself. A toleration makes a tainted node possible but leaves other nodes possible too. If the requirement is “run only on GPU nodes,” use an appropriate required label rule (or a node selector) to select that pool, plus a toleration for its taint if needed. If the requirement is “GPU nodes are preferred but not mandatory,” consider a preferred affinity and understand that a hard taint may still exclude those nodes without a toleration.
For example, dedicated=gpu:NoSchedule on GPU nodes and a toleration in the Pod do not force the Pod to use a GPU node. Without a required node label, the scheduler might place it on a general node with available resources. If the workload actually requests a GPU device, device availability adds another hard resource constraint, but that is separate from the logical targeting described by a toleration.
Avoid “fixing” a scheduling failure by adding a broad operator: Exists toleration for every taint. That can allow placement on nodes reserved for system components or other workloads. Match the exact taint key/effect needed and explain the ownership of that node pool.
Interview question 3: How do hard and soft spread rules change the outcome?
With DoNotSchedule, a placement that violates maxSkew is rejected. With ScheduleAnyway, the scheduler scores placements to favor reduced skew but can still schedule a Pod even if the preferred distribution cannot be met. That is a policy tradeoff: a hard constraint can leave the Pod Pending when a zone has no eligible capacity; a soft constraint may preserve service capacity while accepting temporary imbalance. Neither setting alone guarantees fault isolation if the selector, topology labels, or eligible-node set are wrong.
Explain maxSkew in terms of the count difference between a candidate domain after placement and the global minimum among eligible domains, with documented special handling when minDomains applies. A label or selector mistake can make a mathematically “correct” skew irrelevant to the service you intended to protect. In an interview, draw the counts and candidate node, then compute rather than saying “spread evenly” as an incantation.
If a zone becomes unavailable, hard spread constraints can block replacement Pods exactly when the service needs them. Discuss whether the workload should prefer cross-zone distribution in normal conditions but allow degraded placement during an incident. That decision depends on the service's availability and fault-domain goals, not on a universal preferred YAML value.
Interview question 4: What is the smallest manifest that proves the intent?
For an application restricted to labeled service nodes, allowed on dedicated=api:NoSchedule nodes, and spread across zones, the relevant fields can be shown without copying a full Deployment:
spec:
template:
metadata:
labels:
app: api
spec:
tolerations:
- key: dedicated
operator: Equal
value: api
effect: NoSchedule
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: workload
operator: In
values: [latency]
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: api
This fragment is an illustrative Pod template, not a complete deployable object. In a real cluster, verify labels on all candidate nodes, reserve sufficient resources, and decide how the constraint should behave when there is only one eligible zone. The selector app: api should match the Pods whose placement you want to balance; a typo there silently changes the scheduler's count.
An interviewer may ask you to remove one line. Removing the toleration makes the tainted pool unavailable; removing the required affinity allows other labeled or unlabeled pools depending on remaining rules; removing spread gives up the stated zone balance. This change-impact explanation is more convincing than reciting field names.
Interview question 5: How would you recover without weakening the policy blindly?
First identify the binding constraint from the actual eligible-node intersection. If no nodes match the required label, correct node labeling or the requirement after confirming workload intent. If a taint blocks every otherwise eligible node, add a narrowly matched toleration only if that Pod is authorized for the pool. If the spread rule blocks replacement during a zone outage, decide whether availability should override the normal balance objective and document the temporary change. If resource requests do not fit, add eligible capacity or revisit measured requests; changing affinity cannot create CPU.
Watch the pending Pod and scheduler events after each change, and verify where it actually lands. Do not equate PodScheduled=True with a healthy service; image pull, volumes, probes, and network can still fail. If you relax a constraint during an outage, set an owner and reversal condition so the service does not remain permanently co-located.
The exact recovery is an engineering judgment. Kubernetes documents the control semantics; it cannot decide whether your service should favor cross-zone resilience or quick replacement under a particular incident's customer impact.
Practice the constraint intersection
A concise answer is: “I would inspect FailedScheduling events, then intersect required affinity, taint toleration, resource fit, and hard spread for each candidate node. A toleration only permits a node; spread counts depend on eligible domains and selector. I would change the one policy that conflicts with the workload's actual placement intent, then verify the Pod's new node and readiness.”
| Practice question | What to rehearse |
|---|---|
| Compare cloud compute; design Kubernetes security and CI/CD | Explain when a dedicated node pool is a real requirement. |
| Design VPC, IP plan, management, storage, SSO | Tie node inventory to infrastructure constraints. |
| Explain Kubernetes autoscaling and traffic routing | Connect Pending Pods and eligible capacity. |
| Describe how you use Kubernetes | Give a concrete placement-debugging story. |
| Design secure Kubernetes with CI/CD | Justify pool isolation and narrow tolerations. |
Take one prompt, invent three labeled nodes and one Pending Pod, and compute the eligible set before explaining the fix. That exercise exposes whether you really distinguish affinity, taints, and spread.
Sources and further reading
- Kubernetes: Assign Pods to Nodes — node affinity and label rules.
- Kubernetes: Taints and Tolerations — scheduling effects and toleration semantics.
- Kubernetes: Pod Topology Spread Constraints — skew, eligible domains, selectors, and policy defaults.
- Kubernetes: Scheduling Framework — scheduler filtering and scoring.