Articles
Kubernetes Storage Interview Questions: PVC Binding, Access Modes, and Recovery
Prepare for Kubernetes storage interviews with PVC binding, access-mode traps, node failure evidence, and a concrete data recovery acceptance test.
A PVC is Bound, the replacement Pod is still waiting, and someone suggests deleting both objects to “start clean.” In a Kubernetes storage interview, that proposal should change the conversation. The immediate problem is availability; the next command could turn it into data loss.
A strong answer follows the data through separate boundaries: claim selection, volume provisioning, scheduling, attachment, mounting, and application recovery. Each boundary needs its own evidence. A green Kubernetes object is useful, but it does not prove that the right database opened the right files.
Evidence boundary: Kubernetes documentation supports the platform facts linked below. The incident, diagnostic matrix, and recovery exercise are original preparation scenarios, not reported interview questions or results from a production cluster.
For an early rehearsal, open Design a production file storage service and add a failed-node constraint. Before proposing replacement infrastructure, explain which identifiers and application records you would preserve. This turns a broad design answer into a testable recovery argument.
What should you establish before troubleshooting a PVC?
Ask what the workload stores, which failure occurred, and what recovery means. A disposable build cache, a single-writer ledger, and shared uploaded files require different answers. Establish the namespace, claim name, storage driver, topology, expected writer count, and last known successful application operation.
Official fact: A PersistentVolume represents provisioned storage, while a PersistentVolumeClaim requests storage. Binding links a claim to a matching volume. Access modes and reclaim policy describe additional behavior; they do not replace an application recovery protocol. See the Persistent Volumes documentation.
For an interview, draw a short chain:
Pod → PVC → PV → storage driver → backend volume
Annotate it with identities, not just names. Record the PVC UID, bound PV, backend volume handle, and node. In our original ledger example, ledger-0 should open the existing claim data-ledger-0. A newly created empty claim with a familiar name would not satisfy that requirement.
Begin with read-only evidence:
kubectl -n finance get pod ledger-0 -o wide
kubectl -n finance describe pod ledger-0
kubectl -n finance get pvc data-ledger-0 -o yaml
kubectl -n finance describe pvc data-ledger-0
kubectl get storageclass
These commands are an investigation outline, not a repair script. Discover the actual PV name from the claim before inspecting it. Avoid assuming that a copied example's namespace or object names match the target cluster.
Why can a PVC remain Pending without a broken provisioner?
Official fact: A StorageClass using WaitForFirstConsumer delays binding or provisioning until a consuming Pod exists, allowing its placement constraints to influence storage selection. Setting nodeName bypasses the scheduler and can leave a claim Pending in this mode. Kubernetes recommends scheduling constraints such as a node selector instead. See Storage Classes.
That gives you a useful interview counterexample: an unused Pending claim can be expected. Restarting the storage controller before examining the consumer is premature.
In the original exercise, the ledger Pod needs a node in zone B. The class supports delayed binding, but its allowed topology offers only zone A. The important observation is the incompatible constraint set. More controller restarts cannot produce a node satisfying both requirements.
Ask for the Pod's scheduling events, the class's binding mode, and the volume's topology. Then distinguish three hypotheses: no consumer has triggered provisioning, the consumer cannot be placed, or provisioning was attempted and failed. Those hypotheses require different evidence and different owners.
Do not “fix” the exercise by removing every placement constraint. Some constraints protect availability or compliance requirements. Explain which restriction appears accidental, what changing it would permit, and how you would verify the resulting placement before exposing the application.
What does Bound prove, and what remains unproven?
Treat Bound as a checkpoint in the control plane. Continue asking whether the workload reached the storage and whether the storage contains usable application state.
This original diagnostic matrix prevents several unrelated failures from collapsing into “PVC issue”:
| Observed state | Next evidence to collect | What the observation does not prove |
|---|---|---|
| Claim Pending, no consumer | Binding mode and intended Pod | That provisioning has failed |
| Claim Pending, consumer unschedulable | Scheduler events and compatible topology | That the driver is unhealthy |
| Claim Bound, attachment error | Target node, attachment state, backend ownership | That a filesystem is corrupt |
| Volume attached, mount error | Node-side driver events and mount prerequisites | That another claim is needed |
| Mount succeeds, application rejects files | Application logs, ownership, format, expected path | That the data is lost |
| Application starts, records are missing | Volume identity and recovery checkpoint | That service health means correct recovery |
For the ledger, suppose the claim still references the expected volume, but the new Pod cannot attach it after a node failure. That moves the investigation beyond binding. Preserve the backend identity and examine whether the old node might still have access.
A useful follow-up is: “What observation would make you abandon your current hypothesis?” If the volume is mounted successfully and the database reports an unsupported data format, changing the StorageClass is no longer the leading intervention. The evidence has moved the failure boundary into the application.
Does ReadWriteOnce mean that only one Pod can write?
Official fact: ReadWriteOnce concerns a single node; multiple Pods on that node may access the volume. ReadWriteOncePod constrains access to one Pod and requires supported CSI components. ReadWriteMany allows access across nodes where the provider supports it. Access modes should not be treated as a general filesystem write-protection mechanism. See access modes.
The interview implication is concrete. If two ledger Pods land on the same node, RWO alone is not your database leader-election mechanism. If the application assumes one writer, describe how that assumption is enforced and tested.
Conversely, selecting RWX does not make an application safe for concurrent modification. Imagine two image processors updating the same manifest file. Shared reachability gives both processes access; it does not define atomic replacement, locking, or conflict handling. Explain the application's write protocol before choosing storage from an acronym.
A concise answer is: “I would select an access mode supported by the driver and required placement, then separately demonstrate the application's writer-safety property.” That is more precise than claiming Kubernetes automatically serializes all file writes.
How should you handle a node failure without creating two writers?
Official fact: Kubernetes warns that force-deleting a StatefulSet Pod can violate its identity guarantees if the old process remains alive. A replacement must not become a second process acting under the same identity. Review force deletion of StatefulSet Pods.
In our scenario, the control plane cannot contact the old ledger node. That is evidence of lost communication, not proof that its process stopped. Before proposing a forced detach or replacement, establish the storage system's fencing mechanism and the application's failover protocol.
The original decision sequence is deliberately conservative about evidence:
- Preserve the claim, volume, and application checkpoint identities.
- Establish whether the old writer can still reach storage.
- Use the documented infrastructure or application fencing procedure.
- Confirm the replacement uses the intended volume.
- Validate recovered application state before reopening writes.
This is not an instruction to wait indefinitely. It is a way to explain the availability-versus-integrity decision. If the interviewer imposes a short recovery objective, discuss preplanned replication or failover capacity rather than inventing a safe universal detach timeout.
Official fact: StatefulSets provide stable workload identity and storage relationships. They do not, by themselves, implement database replication or backup. See StatefulSets. In your design answer, assign those responsibilities explicitly to the database, storage platform, or backup system.
What survives deletion, and how would you prove a restore?
Official fact: Reclaim policy affects what happens after a claim is released. Retain leaves manual reclamation work; Delete can remove the underlying storage for supported volume types. Read the actual PV policy before treating object deletion as harmless cleanup. See Persistent Volumes.
For the ledger exercise, write down three separate objects: the Kubernetes claim, the backend volume, and the backup. Ask which operation affects each one. A retained volume is not an independently tested backup, and a recreated PVC is not proof of restored content.
Official fact: Kubernetes VolumeSnapshots rely on snapshot APIs, controllers, and CSI driver support. Snapshot deletion policy also matters. The existence of the API does not mean every installed driver implements snapshots. See Volume Snapshots.
Engineering inference: Application consistency must be demonstrated using the application's own recovery guarantees. A storage snapshot completing successfully is insufficient evidence that an arbitrary database transaction was captured in a usable state. Coordinate quiescing or recovery logs where the chosen database requires them.
Use this original acceptance exercise. Before the simulated failure, record acknowledged ledger transaction T420, its expected balance, and the checkpoint time. Restore into an isolated environment. Confirm the restored process uses a new intended volume, opens the expected database, and satisfies the agreed recovery point. If the objective permits losing five minutes, calculate the actual gap; do not quietly redefine success as “it booted.”
Check that transaction identifiers are not duplicated after replay, balances reconcile, and a new test write survives a restart. Record recovery duration separately from data loss. A fast restore with unacceptable missing transactions fails a different requirement than a correct restore that takes too long.
Official fact: Supported CSI volume cloning can provision a new volume from an existing PVC. See CSI Volume Cloning. In an interview, distinguish that mechanism from your validation plan: copying bytes still leaves identity, application compatibility, and cutover checks to perform.
Before production cutover, name the rollback point. Once new writes begin on the restored system, switching back can discard or fork those writes. Your plan should say how ownership transfers and how post-cutover changes would be reconciled.
Which questions turn these concepts into interview practice?
The selection emphasizes operational ownership and storage correctness rather than repeating an autoscaling question set.
| Interview question | Storage-specific follow-up to practice |
|---|---|
| Describe how you use Kubernetes | Trace one real workload from Pod to backend volume. |
| Design and operate a monolith on Kubernetes | Explain restart safety when the database volume remains attached elsewhere. |
| Design a production file storage service | Separate metadata recovery from file-content recovery. |
| Design highly available blob storage service | Define durability, availability, and restore acceptance independently. |
| Design VPC, IP plan, management, storage, SSO | Explain how storage placement constrains infrastructure recovery. |
For focused practice, start with Design a production file storage service. Add the ledger failure above and answer aloud without deleting any object in your proposed first step. Your explanation should identify the failed boundary, the safest discriminating observation, and the evidence required to reopen writes.