Kubernetes Job Interview Questions: Retries, Duplicate Work, and CronJob Overlap

If a Kubernetes Job has one completion and one Pod at a time, can it charge a customer twice? Yes, if the program's side effect is not safe to repeat. Kubernetes can retry a failed attempt, and its own documentation says the same program may sometimes start twice even with parallelism: 1, completions: 1, and restartPolicy: Never. A CronJob adds a second question: whether scheduled runs may overlap. The interview answer must separate scheduling, execution attempts, and the business effect.

This guide is based on Kubernetes Job and CronJob documentation. The export scenario below is an original exercise, not a report of a named employer's interview. For adjacent practice, Design a Durable Cron Job Scheduler asks you to give a logical scheduled occurrence its own identity.

A compact answer to Job and CronJob questions

A Job asks Kubernetes to run Pods until a specified number of completions succeeds or a failure policy ends the attempt. backoffLimit limits retries; it does not make the container's external side effects atomic. With restartPolicy: Never, a failed Pod can be replaced. With OnFailure, a container can restart inside a Pod, and its restart count also matters to the Job's backoff calculation. The Job controller documentation distinguishes these paths.

A CronJob creates Jobs on a schedule. concurrencyPolicy governs Jobs created by the same CronJob: Allow permits overlap, Forbid skips starting a new Job while an earlier one is active, and Replace replaces the active Job with a new one. It is not a global lock across different CronJobs or manual Jobs. Kubernetes describes CronJob creation as approximate; two Jobs or no Job can arise for a scheduled time, so the work itself should be idempotent. Treat schedule controls as workload management and business idempotency as application responsibility.

Original interview case: the nightly invoice export

Imagine a batch process that runs at 02:00 UTC. It selects all paid invoices for a day, writes a CSV to object storage, and sends the accounting team a notification. Usually it finishes in eight minutes; after a database slowdown, it takes 75 minutes. At 03:00 another schedule fires. A Pod may also be evicted after the upload but before it records success.

Ask four clarifying questions before proposing YAML. Is the output one immutable export per accounting day, or can late invoices revise it? May two dates run concurrently? Is notification allowed once per logical export or once per attempt? What should happen after a missed schedule: catch up, skip, or require operator review? Those answers define correctness. concurrencyPolicy: Forbid alone cannot answer them because it only decides whether this CronJob starts another Job while one remains active.

Suppose the product owner wants one final export for each accounting date and one notification after the export is durably committed. Model a logical run key such as (tenant_id, accounting_date, export_version). The Kubernetes Job UID and Pod name are attempt identifiers, not the business key. Persist one run record with a uniqueness constraint on the logical key. Each Pod may retry claiming or resuming that record; it should not independently invent a second export because it has a different Pod name.

For the output, write to an attempt-specific temporary key, validate its row count and content digest, then perform a controlled publish step to a versioned final key or update a database pointer transactionally. For notification, use a durable outbox row keyed by the logical run, dispatched after the final pointer commits. The exact implementation depends on the database and storage guarantees; this is an editorial design inference, not a Kubernetes-provided transaction. It prevents “CSV uploaded, Pod died, duplicate CSV and duplicate email on retry” from becoming the default behavior.

What a retry limit actually limits

Kubernetes defaults .spec.backoffLimit to 6 for a normal Job, unless per-index retry configuration changes that default. Failed Pods are recreated with exponential backoff. An activeDeadlineSeconds can stop the Job after a total elapsed duration even if the retry budget is not exhausted. A failure policy can instead fail, ignore, or count particular Pod failures according to exit codes or Pod conditions. These details are version-sensitive; cite the current Job documentation in an interview rather than treating a memorized default as the design.

The important reasoning is broader than the number six. If a task times out after a remote API accepted a charge, a fresh Pod cannot infer that nothing happened. Retries mean another attempt may run; they do not imply the external action is safe to repeat. backoffLimit: 0 reduces automatic retries but cannot erase ambiguity when a Pod is disrupted or a CronJob creates an unexpected duplicate. An answer that relies only on “disable retry” fails the crash-after-commit case.

Be precise about where the work restarts. OnFailure can restart a container in the same Pod, so kubectl get pods may show one Pod while the program ran more than once. Never makes failed Pods easier to inspect but still allows replacement Pods. Count execution attempts from Pod status, container restart counts, Job status, events, and application run records. Do not use a single Pod count as proof of exactly-once execution.

Overlap policies do not provide exactly-once effects

Choose Allow when concurrent scheduled occurrences are valid and the workload has enough capacity. Choose Forbid when overlapping runs would contend or violate a contract, recognizing that a long run may cause later schedules to be missed. Choose Replace when the latest occurrence supersedes the previous one and the old work can be cancelled safely. For a financial export, Replace is risky unless partial uploads, leases, and notification state are recoverable. “The old Pod receives termination” does not undo side effects it already committed.

Kubernetes's CronJob documentation says the concurrency policy applies only to Jobs created by that CronJob. If an operator manually starts another Job from the same template, or a second CronJob targets the same tenant/date, neither is stopped by Forbid. The unique business run key and idempotent commit path still matter. This is the distinction an interviewer is likely probing when they ask, “Can the Jobs overlap anyway?”

Also define the timezone and missed-run policy. A schedule is interpreted under .spec.timeZone when supplied; time-zone changes and daylight-saving transitions matter for local business dates. startingDeadlineSeconds bounds how late a missed schedule can start, but an aggressively small value may cause a run not to be scheduled. Observe the Job's intended scheduled timestamp where supported rather than deriving the business date solely from Pod start time. A late Pod can start on the following calendar day.

Read the evidence before changing the manifest

In the exercise, accounting received two notifications for one date. Do not immediately change concurrencyPolicy and declare the incident fixed. Assemble a timeline: CronJob schedule time, Jobs created and their owner references, Pod UIDs and restart counts, application logical run keys, final object versions, and outbox/notification deliveries. Two notifications could come from two Jobs, two Pods within one Job, two container restarts, a dispatcher retry, or an operator's manual run. Each cause needs a different repair.

The useful kubectl questions are: Did the CronJob controller create one or more Jobs? Which Pod belongs to each Job? What do Job conditions and events say about failure and replacement? Was the old Pod still terminating when a replacement started? For logs, include --previous where a restarted container's prior output matters. Correlate timestamps with application IDs; Pod names alone do not identify the business operation.

Kubernetes notes that replacement behavior can lead to overlap while terminating Pods wind down. Even if a terminal Job condition is delayed until Pods finish on newer versions, that does not retroactively roll back an external write. A robust run record should record attempts and a final outcome separately. If a lease is used to coordinate active attempts, explain how an expired lease prevents a stale worker from publishing after a newer worker has taken over. A fencing token or conditional commit may be needed; a time-based lease by itself is not an authorization to overwrite newer work.

A useful failure matrix for the invoice export

Failure point	What a retry may repeat and Application invariant
Pod dies before selecting invoices	What a retry may repeat: SelectionApplication invariant: Same logical date maps to one run record.
Pod dies during temporary upload	What a retry may repeat: Partial object writesApplication invariant: Temporary keys are attempt-scoped and removable.
Pod dies after final pointer commit	What a retry may repeat: Publish stepApplication invariant: Conditional commit sees the already published version.
Pod dies after outbox insert	What a retry may repeat: Notification dispatchApplication invariant: One outbox event per logical run; consumer deduplicates delivery.
New schedule fires during slow old run	What a retry may repeat: Concurrent runApplication invariant: Overlap policy and per-date run identity are explicit.

This matrix is an original exercise asset. Use it to explain why “exactly once” is too vague: exactly one Job object, one Pod attempt, one final object, and one email delivery are different claims. You can often guarantee a unique database state for a logical export, while external email delivery may need idempotency keys or a documented at-least-once delivery contract. State exactly which effect your design can enforce.

How to answer follow-up questions aloud

If asked whether parallelism: 1 ensures one execution, answer no and cite Job replacement/restart behavior. If asked whether Forbid ensures no overlap anywhere, answer no because it scopes only one CronJob's Jobs. If asked whether Replace is a safe fix for a long-running job, ask whether partial effects can be cancelled and replayed. If asked why a run appeared late, examine schedule time, startingDeadlineSeconds, controller availability, and timezone, not only Pod start time.

Then give a compact design: “I choose an explicit business run key, persist attempt state, make the final effect conditional or idempotent, and keep schedule policy separate from correctness. I inspect Job, Pod, and application evidence to identify which layer duplicated work.” This response shows what you would implement and how you would prove it worked.

Final interview takeaway

Kubernetes Jobs manage retries and completions; CronJobs manage approximate scheduling and same-CronJob overlap. Neither can promise one external effect by configuration alone. Practice the durable scheduler design question by separating a logical occurrence, its attempts, and its committed effect. If you can trace the duplicate-notification incident across those three layers, you can defend both your manifest choice and your application design.

Sources and Further Reading

• Kubernetes: Jobs
• Kubernetes: CronJobs
• Kubernetes: Pod failure policy
• Kubernetes: Automatic cleanup for finished Jobs

Research checked September 28, 2026. Cluster version, controller settings, and application side effects can alter the observed path; inspect the running environment when debugging.