cswork / libraryOpen practice workspace

Articles

Multi-Resource Scheduling Interview Questions: Dominant Shares, Fairness, and Tenant Quotas

Work through multi-resource scheduling interviews with dominant shares, weights, tenant quotas, placement constraints, and safe resource reservations.

A scheduler can give two tenants the same number of jobs and still allocate resources unfairly. One tenant's tasks may consume twice as much memory, while another tenant saturates CPU. Multi-resource scheduling interview questions ask you to make that imbalance measurable before proposing a queue or worker fleet.

The useful starting point is a resource vector, a normalized dominant share, and an explicit definition of eligibility. Fairness policy, hard quotas, and physical placement are separate decisions. Practise those decisions with Design a Fair GPU Training Job Platform.

Evidence boundary: The original Dominant Resource Fairness paper establishes a formal allocation model and its properties under stated assumptions. Kubernetes documentation describes quota admission behavior. The workloads and production design choices below are original interview exercises, not claims about any employer's interview process.

What is a dominant resource share?

For tenant i with allocation x across resources and cluster capacity C, normalize each component: CPU allocated divided by CPU capacity, memory allocated divided by memory capacity, and so on. The dominant share is the maximum of those fractions. It is dimensionless; adding raw CPU cores to gigabytes would be meaningless.

Suppose a cluster has 100 CPU units and 100 GiB of schedulable memory. A tenant holding 20 CPU units and 40 GiB has shares 0.20 and 0.40, so its dominant share is 0.40. This calculation says nothing about whether those resources sit on suitable nodes or whether the tenant's jobs are making progress.

In the simple unweighted DRF model, allocation increases the least dominant share among eligible users, subject to feasibility. The paper's formal properties belong to its model, including assumptions about task resource demands. They are not blanket guarantees for a production scheduler with arbitrary priorities, indivisible jobs, changing estimates, placement constraints, or multiple admission layers. Original DRF paper.

Start an interview by naming the denominator. Physical capacity, available capacity after system reservations, and capacity of a particular tenant pool are different quantities. Using a changing denominator without recording why can make a tenant's reported fairness improve even though its allocation has not changed.

Work through a two-resource allocation

This original exercise has two continuously backlogged tenants with fixed task demands. Tenant A needs 2 CPU units and 1 GiB per task. Tenant B needs 1 CPU unit and 2 GiB per task. Assume a homogeneous aggregate pool, divisible allocation for the first calculation, and no separate quota cap.

With a tasks for A and b tasks for B, CPU consumption is 2a+b and memory consumption is a+2b. A's dominant share is 2a/100; B's is 2b/100. Equal dominant shares imply a=b. Both capacity constraints then become 3a<=100, giving a=b=100/3 and a dominant share of 2/3 for each.

This is not equal allocation of each resource. A receives twice as much CPU as B, and B receives twice as much memory as A. Their largest normalized allocations are equal. That difference is the reason to use a multi-resource fairness model.

Allocation checkpointResource accounting
A=30 tasks; B=30 tasksCPU=90; memory=90 GiB; both dominant shares=60%.
A=33 tasks; B=33 tasksCPU=99; memory=99 GiB; both dominant shares=66%.
A=34 tasks; B=33 tasksCPU=101; memory=100 GiB; infeasible.
Fractional ideal: A=B=100/3Both resources reach 100; dominant shares reach 2/3.

For indivisible tasks, 33 and 33 is a feasible equal-share checkpoint. The remaining one CPU unit and one GiB cannot fit either task. The fractional answer is therefore a mathematical reference, not a runnable placement plan. State the rounding and tie-breaking rules rather than silently scheduling one third of a process.

Why are weights, guarantees, and quotas different?

A weight changes relative treatment among competitors. A hard quota forbids an allocation beyond a limit. A guarantee promises capacity under some defined conditions. A burst allowance permits temporary borrowing with rules for reclamation. Calling all four a tenant quota hides the policy the scheduler actually implements.

For an illustrative weighted policy, compare dominant_share/weight. With A's weight 2 and B's weight 1, equal weighted scores in this same fractional exercise imply a=2b. CPU reaches its bound when 5b=100, so b=20 and a=40. Memory use is 80 GiB. A's dominant share is 0.80 and B's is 0.40; divided by their weights, both scores are 0.40.

Now cap A at 60 CPU units. A cannot exceed 30 tasks regardless of its weight. The scheduler must consider the remaining eligible work and capacity after that cap applies. It should not keep selecting A, failing admission, and blocking everyone behind it.

Kubernetes ResourceQuota limits aggregate namespace resource consumption and can reject requests that violate quota. That admission mechanism does not itself promise a DRF allocation or a minimum share of cluster capacity. Treat it as a useful official example of why enforcing a ceiling and choosing a fair next tenant are separate responsibilities. Kubernetes ResourceQuota.

Ask how tenant hierarchy works. If teams can create unlimited child identities, a flat per-identity fairness rule may let one organization gain more aggregate service. Account and enforce allocation at the business boundary that owns the entitlement; introduce sub-queues only after the parent policy is clear.

What if the least-served tenant cannot fit?

Aggregate feasibility is necessary but insufficient. A job requesting four GPUs on one node cannot run on four separate nodes with one free GPU each. Memory cannot always move to the node that has spare CPU. Locality, accelerator type, affinity, and gang scheduling can further restrict placement.

Separate selection from placement. Selection identifies an eligible tenant according to policy; placement tries a concrete job against a resource snapshot. If placement fails, record the reason and consider another feasible candidate under a documented bypass rule. A strict global queue that never bypasses an unplaceable head job may leave usable capacity idle.

Bypassing also has a cost. Small jobs can continually occupy the holes a large job needs, causing starvation. Possible remedies include bounded bypass, reservations, drain periods, or pools for incompatible workload shapes. Present these as design options with consequences, not as properties automatically supplied by dominant shares.

For a gang of workers, reserve the complete required set before launching, or explain how partial launches time out and release reservations. A fairness score based only on running tasks can reward a tenant whose pending reservations occupy substantial capacity. Include held resources in the accounting contract.

How do you reserve capacity without overscheduling?

Two scheduler replicas can read the same free capacity and each launch a job that fits alone. Together they exceed the pool. A priority heap or accurate share formula cannot prevent this race.

Use an authoritative reservation transition with a version or compare-and-swap condition. The transition checks the current tenant budget and physical resource availability, then records a reservation with a unique attempt identity. Workers acknowledge acquisition; expired attempts release capacity through an idempotent state change.

A recommended record includes tenant, job, attempt, resource vector, pool, reservation state, owner epoch, and lease deadline. The precise schema is an interview design proposal. Its purpose is to make reconciliation possible when a worker starts but its acknowledgement is lost.

Fencing matters when leadership changes. An old scheduler must not launch work after a new leader has superseded its authority. A lease alone does not stop a delayed message from arriving; workers or the reservation service must reject stale epochs where the design depends on that guarantee.

Resource release should also be idempotent. A duplicate completion must not add capacity twice. Reconcile the ledger with actual running allocations, but do not erase a live reservation merely because one telemetry sample is missing. Separate suspected state from confirmed terminal state.

Which demand should fairness use?

Requested resources make admission predictable, but inaccurate declarations distort allocation. Measured usage captures actual consumption, yet noisy or delayed measurements can oscillate and reward under-reporting. Explain whether the contract allocates reservations, consumption over a time window, or a mixture.

For memory, oversubscription can cause hard failures rather than a gentle slowdown. For CPU, some workloads tolerate time sharing. GPUs may be exclusive, partitioned, or shared under device-specific constraints. A single abstract resource vector should not erase these operational differences.

Unknown job duration creates another distinction: fair concurrent allocation is not the same as fair completed work, equal waiting time, or equal billable cost. A tenant running slow tasks may hold a fair share while other tenants finish more jobs. Choose the fairness objective the product actually needs.

When a task changes phase, its resource shape may change. Recompute accounting at explicit transitions or reserve a conservative envelope. Frequent recalculation with instantaneous measurements can thrash placements; use documented measurement windows and hysteresis where appropriate.

How would you test the scheduler's claim?

Replay a deterministic workload with resource demands, arrival times, completions, and tenant policy changes. Check feasibility at every reservation transition, not only at the end. Then distinguish fairness violations from unavoidable fragmentation and from a tenant that simply has no eligible demand.

Useful adversarial cases include a capped tenant at the head of the queue, a large unplaceable gang, duplicate completion, leader failover during launch, and a tenant creating many sub-queues. Add a case where demand falls below entitlement so unused capacity can be borrowed under the chosen policy.

Measure normalized shares, queue age, rejected admissions, placement failures by constraint, reservation age, and actual resource utilization together. High utilization alone can hide a starved tenant; equal shares alone can hide a cluster that is half idle.

A strong spoken answer might be: “I calculate dominant shares from reservations in the appropriate pool, select the lowest policy-adjusted eligible tenant, and atomically reserve a feasible job. I expose fragmentation and quota rejection separately, and reconcile attempts after failover.” Each clause is a commitment you can test.

Sources and Further Reading

Primary references checked September 30, 2026. Product details and supported versions can change; the worked examples are original practice material.