Articles

OpenMP Interview Questions: Data Sharing, Scheduling, and Reduction Errors

Practice OpenMP interview questions with data-sharing traces, private versus firstprivate, reduction initialization, loop scheduling, and numeric checks.

To answer an OpenMP interview question, trace who owns each variable, its initial value, and when the final result becomes available. Changing the schedule cannot rescue a shared accumulator that multiple threads update without synchronization.

This article uses official OpenMP 5.2 semantics and original C++ exercises. Version 5.2 is the cited reference, not a claim about the newest standard or complete compiler support. There are no candidate reports or employer-specific question-frequency claims. The local exercises verify arithmetic and ownership models; they do not claim a measured parallel speedup.

Explain the storage and synchronization contract before adding more threads.

Why is this parallel sum incorrect?

Consider this original defective loop over eight integers:

long long sum = 0;
#pragma omp parallel for shared(sum)
for (int i = 0; i < 8; ++i) {
    sum += i + 1;
}

The serial reference is 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 = 36. However, the loop's updates target the same accumulator. sum += value involves reading the old value, computing a new value, and writing it back. An iteration assignment does not make that sequence indivisible.

Official data-sharing fact: shared makes references use the original variable's storage. The loop iteration variable has its own predetermined sharing rules; its being private does not make the accumulator private. Shared clause, data-sharing attribute rules.

A simplified lost-update illustration starts with sum zero. Thread A reads zero while adding one. Thread B also reads zero while adding two. A writes one, then B writes two. The contribution of one has been overwritten. This explains the defect; it does not predict a guaranteed output for the whole program.

Do not promise that the faulty loop returns 35, 2, or any other specific number. A race invalidates reliable reasoning about the result. A single run that happens to print 36 does not prove that the accesses are safe. OpenMP memory model.

Repair the accumulator's ownership or synchronize its update. First changing static to dynamic scheduling merely changes when the defect may appear.

How do private and firstprivate change the answer?

Official privatization facts: private creates new storage according to the privatization rules. For an ordinary C++ scalar such as this accumulator, it does not copy the outer initialized value. Reading the new scalar before initializing it is invalid. firstprivate adds initialization from the original item before the construct executes. Private clause, privatization rules, firstprivate clause.

This original comparison matrix assumes a plain scalar, no aliases to the original object, and no additional clauses that publish a result:

Accumulator clause	Storage used during the loop	Starting value	Does it compute the outer sum?
`shared(sum)`	The original scalar	Outer value	Unsynchronized updates race
`private(sum)`	Private copies	Not initialized from outer scalar	No; initialize before reading, then combine explicitly
`firstprivate(sum)`	Private copies	Copies of outer value	No automatic combination into the outer scalar
`reduction(+:sum)`	Reduction copies	Addition identity, zero	Yes; combines partial results with original value

For the simple loop with an outer sum initialized to zero, firstprivate(sum) removes the shared update to that scalar, but its private totals do not become the outer sum. After the construct, the outer sum remains zero under these stated assumptions. The loop must both avoid conflicting updates and publish the required total.

Moving a temporary inside the loop can make its storage private, but check every object it accesses. A private pointer copied from an outer pointer can still point to the same shared allocation. Copying the pointer value does not duplicate the pointed-to array. If all threads write element zero through their pointer copies, the underlying conflict remains.

Official default-clause fact: default(none) requires explicit attributes for referenced variables whose attributes are not predetermined. It helps expose assumptions; it does not itself synchronize shared operations. Default clause.

Describe it as a review aid. A compiler can reject missing annotations, while an explicitly declared shared accumulator can still be logically wrong.

Why does reduction work, and what happens to a seed?

Official reduction facts: reduction creates private copies initialized by the reduction identifier, then combines them with the original item's value. For built-in C/C++ addition, the private initializer is zero and the combiner adds contributions. Reduction clause, built-in reduction identifiers.

Here is the original corrected loop with a deliberately nonzero starting value:

long long sum = 10;
#pragma omp parallel for default(none) reduction(+:sum) schedule(static, 2)
for (int i = 0; i < 8; ++i) {
    sum += i + 1;
}
// After the combined parallel-for completes, sum is 46.

The expected arithmetic is 10 + 36 = 46. The ten is not copied into every addition-reduction accumulator. With a two-thread static-two illustration, one partial contribution is 1 + 2 + 5 + 6 = 14; the other is 3 + 4 + 7 + 8 = 22. Combining those with the original ten gives 46.

This distinction matters when repairing a manual accumulation. If you initialize every thread's local sum to ten and later add all those locals, a two-thread implementation produces 56. You have counted the seed twice. A correct manual version starts partial contributions at zero and includes the global seed once.

A reduction is appropriate when the operation and intended combination agree. It does not make an arbitrary sequence-dependent recurrence independent. If iteration i reads an output from iteration i−1, distributing iterations requires a different algorithm or an explicit dependency strategy.

Also check representable ranges. A reduction avoids the update race, but it cannot make signed integer overflow valid. The small positive fixture fits easily in long long; a production calculation needs a justified bound or a different numerical representation.

How should you choose static, dynamic, or guided scheduling?

Official scheduling facts: with an explicit chunk size, static scheduling assigns chunks round-robin by thread number. Dynamic scheduling lets a thread request another chunk after finishing its current one. Guided scheduling reduces chunk sizes as work proceeds. Runtime scheduling uses the runtime schedule setting. Official OpenMP 5.2 reference guide.

For the earlier eight iterations, this original two-thread, static-two assignment is fully specified:

Thread	Iterations assigned	Values added	Partial sum
0	0, 1, 4, 5	1, 2, 5, 6	14
1	2, 3, 6, 7	3, 4, 7, 8	22

Do not apply this exact mapping to schedule(static) without a chunk size. Do not claim a unique mapping for a dynamic run without observing it. The example also assumes a team of two; requesting a thread count and observing an actual team are separate steps.

Now assign work costs 9, 9, 1, 1, 9, 9, 1, 1 to iterations zero through seven. Under the illustrated static-two mapping, thread zero receives cost 36 and thread one cost 4. The sum stays correct with reduction, while completion time can be dominated by imbalance.

Dynamic chunks may distribute that skew more evenly, but they also introduce scheduling work and can change locality. These are performance hypotheses to measure, not guaranteed wins. For uniform adjacent-array work, start by measuring static scheduling; variable-duration work gives a reason to compare alternatives.

Keep the input, compiler settings, actual team size, and workload fixed when comparing schedules. Measure whole-operation time as well as per-thread work. Include a serial baseline: a tiny loop may spend more time forming a team than performing its additions.

Can a correct reduction disagree with the serial result?

Yes, especially for floating-point arithmetic. Official reduction-order fact: the order of combining values is unspecified, so sequential and parallel executions are not guaranteed to produce bitwise-identical results. Reduction scoping semantics.

An original binary floating-point illustration uses 1e16, −1e16, and 1. In common double arithmetic, (1e16 + -1e16) + 1 evaluates to one, while 1e16 + (-1e16 + 1) evaluates to zero because the inner addition loses the unit at that scale. Both expressions have the same real-number sum but different rounding paths.

Therefore, a last-bit difference alone does not diagnose a race. Investigate storage and synchronization separately from the numerical error budget. If the discrepancy is large, investigate it before blaming floating-point rounding. Check skipped iterations, seed duplication, overflow, invalid values, and the conditioning of the calculation.

State the comparison contract before writing the test. Exact equality is appropriate for the bounded integer fixture. A floating-point result may need absolute and relative tolerances justified by the data scale and downstream requirement. Near zero, relative error alone is often unsuitable.

If bitwise reproducibility is required, propose a documented algorithm: fixed partitions, a fixed combination order, and controlled arithmetic settings, then test that implementation. Merely specifying a static schedule does not establish every one of those conditions. Improved summation methods can reduce error without automatically guaranteeing identical results across environments.

What do barriers and compiler checks add to debugging?

A barrier after a racy update cannot reconstruct an overwritten contribution. Synchronization that governs when results are consumed and synchronization that protects an individual update solve different problems.

Official nowait fact: nowait removes synchronization that would otherwise occur at the end of a supported construct, including its implicit barrier. Nowait clause. If a later step consumes a reduction result within an enclosing parallel region, show the synchronization that makes the result ready before use. Avoid deleting a barrier simply because the independent iterations look safe.

For a shared scalar, an appropriately applied atomic update can protect that operation. A broader critical region may protect a multi-step invariant, such as checking a balance and then updating it. Neither choice is an automatic replacement for a reduction's partial-result strategy or a guarantee of speed. Explain which conflicting accesses share the same protection. Official synchronization reference.

Before treating any timing as OpenMP evidence, verify the compiler and runtime. A program that compiles while ignoring pragmas may run serially and appear correct. Record the OpenMP-enabled build command, compiler version, _OPENMP value where available, and actual team size from a parallel region.

Local verification boundary: the Apple Clang available for this article's preparation rejects -fopenmp. The saved checks verify the serial reference, static-two partition arithmetic, seed handling, and rounding example in a separate local model. They do not execute an OpenMP team or demonstrate parallel throughput.

For your own supported environment, compare the corrected program across an empty input, one element, more threads than elements, skewed costs, and several team sizes. Confirm the exact integer reference before timing. Do not use a test that requires the defective race to fail every run; that would confuse reproducibility of a symptom with proof of a defect.

Practice ownership and work distribution

Question	What to practice
Improve concurrency beyond a single lock	Compare shared serialization with partitioned work.
Fix race condition in concurrent deposit	Trace a read-modify-write conflict.
Design a Thread-Safe Shared Counter	Specify every access to the shared invariant.
Scale crawler with thread pool	Reason about uneven work and worker capacity.
Design a multithreaded CPU 1D convolution	Assign outputs while sharing read-only inputs safely.

Start with Fix race condition in concurrent deposit. Draw the conflicting reads and writes, then explain how an OpenMP reduction changes the sum exercise's storage and final combination. Finish by stating what your tests prove and which performance claims still need measurement.