cswork / libraryOpen practice workspace

Interview guides · Bombardier

Bombardier Software Engineer Interview Questions & Guide 2026

System design practice for Bombardier software engineer candidates: reach a data model and an API contract, not a box diagram.

The 60-second brief

As a Software Engineer at Bombardier, you are at the intersection of high-stakes aerospace engineering and modern digital transformation. This role is not just about writing code; it is about building the systems that power one of the world's most prestigious aviation brands. You will contribute to business operations ranging from supply chain optimization and strategic planning tools to digital and AI-driven interfaces that support Bombardier's manufacturing and sales operations.

Getting the code to run is the floor. What usually separates answers is the case checked without prompting: empty input, a single element, duplicate keys, or a value that overflows the integer type you chose.

Focus areas

  • Keep a site operating during a network partition
  • Make every external side effect safe to retry
  • Reserve inventory without overselling under concurrent allocation

Overview

The impact of your work is tangible. Whether you are developing full-stack applications for internal stakeholders or building analytical tools that drive profitability, your code directly influences how Bombardier designs, manufactures, and supports its fleet. You will work in a high-compliance, high-precision environment where the emphasis is on reliability, scalability, and seamless integration with existing engineering workflows. This is an ideal role for engineers who thrive on technical complexity and want to see their solutions applied in a mission-critical global industry.

The Bombardier interview process

Bombardier candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screening

The title covers product work, platform work, infrastructure, mobile and frontend, and those are different jobs with different loops behind them. A screening call is the cheapest place to find out which one the seat is, and asking reads as experienced rather than fussy. The questions that separate them: what the team is on call for, what the last three projects were, and whether any round happens inside an existing repository instead of a blank file. Then say which of that you have done and which you have not. Claiming the whole posting is the fastest way to be found out one round later.

What to demonstrate

  • Whether you can locate your experience inside one flavour of the role honestly instead of claiming the entire requirements list
  • Whether you name what you have not done, which an experienced screener reads as a level signal and can plan the loop around
  • Whether what you want next matches what the seat is: someone who wants greenfield work landing on a team that mostly operates an existing system is a hire that leaves within the year

How to prepare

  • Mark every line of the posting as done, adjacent or new, and write one sentence for each adjacent line naming the closest thing you actually built
  • Split your last two years into rough percentages across feature work, operating and debugging live systems, and design or review, so a question about scope gets numbers rather than adjectives
  • Bring three questions that discriminate between seats: what the team is paged for, how much of the work is changing existing code versus standing up something new, and what shipped in the last quarter

Technical Interviews

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

  • In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
  • In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
  • Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

  • Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
  • Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
  • For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Resume Discussion

The person on this call usually cannot evaluate your code and does not need to. They write a short paragraph, and that paragraph is what a hiring manager skims when deciding who to put on your loop. So the test is not whether your work was hard, it is whether a non-engineer can repeat it correctly. Name systems by what they did rather than by their internal codename, give each project a shape (what was breaking, what you changed, what happened after), and keep the whole walkthrough near ninety seconds. Depth that cannot survive a paraphrase reads as vagueness.

What to demonstrate

  • Whether a non-engineer can restate your projects without distorting them, since their paraphrase is what travels to the hiring manager, not your sentences
  • Whether each project has a shape rather than a stack list: the failure or constraint, the change you made, the result and how it was measured
  • Whether you can say what was yours inside a team project without either inflating it or disappearing into the plural

How to prepare

  • Rewrite each headline project as two sentences with no internal system names and no acronyms outside your company, then say them to someone outside engineering and have them repeat them back. Fix whatever came back wrong
  • Attach one measured number to each project: the baseline, the change, and the window it was measured over. Where nothing was ever measured, say that plainly rather than reaching for a plausible percentage
  • Time the background walkthrough against a clock. If it runs past two minutes, compress the earliest role to a single clause and spend the recovered time on the most recent one

Group Exercises

Because the format is not fixed, the first job in the room is classification. Listen to the opening question and decide what it is: a probe into work you have already described, a fresh problem to solve now, or a conversation about how you operate. Each wants a different register, and the common failure is forcing a rehearsed structure onto a question that did not ask for it. Running a full design ritual on a ten-minute debugging question reads as not listening. When you cannot tell which it is, ask how long they want to spend and answer at that depth.

What to demonstrate

  • Whether the shape of your answer matches the question, so a yes-or-no gets answered before it is justified and an open prompt gets a direction before a detour
  • Whether you check how much depth is wanted instead of deciding for them, and whether you stop when the answer is complete rather than continuing until someone interrupts
  • Whether you can be redirected in the middle of an answer without restarting it from the beginning
  • Whether a question outside your experience gets an honest boundary followed by reasoning from what you do know, instead of a confident answer with nothing behind it

How to prepare

  • Rehearse one project at three lengths, roughly thirty seconds, three minutes, and a full walkthrough at the depth of a design review, and practise switching between them when someone interrupts mid-telling
  • Have someone ask you five questions of deliberately mixed type in one sitting without telling you the types, and score only whether you identified each one correctly before you started answering
  • Draft the sentence you will use to check depth, along the lines of asking whether the short version is useful here or they want the detail, and use it in a real conversation this week so the day of the round is not its first outing

Where candidates lose points

Judging delivery windows and cut-offs in UTC

A delivery appointment, a receiving cut-off and a same-day promise are all statements about local wall-clock time at a specific stop, so evaluating them in UTC misclassifies on-time performance for any leg that crosses a zone, and misses cut-offs by an hour on the two days a year a region changes offset. The subtlety that catches experienced engineers is that converting a future local time to a UTC instant at write time bakes in the offset rules as they were known then: when a jurisdiction changes its rules, an appointment stored that way silently moves, which is why a future window is better stored as local wall time plus an IANA zone identifier and resolved at read time, while a past event is correctly stored as an instant. Duration arithmetic needs the same care, because a transit that spans a DST boundary is genuinely 23 or 25 hours long in local terms even though the elapsed instant count is unchanged. Storing only a date with no zone - common on partner feeds - is the version of this that cannot be fixed downstream, so the zone has to be captured at ingestion.

Holding a database transaction open across a physical or third-party operation

It is natural to open a transaction, lock the position, call the rating or tendering API, and commit on the response, and it works perfectly until the partner's p99 goes from 200 milliseconds to thirty seconds. At that point every request holding a lock on a hot item-node pair queues behind it, the connection pool fills with transactions that are waiting on the network rather than on the database, and an unrelated service sharing the pool fails at the same moment. In this domain the effect is amplified because demand concentrates on a few hot rows during a promotion or a seasonal peak, exactly when partner latency is also degraded. The structural fix is to keep transactions short and local - commit the state change together with an outbox row, let a relay perform the external call, and reconcile asynchronously - accepting at-least-once delivery and making the effect idempotent rather than trying to stretch a transaction over something the database cannot roll back.

Assuming the input fits in memory

Ask how large the input is in bytes before committing to an in-memory algorithm; beyond that point the options are a single streaming pass, an external sort with bounded buffers, or a sketch that trades exactness for constant memory. An algorithm that assumes random access to the whole input is a different algorithm from one that sees each element once.

Comparing floating-point values for equality, or holding money in them

Binary floating point cannot represent 0.1 exactly, so repeated addition drifts and an equality check fails on values that are mathematically equal. Store currency as integer minor units or a decimal type, and compare floats against a tolerance you chose for a stated reason.

Questions & practice

Parse partner quantity and date segments without floating point

Easy · parsing · unit-of-measure · time zones

A partner sends a delimited flat file whose segments look like QTY+17:4.500:CS and DTM+232:20260317. Quantities carry up to three decimal places in a named unit; each item has an integer pack factor to eaches that is versioned by effective date. Produce observation_event rows with an integer quantity in eaches, the unit recorded, and occurred_at resolved, rejecting any line whose conversion is not exact. Files reach 4 GB. Single pass, no floating point. State your bounds and what you do with the zoneless date.

Approach

  1. Scan with indices, not a regex. Find the segment terminator, then split the payload on : by scanning forward and slicing — O(bytes) with no backtracking and no per-line allocation beyond the output row. A regex with nested quantifiers over 4 GB is where this job silently becomes CPU-bound.
  2. Parse the decimal as an integer mantissa plus a scale. 4.500 becomes (mantissa = 4500, scale = 3): accumulate digits into an int64, count digits after the point, and never call the runtime's float parser. Reject a mantissa that would overflow int64 when multiplied by the pack factor, checking before the multiply rather than after.
  3. Convert by integer arithmetic and demand exactness: numerator = mantissa * factor; emit numerator / 10^scale only when numerator % 10^scale == 0, otherwise reject the line with a reason code. Worked: 4.500 CS at 12 eaches per case gives 54000/1000 = 54, exact. 0.125 PL at 36 eaches per pallet gives 4500/1000 = 4.5, which is half a sealed unit and must fail at the boundary, not round three services away.
  4. Select the pack factor by the file's effective date, not by the current item master. Factors are properties of a point in time — a supplier moving a case from 12 to 10 must not reinterpret last quarter's file — so the lookup is a range query on the factor's validity interval.
  5. 20260317 carries no zone and no time. Attach the IANA zone of the node the segment refers to, captured at ingestion, because it cannot be recovered downstream; then pick and document one convention for a date-only field (start of local day is the usual choice) and store the raw local string beside the resolved instant so the assumption stays visible. Complexity: O(total bytes) time, O(longest line) working space, one pass.

Worked solution (20 min)

  1. Write the integer conversion as a pure function (mantissa, scale, factor) -> Result<int64> and unit-test it before touching the file reader.
  2. Table-drive the cases: exact conversion, non-dividing conversion, a mantissa near int64 overflow, a zero quantity, and a unit not in the ladder.
  3. Wire the scanner, emitting one rejection row per bad line with the raw segment attached so the partner can be shown what was sent.
  4. Feed a fixture file containing a date on a day the destination zone changes offset, and assert the resolved instant against a hand-computed value.
  5. Re-run the whole file and confirm identical output, including identical rejections.

Expected result: 4.500 CS at factor 12 yields 54 eaches; 0.125 PL at factor 36 is rejected as non-exact rather than stored as 4 or 5; every emitted row carries both the eaches integer and the source unit; the date-only field resolves through the node's zone, not the server's.

Verify it yourself

  • No float type appears anywhere between the file bytes and the stored quantity — grep the parser for it.
  • Summing the emitted eaches and converting back to the source unit reproduces the source decimal exactly for every accepted line.
  • Changing the process time zone leaves every resolved occurred_at unchanged.

Follow-up

  • The same file is re-sent in full tomorrow with three lines changed. What makes the second ingest a no-op for the unchanged lines?
  • A line names a unit that is not in the ladder for that item. Reject the line, reject the file, or quarantine it — and who finds out?
  • The partner starts sending an offset like +0100 on some dates. Does that change your storage decision for a future appointment as well as a past event?

Find the busiest ten-minute window per reader after deduplication

Medium · sliding window · event time · deduplication

One day of observation_event from fixed readers: source_id (reader), subject_id (container), occurred_at, received_at, dedupe_key. 300 million rows, partner and device replays resend whole windows, and some events arrive six hours after they occurred. For each reader, return the 10-minute event-time window containing the most distinct containers, with the window's start. Target linear work per reader after sorting. State the window semantics you chose, and what the job does with an event that lands after its window was already reported.

Approach

  1. Deduplicate before windowing, on dedupe_key, and keep the two concerns separate. A replayed batch is a no-op only if the duplicate is removed first; deduplicating inside the window logic counts a resend as traffic and reports a congestion peak that never happened. Dedupe is a hash set per reader-day or a sort-unique on the key — the same sort you need anyway.
  2. Partition by source_id and sort each partition by occurred_at, never by received_at. Arrival order is an artefact of device connectivity and partner batch cadence; a reader that went offline for six hours would otherwise show its entire backlog as one impossible burst. Sorting dominates: O(p log p) per reader of size p, O(n log n) overall.
  3. Two pointers over the sorted partition with a hash multiset container -> count and a separate distinct integer. Advance the right pointer, incrementing the count and bumping distinct only on a 0-to-1 transition; advance the left pointer while occurred_at[left] <= occurred_at[right] - 600s, decrementing and lowering distinct only on a 1-to-0 transition, and erasing the key from the map so the map's size and distinct never diverge. Each element enters and leaves once: O(p) amortised after the sort, O(window occupancy) space.
  4. Justify the candidate set rather than scanning every possible start. With half-open windows [t, t+600), sliding a window right until its left edge coincides with an event can only add events and never removes one, so the maximum is attained at some event's occurred_at; checking the p windows anchored at events is sufficient and exhaustive. State the half-open convention explicitly — an event exactly at t+600 belongs to the next window, and flip-flopping on that produces off-by-one disagreements between this job and whatever reads its output.
  5. Handle lateness by choosing a policy, not by hoping. Either run over a closed event-time batch with a stated lateness allowance — the day plus six hours, so the six-hour tail is inside the batch and the day-late tail is not — or emit under a watermark and restate the affected windows when a late event lands. Both are defensible; what is not defensible is silently dropping the late event, because that is how the busiest window quietly becomes the one with the best connectivity.

Follow-up

  • Two events for one container arrive from two readers a second apart. Is that congestion, a misread, or one container passing two portals?
  • You need this continuously rather than as a daily batch. What changes, and where does the memory go?
  • A device's clock is three hours fast. What does that do to this result, and how would you detect it from clock_offset_ms?

Size dock doors from overlapping appointment windows

Medium · sweep line · intervals · time zones

For one node and one local calendar day you have up to 200,000 planned dock appointments from shipment_leg: leg_id, planned_arrive_at, planned_depart_at (both TIMESTAMPTZ), plus the node's IANA zone. A door is occupied over [arrive, depart). Return the minimum number of doors that lets every appointment start on time, and the maximal interval over which that peak is sustained. Target O(n log n). Say how you derive the day's boundaries and how you break ties between an arrival and a departure at the same instant.

Approach

  1. Name the result before computing it: with one interchangeable door type, the minimum door count equals the maximum number of simultaneously occupied intervals. That equality is not a heuristic — interval graphs are perfect, so their chromatic number equals their clique number, and greedy left-to-right assignment achieves it. Say this, because it is what licenses solving a scheduling question with a counter.
  2. Emit 2n endpoints, (t, +1) at each arrival and (t, -1) at each departure, sort by t, and at equal t order the -1 before the +1. Half-open occupancy means a trailer leaving at 10:00 frees the door for one arriving at 10:00; ordering arrivals first inflates the answer by exactly the number of back-to-back handoffs, which on a well-packed schedule is most of them.
  3. Scan once with a running counter, tracking the maximum and the endpoint index where it was first reached. The peak is attained on a half-open interval between two consecutive endpoints, [t_i, t_{i+1}), not at an instant — report it that way or the operations team cannot act on it. Extend the interval while the counter stays at the maximum.
  4. Complexity: O(n log n) dominated by the sort, O(n) space. If rows already arrive ordered by planned_arrive_at from an index, use the min-heap variant instead — push each departure, pop all departures at or before the current arrival, and the heap size is the current occupancy. Same time bound, O(peak) space rather than O(n), which matters when peak occupancy is 40 and n is 200,000.
  5. Derive the day's boundaries from the zone, not from arithmetic on instants. Convert local midnight and the next local midnight to instants in the node's IANA zone; on a transition day that span is 23 or 25 hours, so adding 86,400 seconds silently drops or duplicates an hour of appointments. Decide explicitly what a zero-length dwell means — under half-open semantics it occupies nothing — and state it.

Follow-up

  • Doors are typed: some break pallets, some are parcel-only. Does max-overlap still equal the door count?
  • Appointments have a tolerance — a trailer may start up to 20 minutes late without penalty. How does that change the objective, and is it still solvable by a sweep?
  • Half the appointments are actuals and half are plans. Which do you sweep for tomorrow's staffing, and which for last week's utilisation report?

Decide whether available-to-promise is derived or stored

Hard · denormalisation · read path · cache staleness

Available to promise at one node equals inventory_position.qty in state on_hand minus the sum of allocation.qty in status held or committed for that key. The read sits inside checkout: 4,000 reads per second at peak, p99 budget 20 ms. inventory_position holds 40 million rows; during a promotion a single hot item-node pair carries about 2,000 open allocations. Decide whether to compute this per read or to maintain available_qty on the position row. Give both paths, the failure mode of each, and what the browse read may do that the allocate read may not.

Approach

  1. Cost the derived read in tuples, not in plans. One index probe into the position is trivial; the aggregate over allocation touches one tuple per open claim on that key. Even if only a tenth of peak lands on the hot pair, 400 reads per second times 2,000 rows is 800,000 index tuples per second for one item at one node, and it gets worse exactly as the key gets hotter. A partial index on status IN ('held','committed') does not help, because the row count is the cost, not the selectivity.
  2. Maintaining available_qty on the position row is nearly free on the contention axis, which is the point most candidates miss. The allocator already takes a row lock or a compare-and-set on that same position row, so the extra column costs no lock that was not already held, and the read collapses to a single primary-key probe well inside 20 ms.
  3. It is not free on the correctness axis. available_qty becomes a second derived value that can drift, so it joins qty under the same reconciliation job with its own re-sum against the allocation table, and a drift alert on that comparison is the operational price of the read speed.
  4. Split the two reads rather than arguing about one. The browse read may serve from a replica or a short-TTL cache and must err low, showing less than is there. The allocate read must be on the primary and inside the same transaction as the conditional update, because replication lag peaks during the write burst that made the key hot. Conflating them is what makes a replica read look safe in a benchmark.
  5. Pick the TTL against the drain rate, with a number. If a promotion burns 2,000 units in a minute, a 30-second cache is a thousand units stale, so the displayed figure should be a banded signal such as in stock or low stock rather than an exact count that the allocate step will then contradict.
  6. Do not denormalise availability across nodes into a single network number. A yes-somewhere answer cannot be allocated against, so sourcing will disagree with the page, and the customer sees a promise retracted after checkout rather than an honest out-of-stock before it.

Worked solution (40 min)

  1. Load a position with 2,000 open allocations and measure p99 for the derived read at 400 requests per second against the same read from a maintained available_qty column.
  2. Add available_qty, backfill it per key under the same lock the allocator takes, and write the re-sum query that proves it equals qty minus held and committed.
  3. Run both read paths in shadow for a fixed window, logging every disagreement with its key and magnitude.
  4. Point the browse read at a replica with a 30-second TTL and the allocate read at the primary, then measure how often the browse figure is higher than the allocatable truth.

Expected result: The maintained column serves the read in a single primary-key probe inside the 20 ms budget at 4,000 reads per second, the derived path misses it on the hot key, and shadow comparison shows zero disagreements on the primary path while the replica path is stale in the low-availability direction only.

Verify it yourself

  • p99 for the maintained read is flat as open allocations on the key grow from 10 to 2,000; the derived read's is not.
  • The re-sum finds zero drift after a sustained allocate-and-release workload.
  • Every observed browse-versus-primary disagreement is an under-report, never an over-report.
  • Killing an allocator mid-transaction leaves available_qty and the allocation rows consistent after the expiry sweeper runs.

Follow-up

  • Roll this out on a live system. How do both paths run side by side, what do you compare, and what result gates the cut-over?
  • The reconciliation job finds available_qty drifted by 3 on one key. What do you do, and what must you not do?
  • Lot-tracked items multiply the position rows per item-node. Does your answer change, and where does the read now spend its time?

Write the reconciliation query that finds ledger and position variance

Easy · aggregation · outer joins · reconciliation

inventory_movement(movement_id, item_id, node_id, lot_id, state, delta_qty) is the ledger; inventory_position(item_id, node_id, lot_id, state, qty, version, last_movement_id, reconciled_through_movement_id) is its running sum, written in the same transaction as each movement. Write one query that reports every key whose stored qty disagrees with the sum of its movements, including keys present in only one of the two tables, bounded by a watermark movement_id you pass in. State what the job does with each variance, and how you avoid re-summing four billion rows every night.

Approach

  1. Aggregate once and join once: a CTE doing SELECT item_id, node_id, lot_id, state, SUM(delta_qty) AS ledger_qty FROM inventory_movement WHERE movement_id <= :w GROUP BY 1,2,3,4, then FULL OUTER JOIN inventory_position USING those four columns. One pass plus one hash aggregate, O(n) in movements scanned; a correlated subquery per position row instead costs one index scan per key and is orders of magnitude slower at 40 million keys.
  2. Use FULL OUTER JOIN, not LEFT. A key with movements and no position row is the signature of a lost position update, and a position row with no movements is the signature of a write that bypassed the ledger. Both disappear under a LEFT JOIN from either side.
  3. Compare with IS DISTINCT FROM and COALESCE the join keys out of both sides, so a missing row reads as a variance rather than as NULL = NULL evaluating to unknown and being filtered away.
  4. Run both sides in one snapshot: either a single statement, or a REPEATABLE READ transaction. Reading positions in one transaction and movements in another manufactures variances for every key written in between.
  5. Make it incremental with reconciled_through_movement_id: sum only movements in (reconciled_through, :w] and add that delta to the previously verified total. The catch is that sequence values are handed out before commit, so a movement whose id is below :w can become visible after the scan and would then never be re-summed. Set :w to a max(movement_id) observed a few minutes earlier, beyond the longest write transaction, and full-scan one partition per night on a rotation as a backstop.
  6. A variance is reported, not repaired. Rewriting qty to match the ledger hides the double-applied retry that caused it; only a physical count may produce a cycle_count_adjust movement, and the location should be flagged unreliable to planning until it is counted.

Follow-up

  • The job finds 14 variances, all on one node, all negative. What do you look at first, and what would make you suspect the reconciliation job itself rather than the data?
  • How do you run this without the nightly scan competing with the ATP read path for buffer cache and I/O?
  • What does the query return for a key whose only movements are a receipt and its exact reversal, and is that row worth reporting?

How would you approach troubleshooting a motor or system that fails to start?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Say what you would check first and why it is the highest-information step.
  3. Work from the requirement backwards to the design.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

How would you modify an existing system (like a train fleet or aircraft part) for new accessibility requirements?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Work from the requirement backwards to the design.
  3. Clarify what is being asked and what a complete answer contains.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Allocate stock on hot item-node pairs without overselling

Medium · lock contention · concurrency control · sharding · hot keys

Allocation peaks at 3,000 writes per second and 70 percent of it lands on about 50 (item, node) pairs during a promotion, roughly 42 per second against a single position row. The measured allocate transaction is 4 ms. Design the call so the availability check and the allocation insert cannot interleave: name the mechanism, state its throughput ceiling on one row, and say what the losing caller sees. Then say what changes when one order line must claim quantity at two nodes atomically.

Approach

  1. Put the check and the write in one atomic step, in one of two shapes. Pessimistic: SELECT the position FOR UPDATE, verify on_hand minus outstanding held and committed allocations covers the request, insert the allocation and bump version. Optimistic: UPDATE inventory_position ... WHERE key = ... AND version = ? AND qty - reserved >= ?, and test the affected-row count. A read followed by an unconditional insert is the defect, and no isolation level repairs it.
  2. Do the ceiling arithmetic. The row lock is held from the moment it is taken until commit, so a 4 ms transaction supports roughly 250 allocations per second on one row against the 42 demanded, about six times headroom. That headroom is the entire budget, which is why nothing remote may sit inside the transaction: a 200 ms carrier or rating call drops the same row to about 5 per second, eight times under demand, and the pair queues until the pool is exhausted.
  3. Be exact about isolation rather than waving at it. Under PostgreSQL REPEATABLE READ a second writer to the same row aborts with a serialization failure the application must catch and retry; under InnoDB REPEATABLE READ a locking read sees the latest committed row and blocks instead, so identical code is safe on one engine and not the other. READ COMMITTED with a locking read or a version-predicated UPDATE is the portable form.
  4. Define the loser's experience. A serialization failure or a zero-rowcount CAS is an expected outcome, not an error to log and swallow: retry with jittered backoff a bounded number of times, then return a typed insufficiency so sourcing picks another node. Uncapped retries against one hot row are the real outage, so cap attempts and shed.
  5. Shed without overselling by making the hot key single-writer: route allocations through a log partitioned on hash(item_id, node_id) so one process touches the row and contention becomes queueing instead of lock waiting. The costs are head-of-line blocking per key and duplicate delivery on rebalance, so the consumer still needs the partial unique index on allocation to make a replay fail closed rather than double-claim.
  6. For two nodes atomically: inside one database it is one transaction taking both position rows in a deterministic order, ascending by primary key, so two lines claiming the same pair cannot deadlock. Across shards no such transaction exists, so it becomes a saga - hold at A, hold at B, release A by a compensating status change on failure - and the pair is briefly held but uncommitted, which is a state the order line's machine must represent.

Worked solution (30 min)

  1. Write both implementations against inventory_position and allocation: SELECT ... FOR UPDATE, and UPDATE ... WHERE version = ? with a rowcount test.
  2. Compute the single-row ceiling from the 4 ms hold, compare it to 42 per second, then recompute it with a 200 ms external call inside the transaction and state what breaks first.
  3. Write the concurrency test that fails without protection: N threads allocating against a position of 1, asserting exactly one allocation in status held and exactly one version increment.
  4. Extend to two nodes in one shard with the lock-ordering rule, then write the cross-shard saga with its compensating release and the line status that represents the half-held state.

Expected result: An atomic check-and-claim - a row lock or a version-predicated UPDATE whose affected-row count is tested - with allocation's partial unique index making a retry fail closed; a stated ceiling near 250 allocations per second on one row at a 4 ms hold against 42 demanded; no remote call inside the transaction; and a cross-shard split expressed as a saga with an explicit compensating release rather than a distributed transaction.

Verify it yourself

  • With 200 concurrent allocators against qty 1, exactly one succeeds, no position goes negative, and no second allocation row exists.
  • Every failure mode maps to a named caller behaviour: retry with jitter, or re-source.
  • The transaction contains no network call other than to the database.
  • The throughput ceiling claimed for the chosen mechanism is derived from a measured hold time rather than asserted.

Follow-up

  • Two lines claim the same two nodes in opposite order and the engine kills one. What is the lock ordering rule, and what does the survivor's retry look like?
  • Soft allocations expire. What does the sweeper do, and why is it a correctness component rather than housekeeping?
  • Demand on one pair reaches 400 per second. Which of the two mechanisms do you move to, and what do you give up?

Wave release screen slows linearly with the line count

Easy · n+1 · batching · latency

A wave-release screen lists open fulfilment_order_line rows for one node and, per line, its held allocation rows and the on_hand inventory_position quantity for that (item_id, node_id, lot_id, state). p95 is 40 ms for a 25-line wave and 3.4 s for an 800-line wave. The database reports 1,601 statements for the 800-line request, each under 1 ms, and no individual query is slow. Diagnose the cause, give the fix, and state the statement count and p95 you expect afterwards.

Approach

  1. Read the shape out of the numbers before forming a theory. 1,601 statements for 800 rows is 1 + 2N: one driver query plus two per line. Every statement being sub-millisecond rules out a bad plan, so the time is round trips, not work.
  2. Check that the arithmetic accounts for the whole regression. 1,600 round trips at roughly 2 ms each is about 3.2 s against a 40 ms baseline, which covers the observed 3.4 s. If it had only covered half, there would be a second defect and the N+1 fix alone would disappoint.
  3. Identify the two per-row statements by their normalised text: a single-row allocation lookup keyed by order_line_id and a single-row inventory_position lookup keyed by the four-column primary key. Confirm they are lazy relationship loads by removing those two fields from the response and watching the count fall to 1.
  4. Replace them with two batch statements. Collect the order_line_ids and item_ids from the driver query, then SELECT ... FROM allocation WHERE order_line_id = ANY($1) AND status IN ('held','committed') and SELECT ... FROM inventory_position WHERE node_id = $1 AND item_id = ANY($2) AND state = 'on_hand', and join them into maps in application memory. Three statements per request, independent of wave size.
  5. Verify the access paths rather than assuming them. The partial unique index on allocation (order_line_id, node_id, lot_id) WHERE status IN ('held','committed') leads with order_line_id, so it serves the first batch. inventory_position's primary key is (item_id, node_id, lot_id, state), so the second batch scans on the item_id, node_id prefix and filters on state; lot_id sits between them and is unconstrained, which is fine for a few hundred keys but is the reason this is a prefix scan rather than a point lookup.

Follow-up

  • The screen adds a per-line column showing the last movement's reason_code. How do you fetch that for 800 lines without reintroducing the fan-out?
  • A wave can now reach 12,000 lines. At what point does ANY($1) stop being the right tool, and what replaces it?
  • Nothing here was a slow query. What monitor would have caught this before an operator did?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

  • Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
  • Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
  • Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

  • Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
  • Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
  • Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

  • Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
  • Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
  • Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

  • For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
  • Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
  • Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
  • Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

  • Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
  • Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
  • Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
  • Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

  • Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
  • Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
  • Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

  • Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
  • Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
  • Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

Know where your numbers came from

Bring the two or three numbers the story rests on and know how they were collected. A p99 whose timer starts inside your handler excludes the time a request spent queued, so it can sit flat while users wait longer. Give the window, the percentile and what the measurement left out, or drop the number.

Describe a situation where you had to manage a conflict within your team.

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Pick a story where you made the decision, not one where you watched it.

Follow-up

  • How did you know your change caused the improvement?
  • What did you decide not to do, and why?

Describe your experience with C/C++, Java, or Full Stack development.

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. Pick a story where you made the decision, not one where you watched it.
  3. State the situation in two sentences and spend the rest on the reasoning.

Follow-up

  • What would you do differently if you ran that again?
  • What did you decide not to do, and why?

Tell me about a time you had a disagreement with a manager or coworker. How did you resolve it?

Medium · behavioural and engineering judgement

Approach

  1. State the situation in two sentences and spend the rest on the reasoning.
  2. Close with what you would do differently, concretely.
  3. Give the blast radius: what could have broken, and what you measured.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?

Describe a time you had to overcome a significant challenge at work.

Medium · behavioural and engineering judgement

Approach

  1. Name the disagreement and how you resolved it with evidence.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • What did you decide not to do, and why?
  • What would you do differently if you ran that again?