cswork / libraryOpen practice workspace

Interview guides · Cruise

Cruise Software Engineer Interview Questions & Guide 2026

Software engineer prep for Cruise: design exercises weighted toward the constraints of supply chain and logistics work.

The 60-second brief

As a Software Engineer at Cruise, you are responsible for building the intelligent software systems that power full-scale autonomous vehicle (AV) fleets. Your work directly impacts how self-driving vehicles interact with complex real-world environments, navigate urban roads safely, and deliver seamless transportation to users. Whether working on onboard perception, motion planning, embedded control systems, remote assistance, or massive cloud infrastructure, your software drives the core technology of autonomous mobility.

The shape of the workload matters more for your prep than the industry label does. Read-heavy serving, write-heavy ingestion and scheduled batch processing have different binding constraints and fail in different places, so find out which one the team lives in before picking design topics.

Focus areas

  • Make every external side effect safe to retry
  • Keep a site operating during a network partition
  • Reconcile a movement ledger against derived positions

Overview

The engineering challenges at Cruise exist at the intersection of high-concurrency cloud computing, low-latency edge processing, and safety-critical hardware integration. As a Software Engineer, you will design systems that process terabytes of sensor telemetry—including LIDAR, radar, and camera feeds—in real time. The code you write must satisfy rigorous safety constraints while maintaining low latency, fault tolerance, and high availability across millions of miles of real-world driving.

Joining Cruise offers the opportunity to tackle some of the hardest engineering problems in robotics and distributed systems. You will collaborate with cross-functional teams spanning systems engineering, vehicle hardware, AI research, and product operations. Success in this role requires strong foundational computer science skills, clean system design capabilities, and a pragmatic approach to solving complex, real-world problems under demanding reliability standards.

The Cruise interview process

Cruise candidates report 6 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Phone Screen

The title covers product work, platform work, infrastructure, mobile and frontend, and those are different jobs with different loops behind them. A screening call is the cheapest place to find out which one the seat is, and asking reads as experienced rather than fussy. The questions that separate them: what the team is on call for, what the last three projects were, and whether any round happens inside an existing repository instead of a blank file. Then say which of that you have done and which you have not. Claiming the whole posting is the fastest way to be found out one round later.

What to demonstrate

  • Whether you can locate your experience inside one flavour of the role honestly instead of claiming the entire requirements list
  • Whether you name what you have not done, which an experienced screener reads as a level signal and can plan the loop around
  • Whether what you want next matches what the seat is: someone who wants greenfield work landing on a team that mostly operates an existing system is a hire that leaves within the year

How to prepare

  • Mark every line of the posting as done, adjacent or new, and write one sentence for each adjacent line naming the closest thing you actually built
  • Split your last two years into rough percentages across feature work, operating and debugging live systems, and design or review, so a question about scope gets numbers rather than adjectives
  • Bring three questions that discriminate between seats: what the team is paged for, how much of the work is changing existing code versus standing up something new, and what shipped in the last quarter

Technical Screening

Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.

What to demonstrate

  • Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
  • Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
  • Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
  • Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply

How to prepare

  • For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
  • For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
  • Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last

Interview Loop

Where the day includes a partner from product, design or data, that conversation is weighted like the technical ones and prepared for least. They are deciding one thing: whether having you in the room makes their decisions cheaper. That means options with costs attached, not implementation detail and not "it depends". An estimate someone can plan against — a range, the assumption that would push it to the high end, and what you would drop to hit the low one — is worth more than a confident single number, which everyone present already knows is wrong.

What to demonstrate

  • Whether an estimate comes as a range with the assumption most likely to break it, and states what a specific scope cut would actually buy
  • Whether a technical constraint is handed over as a choice with consequences on their side, rather than as a verdict they have no standing to argue with
  • Whether you establish what decision is on the table before proposing anything
  • Whether risk is raised while it can still change the plan, with the trigger that would confirm it, instead of reported afterwards as a slip

How to prepare

  • Take a project that shipped late and write the two-sentence warning you could have given three weeks earlier, naming what you would have needed decided at that point
  • Rehearse one estimate out loud until it arrives in three parts: the range, the single assumption that would blow it, and the smallest thing you would cut to protect the date
  • Rewrite an objection you have actually made — the "we can't do that" version — as two options with their costs, so the choice ends up with the person who owns it

Technical Depth

Most of the time lost in this format is not lost to thinking. It goes to a standard-library call you half-remember, an off-by-one in a loop bound, and a debugging loop that mutates code at random until something passes. When output is wrong, stop re-reading the whole function: take the smallest input that reproduces it and walk the state through by hand, printing intermediates if the environment allows. Guessing at a fix without a failing case you understand is how a five-minute bug becomes twenty, and the clock does not pause while you do it.

What to demonstrate

  • Whether you reach the right structure without a detour, and can write it from memory rather than only recall that one exists
  • Whether overflow is considered where the language has fixed-width integers, since a signed 32-bit value stops at 2,147,483,647 and then wraps in Java, is undefined behaviour in C++, and does not arise in Python, whose integers grow instead
  • Whether recursion depth is treated as a constraint on large inputs, given that CPython's default limit is 1000 frames and a deep recursion can exhaust the stack in any language where an iterative version would not
  • Whether a failing case is isolated and explained before any edit is made to the code

How to prepare

  • From an empty file and with no references open, implement the pieces you lean on most: a heap push and pop, an iterative DFS with an explicit stack, and a binary search whose midpoint is written lo + (hi - lo) / 2, which avoids the overflow that (lo + hi) / 2 can hit in a fixed-width integer type
  • Time yourself on the ten library calls you look up most, such as sorting with a custom comparator, splitting and joining strings, and finding the next key at or above a value in an ordered map, until the lookup is gone
  • Take a solution you know is broken and, before touching it, write one sentence naming the input, the expected value and the actual value. Repeat until you do it without deciding to.

Technical Breadth

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

  • In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
  • In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
  • Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

  • Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
  • Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
  • For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Behavioral Round

This round is deciding whether a change you make without supervision can be allowed to reach production. It is scored on what you knew at the moment you decided, not on how it turned out, so a story that opens with the result and works backwards reads as luck retold as judgement. Say what the options were, what you did not know, what you did to shrink the unknown before committing, and what you accepted as the worst plausible case. The detail that separates answers is a bound: how many users, how much data, and for how long, if you had been wrong.

What to demonstrate

  • Whether the reasoning you give was available at the time you decided rather than after the result came in, since a story whose deciding evidence arrived later describes an outcome and not a judgement
  • Whether you can put units on the exposure (users, rows, minutes of degraded service) and whether the containment you chose actually bounded it: a canary bounds the request path it fronts, while a background job writing to a shared table reaches every user regardless of which version served their requests
  • Whether the reversal path existed before you shipped or was improvised during the incident, and whether it restores state or only stops further damage

How to prepare

  • For your three largest changes, write down the one thing you would have had to be wrong about for it to fail, and what your best estimate of it was on the day you shipped. If you never held an estimate, that is the gap the follow-up questions will find
  • Write the undo procedure for one of those changes as it existed at the time, then mark which steps restore data and which only stop new damage. Turning a flag off or reverting a deploy ends the new writes; rows already written come back only from a copy you kept, and a dropped column comes back empty unless something outside the schema holds the values
  • Rehearse one story from the decision point forward and stop before the outcome, then have someone ask what you would do next. If the story only works with the ending attached, it is an anecdote rather than a decision you can defend

Where candidates lose points

Floating-point quantities and implicit unit-of-measure conversions

Binary floating point cannot represent most decimal fractions exactly, so a chain of pallet-to-case-to-each conversions accumulates a residue that a later rounding turns into a unit that was created or destroyed, breaking conservation with no concurrency involved at all. The bug is slow and non-local: it surfaces as a cycle-count variance weeks later, at a node nobody changed, and it is untraceable because the arithmetic looked correct at every individual step. The fix is to hold quantities as integers in the smallest transacting unit, carry the UoM explicitly on every row that carries a number, store conversion factors as integers that are versioned in time, and reject a conversion that does not divide exactly instead of rounding it. The same discipline applies to money on the freight side, where an accessorial in floating-point dollars produces invoice reconciliation differences of a cent that cost more to investigate than the freight.

Ordering events by arrival rather than by event time

Offline handhelds, batch partner feeds and store-and-forward gateways deliver observations out of order as a matter of course, so a pipeline that folds whatever arrived last will flap a delivered shipment back to in transit and recompute on-time performance from the wrong facts. Message-broker ordering guarantees do not rescue this: per-partition ordering only holds within a partition, so unless the producer keys by the entity being tracked, two events for one leg can land on different partitions and be processed concurrently. The defence has two halves that are often confused - deduplicate on a stable key, then fold with a monotonic status lattice ordered by occurred_at - and both are needed, because deduplication alone still lets a stale event win. Note also that occurred_at is device-reported and therefore sometimes wrong, which is why storing received_at and a measured clock offset beside it is what makes event-time logic auditable instead of merely plausible.

Choosing a schema before the access patterns are known

Write the queries first, with their filters, sort orders, cardinalities and which ones sit on the latency-critical path, then design tables and indexes to serve them. An index nothing queries still costs write throughput and storage, and a hot query with no supporting index becomes a full scan that only hurts once the table is big.

Starting work without saying what you are about to spend time on

State the plan before executing it: the approach, roughly how long it will take, and what you intend to leave hand-waved. That gives the interviewer a chance to redirect you in ten seconds rather than watching you spend fifteen minutes on the wrong sub-problem.

Questions & practice

Write a function to reverse a linked list in-place, handling all edge cases cleanly.

Medium · data structures and algorithms

Approach

  1. Walk one small example through your approach before writing the whole thing.
  2. Name the brute-force solution and its complexity before improving on it.
  3. State the target complexity and say which constraint rules the naive version out.

Follow-up

  • Which test case would catch an off-by-one here?
  • What is the worst case, and how likely is it on real data?

Construct a word completion engine using a Trie combined with Depth-First Search (DFS).

Medium · data structures and algorithms

Approach

  1. State the target complexity and say which constraint rules the naive version out.
  2. Choose the data structure from the access pattern, not from familiarity.
  3. Walk one small example through your approach before writing the whole thing.

Follow-up

  • How does this change if the input no longer fits in memory?
  • Which test case would catch an off-by-one here?

Design an LRU cache with timestamp-based eviction and thread-safe operations.

Medium · data structures and algorithms

Approach

  1. Choose the data structure from the access pattern, not from familiarity.
  2. Name the brute-force solution and its complexity before improving on it.
  3. State the target complexity and say which constraint rules the naive version out.

Follow-up

  • How does this change if the input no longer fits in memory?
  • Which test case would catch an off-by-one here?

Implement a local key-value datastore in memory that supports multi-level isolated transactions.

Medium · data structures and algorithms

Approach

  1. Name the brute-force solution and its complexity before improving on it.
  2. Restate the input: its shape, its size, and what is guaranteed about it.
  3. Walk one small example through your approach before writing the whole thing.

Follow-up

  • What is the worst case, and how likely is it on real data?
  • How does this change if the input no longer fits in memory?

Find the busiest ten-minute window per reader after deduplication

Medium · sliding window · event time · deduplication

One day of observation_event from fixed readers: source_id (reader), subject_id (container), occurred_at, received_at, dedupe_key. 300 million rows, partner and device replays resend whole windows, and some events arrive six hours after they occurred. For each reader, return the 10-minute event-time window containing the most distinct containers, with the window's start. Target linear work per reader after sorting. State the window semantics you chose, and what the job does with an event that lands after its window was already reported.

Approach

  1. Deduplicate before windowing, on dedupe_key, and keep the two concerns separate. A replayed batch is a no-op only if the duplicate is removed first; deduplicating inside the window logic counts a resend as traffic and reports a congestion peak that never happened. Dedupe is a hash set per reader-day or a sort-unique on the key — the same sort you need anyway.
  2. Partition by source_id and sort each partition by occurred_at, never by received_at. Arrival order is an artefact of device connectivity and partner batch cadence; a reader that went offline for six hours would otherwise show its entire backlog as one impossible burst. Sorting dominates: O(p log p) per reader of size p, O(n log n) overall.
  3. Two pointers over the sorted partition with a hash multiset container -> count and a separate distinct integer. Advance the right pointer, incrementing the count and bumping distinct only on a 0-to-1 transition; advance the left pointer while occurred_at[left] <= occurred_at[right] - 600s, decrementing and lowering distinct only on a 1-to-0 transition, and erasing the key from the map so the map's size and distinct never diverge. Each element enters and leaves once: O(p) amortised after the sort, O(window occupancy) space.
  4. Justify the candidate set rather than scanning every possible start. With half-open windows [t, t+600), sliding a window right until its left edge coincides with an event can only add events and never removes one, so the maximum is attained at some event's occurred_at; checking the p windows anchored at events is sufficient and exhaustive. State the half-open convention explicitly — an event exactly at t+600 belongs to the next window, and flip-flopping on that produces off-by-one disagreements between this job and whatever reads its output.
  5. Handle lateness by choosing a policy, not by hoping. Either run over a closed event-time batch with a stated lateness allowance — the day plus six hours, so the six-hour tail is inside the batch and the day-late tail is not — or emit under a watermark and restate the affected windows when a late event lands. Both are defensible; what is not defensible is silently dropping the late event, because that is how the busiest window quietly becomes the one with the best connectivity.

Worked solution (30 min)

  1. Write the window semantics as one sentence — half-open [t, t+600), anchored at event timestamps — at the top of the function, and make the comparisons match it.
  2. Implement dedupe, sort, then the two-pointer scan with the multiset and the explicit distinct counter.
  3. Build a fixture for one reader: twelve events, one container seen three times inside a window, one duplicate dedupe_key, and one event at exactly t+600.
  4. Hand-compute the expected maximum and window start, then run.
  5. Re-run with the fixture shuffled and with the duplicate resent five more times, confirming the answer is unchanged.

Expected result: The container seen three times contributes 1 to the distinct count; the resent duplicates contribute 0; the event at exactly t+600 falls in the next window; the reported start is an event timestamp, not a rounded clock boundary.

Verify it yourself

  • A brute-force O(p^2) scan over all event-anchored windows agrees on a 1,000-event fixture.
  • The distinct counter equals the multiset's key count at every step — assert it in the loop while testing.
  • Sorting by received_at instead of occurred_at changes the answer on a fixture with a backlogged device, which is the test that proves event time is doing work.

Follow-up

  • Two events for one container arrive from two readers a second apart. Is that congestion, a misread, or one container passing two portals?
  • You need this continuously rather than as a daily batch. What changes, and where does the memory go?
  • A device's clock is three hours fast. What does that do to this result, and how would you detect it from clock_offset_ms?

Add a required unit column and uniqueness to a live ledger

Hard · online migration · lock timeouts · unique constraints

inventory_movement holds 4.2 billion rows on PostgreSQL 15 and takes about 2,000 inserts per second around the clock. Two changes are required with no downtime: add uom_code TEXT NOT NULL, whose truthful value must come from the conversion factors in force at each row's occurred_at rather than from today's item master; and make idempotency_key UNIQUE, although duplicates predating the retry fix already exist. Give the ordered steps, the lock each one takes, its expected duration and its rollback.

Approach

  1. ADD COLUMN with a non-volatile DEFAULT has been metadata-only since PostgreSQL 11, so no rewrite, but it still takes ACCESS EXCLUSIVE for an instant. That lock queues behind any long-running reader while blocking every statement behind it, so one 10-minute analytics scan turns a millisecond change into a 10-minute outage. Set lock_timeout to 2s and retry in a loop rather than waiting.
  2. Reach NOT NULL without a scan under an exclusive lock: ADD CONSTRAINT ... CHECK (uom_code IS NOT NULL) NOT VALID, which is instant, then VALIDATE CONSTRAINT, which takes only SHARE UPDATE EXCLUSIVE and runs concurrently with inserts, then SET NOT NULL, which PostgreSQL 12 and later proves from the validated check and so skips its own scan. Drop the redundant check afterwards.
  3. The truthful backfill is the long pole and must read the versioned conversion factor as of each row's occurred_at, not the current item master, or a supplier moving a case from 12 units to 10 retroactively reinterprets last quarter. Batch by primary-key range, a few thousand rows per transaction, with a sleep between batches and a resumable cursor in a control table. At 5,000 rows a second, 4.2 billion rows is roughly ten days of throttled work: that is a schedule, not a failure, and it must not block the deploy.
  4. The UNIQUE index cannot be built until the duplicates are gone. CREATE UNIQUE INDEX CONCURRENTLY aborts on the first violation and leaves an INVALID index behind that is ignored by queries but still carries update overhead on every insert, and it can only be removed with DROP INDEX CONCURRENTLY. So: find the duplicates, resolve each one as a compensating reversal rather than a DELETE, since every duplicate key is a double-applied movement and deleting it fixes the constraint while leaving the position wrong.
  5. Then CREATE UNIQUE INDEX CONCURRENTLY, which makes two passes over the table and waits out concurrent transactions, so budget hours here, and note it cannot run inside a transaction block: a migration framework that wraps every step in BEGIN will fail on this one. Promote it with ALTER TABLE ... ADD CONSTRAINT ... UNIQUE USING INDEX, which is a brief ACCESS EXCLUSIVE.
  6. Question whether the constraint should cover all history. At roughly 40 bytes per key plus overhead, a btree over 4.2 billion rows is in the hundreds of gigabytes, while a retry never arrives a year later. A partial unique index bounded by a fixed occurred_at literal, rotated forward on a schedule, gives the same protection for a fraction of the size, and the predicate must be a literal because now() is not immutable.

Follow-up

  • Sequence the application deploy against these steps. When exactly may the code start writing uom_code, and when may it start requiring it?
  • The backfill is on day six and the conversion-factor table is itself corrected. What happens to the rows already written?
  • How do you roll back after SET NOT NULL is in place and half the traffic depends on the column?

Two scan-event queries that stopped using their indexes

Medium · indexing · partition pruning · brin correlation

observation_event is range-partitioned monthly on occurred_at, with a btree on (subject_type, subject_id, occurred_at), a BRIN on occurred_at and a GiST on geo. Rows are inserted in received_at order and events are routinely hours late. Two queries regressed: (A) WHERE occurred_at::date = current_date AND observed_status = 'delivered'; (B) a backfill scanning WHERE received_at >= now() - interval '1 day'. For each, say why no index is used and no partition is pruned, and give the fix.

Approach

  1. Query A wraps the column in a cast, so the predicate is over occurred_at::date and not over occurred_at. A btree or BRIN on the bare column cannot be probed, and the planner has no range to prune partitions with, so it plans every monthly partition. Fix it by rewriting to a half-open range: occurred_at >= :start AND occurred_at < :start + interval '1 day', with :start computed by the caller from the relevant IANA zone.
  2. Do not reach for an expression index as the first fix here. The timestamptz-to-date cast is STABLE rather than IMMUTABLE because it depends on the session TimeZone, so PostgreSQL refuses to index it; indexing ((occurred_at AT TIME ZONE 'UTC')::date) is accepted but silently pins the day boundary to UTC, which is the wrong day for any stop outside it.
  3. Query B filters a column that is neither indexed nor the partition key, so it is a full scan of every partition by construction. The subtler half is why the BRIN cannot rescue it: BRIN stores a min and max per block range and only pays when physical order correlates with the column. Rows arrive in received_at order while the indexed column is occurred_at, and late events interleave hours-old timestamps with current ones, so each range's min and max span days and almost every range qualifies.
  4. Measure the correlation rather than assuming it: SELECT attname, correlation FROM pg_stats WHERE tablename = 'observation_event'. Near 1 means BRIN earns its keep, near 0 means it is a full scan with extra steps. Expect received_at near 1 and occurred_at well below it on this insert pattern.
  5. Fix B with a btree on received_at, which stays compact because it is correlated with insert order, or better, drive the backfill from a monotonic cursor it already has such as event_id, which needs no new index on a table taking tens of thousands of inserts per second.
  6. Confirm with EXPLAIN (ANALYZE, BUFFERS) and read the plan, not the clock: check how many partitions appear, whether rows removed by filter dwarfs rows returned, and whether shared buffers read matches the size of one partition or of the table.

Worked solution (25 min)

  1. Load a few million rows across three monthly partitions with occurred_at deliberately shuffled against insert order.
  2. EXPLAIN (ANALYZE, BUFFERS) query A as written and record the partitions scanned and the buffers read.
  3. Rewrite A as a half-open range and re-run, comparing partitions scanned and buffers.
  4. Query pg_stats for the correlation of occurred_at and received_at, then add a btree on received_at and re-plan query B.

Expected result: Query A plans one partition and an index scan after the rewrite, versus every partition before. Query B moves from a full scan of all partitions to an index scan on received_at, and the measured correlation explains why the existing BRIN never helped.

Verify it yourself

  • The rewritten A returns byte-identical rows to the cast version for a day fully inside the chosen zone.
  • Buffers read for A drops by roughly the ratio of one partition to the whole table.
  • pg_stats shows occurred_at correlation well below received_at, matching the insert pattern.
  • Forcing enable_seqscan = off does not make the original A use the index, confirming the cast and not the planner is the cause.

Follow-up

  • Where would you put the partition boundary given events arrive up to a week late, and what happens to a row whose occurred_at falls in a partition that has already been detached?
  • The delivered-today query is asked per node in local time. Does that change your partitioning, your index, or only the caller?
  • Adding a btree to this table costs write amplification. How would you decide whether the backfill is worth it?

Architect a high-throughput ingestion system that ingests vehicle photos, LIDAR point clouds, and positional telemetry to render street-level map views.

Medium · system design and architecture

Approach

  1. Name the failure you are designing for, then the recovery path.
  2. Name the read and write paths separately; they rarely have the same bottleneck.
  3. State the consistency you need, and where you are willing to be stale.

Follow-up

  • What breaks first when traffic grows ten times?
  • How does this behave when that dependency is down for an hour?

Explain your strategy for debugging low-level communication protocol latencies across microservices.

Medium · system design and architecture

Approach

  1. Name the failure you are designing for, then the recovery path.
  2. Name the read and write paths separately; they rarely have the same bottleneck.
  3. Fix the scope first: who calls this, how often, and what they do when it fails.

Follow-up

  • How does this behave when that dependency is down for an hour?
  • What would you drop to keep the system up under load?

How do you design a controller for electromechanical systems with inherent time delays, and how do you prove its stability?

Medium

Approach

  1. Work from the requirement backwards to the design.
  2. Say what you would check first and why it is the highest-information step.
  3. Clarify what is being asked and what a complete answer contains.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Specify the idempotency key contract for allocation requests

Medium · idempotency · concurrency · api contract

The Fulfilment Orchestrator calls POST /allocations with order_line_id, item_id, node_id, lot_id, qty, uom_code, tier and expires_at, and retries from a durable saga step after any timeout. Specify the Idempotency-Key contract: key ownership and derivation, key scope and lifetime, what is fingerprinted, the response when a retry arrives while the first request is still in flight, and what a replay returns given that allocation rows move through held, committed, consumed, released and expired. Relate it to the partial index UNIQUE (order_line_id, node_id, lot_id) WHERE status IN (held, committed).

Approach

  1. Scope the key to (client_id, endpoint, key) and insert the key row in the same transaction as the allocation, with a unique constraint doing the work. A read-then-write check against a cache is not idempotency - it is a race with a smaller window.
  2. Fingerprint the semantically significant fields (canonical serialisation of item, node, lot, qty, uom_code, tier) and store the hash. Same key with a different fingerprint is a caller defect, not a retry: answer 422 with its own code, following the convention in the IETF Idempotency-Key header draft, and change nothing.
  3. Define the in-flight case rather than letting the database decide. A second request arriving while the first holds the key row either waits on the lock - bounded, and a 50 ms caller budget makes that unattractive - or fails fast with 409 plus Retry-After. Starting a second allocation is never an option.
  4. Make the replay return the current representation of the referenced allocation with the first call's status code, not a frozen body. A soft claim that expired between the original call and the retry must not be replayed as live. This is a deliberate departure from byte-identical replay, and the contract must name the fields the caller re-checks: status and expires_at.
  5. Size the lifetime against the business object, not a round number: the key row has to outlive the longest soft expiry plus the orchestrator's maximum retry horizon, or a late retry allocates a second claim against the same line. Retiring keys early is the failure that looks like double allocation months after ship.
  6. Place the partial unique index correctly in the story: it is the backstop that stops two concurrent held claims for one (line, node, lot), not the mechanism. It cannot stop a re-allocation after the first was released, which is exactly why the key contract exists above it.

Worked solution (30 min)

  1. Write the key table DDL: (client_id, endpoint, key) primary key, request_fingerprint, allocation_id, response_status, created_at, plus the retention rule.
  2. Write the happy-path transaction in order: insert key row, insert allocation, decrement the position under its version CAS, commit.
  3. Derive the key deterministically from persisted saga state (order_line_id plus the sourcing attempt number) and confirm a crashed process regenerates the same value instead of minting a fresh UUID.
  4. Enumerate four retry timings - during, after commit, after soft expiry, after release - and write the exact response for each.
  5. Check the fingerprint path by resending the key with qty changed and confirming nothing is written.

Expected result: A contract where the key is caller-derived from durable state, the key row is written in the allocation transaction, a mismatched fingerprint is a 422 that writes nothing, an in-flight duplicate is a 409 with Retry-After, and a replay returns the allocation's current status rather than a stale success.

Verify it yourself

  • N concurrent retries of one key produce exactly one allocation row: one call returns the create status, the other N-1 return 409 with Retry-After, and a retry sent after that call commits replays its status code and allocation_id instead of a 409.
  • A retry arriving after the soft claim expired does not present an expired allocation as held.
  • The partial unique index, dropped in a test, changes nothing about the duplicate-retry outcome - proving the key is the mechanism and the index is the backstop.

Follow-up

  • A line short-ships and is re-sourced to a second node. Does that call reuse the key, and what is the key derived from now?
  • The key store and the allocation rows are in different shards. What does the write path look like, and what have you given up?
  • What does a request carrying an expired key do, and how does the caller tell that apart from a key it never sent?

Availability p99 spikes in a sawtooth locked to the cache TTL

Medium · cache stampede · single-flight · load shedding

Available-to-promise reads sit inside checkout with a 12 ms p99 behind a 60-second per-key cache. During a promotion, p99 spikes to 900 ms in a sawtooth exactly on 60-second boundaries and is normal between spikes. The inventory database shows bursts of thousands of identical single-row position reads inside 50 ms across about 40 item-node keys, and the connection pool saturates during each burst. Request throughput is unchanged. Diagnose, fix, and state which callers may read a cached availability and which may not.

Approach

  1. Separate a stampede from saturation by the shape. Latency that rises smoothly with request rate is capacity; a sawtooth whose period equals the TTL, with normal service between peaks, is expiry-driven. Confirm by overlaying spike timestamps on key expiry times and watching the hit ratio fall to near zero for a few milliseconds each cycle.
  2. Size the herd with Little's law rather than adjectives. Concurrency equals arrival rate times service time, so a key taking 1,200 requests per second with a 30 ms origin read admits about 36 simultaneous origin reads at each expiry, and 40 keys that expire together produce roughly 1,440 against a 100-connection pool. That fourteen-fold oversubscription is the 900 ms, and the arithmetic matching is what confirms the diagnosis.
  3. Explain why the 40 keys expire in lockstep: they were all populated in the same second when promotion traffic arrived, so their TTLs are synchronised and stay synchronised. Jitter of plus or minus twenty percent on the TTL desynchronises them; it does not reduce the per-key herd.
  4. Bound the per-key herd with single-flight. One in-flight origin read per cache key, with concurrent callers awaiting the same result, takes origin load to one read per key per TTL regardless of request rate. Add serve-stale-while-revalidate so the waiters get the previous value immediately and staleness is bounded and measured rather than incidental.
  5. Draw the correctness boundary explicitly, because it is not a performance question. Browse and product pages may read a cached or stale availability. The allocation path may not: the check and the write must be one atomic step against the primary, so a cached number can inform a display but must never authorise a reservation. Checkout sits in between and should either read through with a short TTL or accept that its number is advisory and can fail at allocation.
  6. Decide the degradation direction in advance. If the origin is saturated, the read path should shed to a conservative answer, showing limited availability rather than a precise count and never rounding upward, because an optimistic stale read converts a latency incident into oversells and short-ships.

Follow-up

  • What does your single-flight do when the origin read times out: fail the waiters, or serve them the stale value, and what bounds the staleness?
  • One key gets hot enough that even one origin read per TTL per replica is too much across 200 application instances. What changes?
  • How would you invalidate a key on a movement write without opening a second stampede on the invalidation itself?

Suggested practice checklist

Four days sample coding, design, fundamentals and the practical rounds at deliberately shallow depth, which is enough to surface the topics you did not know were in scope. That map, rather than a guess made on day one, decides where the last three days go.

Day 1 — Coding, one pass at shallow depth

  • Solve one problem from each of six families, an array with two pointers, hash counting, binary search, a tree traversal, a graph traversal and one dynamic program, under a hard twenty-minute cap with no extensions, marking each finished, late, or stalled.
  • For every stall, write the exact move you could not make rather than the subject, so the note reads could not turn the recurrence into a loop rather than bad at dynamic programming.
  • Fix nothing today. The value of the pass is the unfixed record.

Deliverable: Six timed attempts marked finished, late or stalled, each stall carrying a named blocking move.

Day 2 — Design, one pass at shallow depth

  • Spend twenty minutes each on three different shapes, a read-heavy feed, a write-heavy ingest path, and something needing a transaction across two entities, stopping each at requirements, interface and data model.
  • After each, write the first question you could not answer, which is usually a number you could not estimate or a failure mode you had no vocabulary for.
  • Mark which of the three you would be most relieved not to be asked, and treat that as data rather than as a preference.

Deliverable: Three shallow designs, each with the first unanswerable question written at the bottom.

Day 3 — Fundamentals and the practical rounds

  • Answer eight short questions in writing at four minutes each, covering the material that fills the gaps between the big rounds: what happens between a URL and a rendered page, what an index costs on write, when a process is preferable to a thread, and what conditions a deadlock requires.
  • Do one thirty-minute practical task of the kind a take-home compresses: read an unfamiliar two-hundred-line file and write what it does, what you would change, and the one thing you remain unsure of.
  • Score every answer fluent, correct but slow, or absent, and keep the absent ones visible.

Deliverable: Eight scored short answers and one written reading of unfamiliar code.

Day 4 — The rounds that are about you, and the map

  • Deliver three behavioural answers aloud against a timer, a conflict, a failure you owned, and a decision made without enough information, marking any that ran past three minutes or contained no number.
  • Assemble the map: every marked item from days one to three on a single page, sorted by how likely it is to appear in your loop rather than by how uncomfortable it felt.
  • Choose exactly two areas for the remaining three days and write down what you are deliberately abandoning.

Deliverable: A one-page scored map of the whole surface area with two areas chosen and the rest explicitly abandoned.

Day 5 — First chosen area, to the depth you skipped

  • Work the higher-ranked area in four focused blocks, choosing items one level above where you stalled rather than repeating what already works.
  • After each block write the rule you extracted in one sentence with its precondition attached, since a rule carrying no precondition is exactly what fails under a variation.
  • Re-attempt the day-one or day-two item that exposed this area and compare against the original timing.

Deliverable: Four worked blocks, a timed re-attempt against the original, and three one-sentence rules with preconditions.

Day 6 — Second chosen area, where the gap is coverage rather than speed

  • Treat the second area differently from the first. Day five drilled something you could already half-do; this one is usually a topic you had simply never met, so build one worked reference example end to end and keep it, rather than attempting six problems badly.
  • Write down the vocabulary you were missing on day two or three, five terms at most, each with the one sentence that makes it usable in an answer rather than the textbook definition.
  • Redo the shallow attempt that exposed this area and note whether you now fail later in the problem, because moving the failure point is the realistic gain from a single day and is worth more than a score that did not change.

Deliverable: One worked reference example for the newly covered area, a five-term vocabulary list, and a note on where the failure point moved.

Day 7 — Reassemble the loop

  • Sit two rounds back to back with no gap, ordering them so the area you chose second comes last, because the map was built from rested, isolated attempts and the loop will reach your weaker area when you are already spent.
  • Write where the second round suffered from the first, which is normally the point at which structure collapses into narration.
  • Reduce the week to one page holding only the rules you can state without reading them.

Deliverable: Mock notes on cross-round carryover plus a one-page card of rules you can recite from memory.

Review and mentoring: what you let through

Counting review comments or mentees proves nothing. The useful version is a specific change you approved with a reservation you stated, or one you blocked and the delay that cost. Say which standard you were holding and why it was worth the friction. A mentoring story needs the thing the other person can now do without you.

How do you define integrity in software development when under pressure to ship features quickly?

Medium · behavioural and engineering judgement

Approach

  1. Give the blast radius: what could have broken, and what you measured.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?

Describe a time you had a fundamental technical disagreement with a team member or manager and how you resolved it.

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement and how you resolved it with evidence.
  3. State the situation in two sentences and spend the rest on the reasoning.

Follow-up

  • How did you know your change caused the improvement?
  • What did you decide not to do, and why?

Resolve a review disagreement over clamping a position

Easy · code review · invariants · reconciliation · disagreement

A pull request changes the pick path from a plain decrement to qty = GREATEST(qty - :n, 0) on inventory_position, because the CHECK (qty >= 0) constraint has been failing a few times a day and paging the on-call. The author argues the page is noise and a negative on-hand is meaningless anyway. You disagree. Describe a code review disagreement you had: the argument you made, how you kept it about the system rather than the person, what you conceded, and how it ended. State what the clamp destroys and what you proposed be done with the failures instead.

Approach

  1. Name the information the clamp destroys, in one sentence: the constraint failure is the only signal that the ledger and the shelf disagree, and GREATEST turns a detectable variance into a silent one that surfaces later as a picker standing at an empty location for an order the system promised.
  2. Concede the real half of the author's case immediately, because it is real: paging a human on a data condition that a human cannot fix at 3am is bad design. The disagreement is not whether the page should stop but whether stopping the page should also stop the evidence.
  3. Propose the version that satisfies both: catch the constraint violation at the application boundary, write a cycle_count_adjust movement with its own reason code and reference so the correction is a first-class row rather than an invisible edit, fail the pick task with a specific error the floor can act on, and move the alert from a page to a variance report per item-node pair.
  4. Anchor the argument in the append-only model rather than in taste. Positions are derived from inventory_movement, so a clamp makes the position stop equalling the sum of its movements and the reconciliation job now reports a permanent, unexplainable variance on that key.
  5. Say how you handled the disagreement mechanically: the comment thread, then a short synchronous conversation once it became clear the thread was circling, and a written decision so the next person reading the clamp knows why it is not there.
  6. State the concession you actually made, including if you lost. Losing a review and writing the detector anyway is a legitimate and common outcome worth describing plainly.

Follow-up

  • The clamp has already been in production for a month. How do you find out how much stock it silently absorbed?
  • Where should the adjustment movement get its reference, given that no count sheet exists at pick time?
  • The author is more senior than you and unconvinced. What is your next move?