Interview guides · BCG
BCG Software Engineer Interview Guide
BCG software engineer prep: algorithm drills where every complexity claim has to name the assumption under it.
The 60-second brief
As a Software Engineer at BCG, you are not just writing code; you are building the digital backbone for some of the world’s most complex business transformations. You will sit at the intersection of high-level strategic consulting and advanced technical execution, working within multidisciplinary teams to solve problems that traditional software houses rarely encounter.
Most rounds test how you handle an under-specified problem more than what you recall. A wrong first approach you correct out loud is recoverable; a constraint you assumed silently and were never given is the expensive mistake.
Focus areas
- Paginate large result sets with keyset cursors
- Trace a symptom to a mechanism under load
- Make every write idempotent under retry
Overview
Your work directly impacts the tools and platforms that enable BCG consultants and their clients to derive actionable insights from massive datasets. Whether you are working within the Software Engineering track or the Data Engineering channel, your contributions are critical to the firm’s ability to deliver data-driven solutions at scale. You will be expected to balance technical rigor with the agility required in a fast-paced, client-focused environment.
The BCG interview process
BCG candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Initial Project Discussions
Because the format is not fixed, the first job in the room is classification. Listen to the opening question and decide what it is: a probe into work you have already described, a fresh problem to solve now, or a conversation about how you operate. Each wants a different register, and the common failure is forcing a rehearsed structure onto a question that did not ask for it. Running a full design ritual on a ten-minute debugging question reads as not listening. When you cannot tell which it is, ask how long they want to spend and answer at that depth.
What to demonstrate
- Whether the shape of your answer matches the question, so a yes-or-no gets answered before it is justified and an open prompt gets a direction before a detour
- Whether you check how much depth is wanted instead of deciding for them, and whether you stop when the answer is complete rather than continuing until someone interrupts
- Whether you can be redirected in the middle of an answer without restarting it from the beginning
- Whether a question outside your experience gets an honest boundary followed by reasoning from what you do know, instead of a confident answer with nothing behind it
How to prepare
- Rehearse one project at three lengths, roughly thirty seconds, three minutes, and a full walkthrough at the depth of a design review, and practise switching between them when someone interrupts mid-telling
- Have someone ask you five questions of deliberately mixed type in one sitting without telling you the types, and score only whether you identified each one correctly before you started answering
- Draft the sentence you will use to check depth, along the lines of asking whether the short version is useful here or they want the detail, and use it in a real conversation this week so the day of the round is not its first outing
Technical Assessments
What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.
What to demonstrate
- Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
- Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
- Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly
How to prepare
- Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
- Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
- Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live
Cultural Fit Evaluation
The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.
What to demonstrate
- In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
- In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
- Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes
How to prepare
- Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
- Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
- For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first
Where candidates lose points
Assuming an isolation level prevents the anomaly you actually have
Isolation levels are named by the SQL standard but implemented differently, so any claim about one is only true of a named engine. PostgreSQL defaults to READ COMMITTED, where every statement takes a fresh snapshot, so two statements inside one transaction can legitimately disagree about the same row. Its REPEATABLE READ is snapshot isolation: it removes non-repeatable and phantom reads but permits write skew, where two transactions each read a set, each conclude their own write is safe, both commit, and the combined result violates a constraint that no single row expresses. Only SERIALIZABLE closes that, and it closes it by aborting a transaction with a serialization failure (SQLSTATE 40001), which means the guarantee is theoretical unless the application has a retry loop. InnoDB's REPEATABLE READ is a different mechanism again - plain SELECTs read a consistent snapshot while locking reads and writes see the latest committed row - so a read-modify-write inside one transaction can act on a value that the transaction's own earlier SELECT never returned.
Running a schema change as though the lock lasts as long as the statement
In PostgreSQL an ALTER TABLE that needs an ACCESS EXCLUSIVE lock must first wait for every open transaction touching that table, and while it waits, later queries needing a conflicting lock queue behind it rather than overtaking it. A DDL statement that would execute in milliseconds, issued while a thirty-second analytics query is open, therefore stalls all traffic on that table for thirty seconds: the outage length is set by the longest open transaction, not by the change. The defences are specific and worth knowing by name - set lock_timeout low and retry rather than queue, add columns without a volatile default so no table rewrite occurs (from version 11 a non-volatile default is a metadata-only change), build indexes with CREATE INDEX CONCURRENTLY while accepting that it cannot run inside a transaction block and leaves an invalid index behind if it fails, and add constraints as NOT VALID followed by a separate VALIDATE CONSTRAINT, which takes a weaker lock.
Saying 'eventually consistent' without naming the anomaly a user would see
Describe the concrete symptom you are choosing to accept: the author reloads and their own comment is missing for two seconds, or two devices show different balances for a minute. The class of consistency model is a technical label; the tolerable anomaly is the actual product decision.
Reading the constraints as preamble rather than as part of the problem
The bounds are usually there to eliminate the obvious approach: n up to 10^5 makes an O(n^2) scan roughly 10^10 operations, far outside any per-test time budget, and an input larger than memory rules out loading it at all. When a bound is not given, ask for it, then say out loud which approach it kills.
Questions & practice
Can you explain the space complexity implications of your chosen approach?
Medium · data structures and algorithms
Approach
- State the target complexity and say which constraint rules the naive version out.
- Name the brute-force solution and its complexity before improving on it.
- Walk one small example through your approach before writing the whole thing.
Follow-up
- How does this change if the input no longer fits in memory?
- What is the worst case, and how likely is it on real data?
Given an array containing n distinct numbers taken from 0, 1, 2,..., n, find the one that is missing from the array.
Medium · data structures and algorithms
Approach
- Walk one small example through your approach before writing the whole thing.
- State the target complexity and say which constraint rules the naive version out.
- Name the brute-force solution and its complexity before improving on it.
Follow-up
- How does this change if the input no longer fits in memory?
- What is the worst case, and how likely is it on real data?
Collapse a redelivered event batch into per-aggregate high-water marks
Easy · hashing · at-least-once · aggregation
You drain a batch of up to 5,000,000 events, each (aggregate_id BIGINT, aggregate_version INT, event_type, payload). The log guarantees order within one aggregate only; the batch merges 64 partitions, and a relay failover has redelivered a range, so an older version for an aggregate can appear after a newer one. Given a map of last_applied_version per aggregate, produce the events worth applying, at most one per (aggregate_id, version), plus the count discarded. Target O(n) time. State the memory for 2,000,000 distinct aggregates and what you do when it does not fit.
Approach
- One pass, one hash map from aggregate_id to the highest version kept, and a discard counter. An event whose version is at or below last_applied_version for its aggregate is dropped without further work, which is the whole reason the event carries its version rather than a delta. O(n) expected time, O(d) space in distinct aggregates.
- Keep the maximum, never the last occurrence. The redelivered range means the final appearance of an aggregate in the batch can be an older version than one seen earlier in the same batch, so last-wins applies stale state over newer state and the projection regresses with no error anywhere.
- Cost the memory instead of calling it large: an 8-byte key plus a 4-byte version is 12 bytes of payload, and an open-addressed table held at a 0.7 load factor costs roughly 17 bytes per entry before per-slot metadata, so 2,000,000 aggregates is tens of megabytes in a native layout and several times that in a runtime that boxes both key and value.
- If the distinct set exceeds memory, partition on hash(aggregate_id) mod P and reduce each partition independently. Every event for one aggregate hashes to the same partition, so the per-partition result is exact and the merge is concatenation rather than a second reduction.
- Reject sorting the batch by (aggregate_id, version) as the default. It is O(n log n) and buys nothing, because max is associative and commutative and needs no ordering; sorting earns its cost only when the downstream consumer must receive the events in order rather than a per-aggregate winner.
- Separate the two mechanisms out loud: in-batch deduplication does not make the consumer idempotent, because the same event redelivered tomorrow arrives in a different batch entirely. The projection write itself still has to be keyed on (aggregate_id, version).
Worked solution (20 min)
- Write the pass: look up last_applied_version, skip if the event's version is not greater, otherwise upsert into the keep-map only when the incoming version exceeds the version already held, incrementing the discard counter on every skip.
- Hand-trace one aggregate whose events arrive as v5, v3, v4, v5 with last_applied_version = 2, and confirm the output holds v5 once while the counter reads 3.
- Compute the table footprint for 2,000,000 entries at 12 bytes of payload and a 0.7 load factor, then state the multiplier for a runtime that boxes keys and values.
- Add the hash-partitioning fallback and say in one sentence why the per-partition results need no cross-partition merge logic.
Expected result: A single-pass O(n) reduction keyed on aggregate_id that keeps the maximum version rather than the last occurrence, O(d) space with the byte cost stated for 2,000,000 aggregates, a hash-partitioning fallback keyed on aggregate_id, and an explicit statement that batch-local deduplication does not replace a projection write keyed on (aggregate_id, version).
Verify it yourself
- Shuffling the batch order changes nothing in the output, which is the test that separates max-wins from last-wins.
- Feeding the same batch twice produces an identical keep-set and only doubles the discard counter.
- Every emitted event has a version strictly greater than the supplied last_applied_version for its aggregate.
Follow-up
- The payload is a patch rather than a snapshot, so applying only the highest version loses the intermediate changes. What changes in your reduction?
- How do you detect that version 7 arrived while version 6 was never delivered, and what should the consumer do about the gap?
- Two events for one aggregate carry the same version with different payloads. Which one is wrong, and how would you find out?
Write the update path that detects a concurrent edit
Medium · optimistic concurrency · transactional outbox · audit history
resource carries version INT NOT NULL DEFAULT 1. resource_revision holds revision_id, resource_id, version, actor_user_id, change_kind, patch JSONB, request_id, created_at with UNIQUE (resource_id, version). outbox_event holds aggregate_type, aggregate_id, aggregate_version, event_type, payload, status. A PUT carries the version the client read. Write the exact statements for the single transaction that applies the edit, records the revision and enqueues 'resource.updated', and give the handler's branch on zero affected rows. Then say what PostgreSQL 16 does under READ COMMITTED when two of these updates hit one row at once.
Approach
- One transaction, three writes, no network call inside it: UPDATE resource SET title = $3, version = version + 1, updated_at = now() WHERE resource_id = $1 AND tenant_id = $4 AND version = $2; then INSERT the resource_revision row at version $2 + 1; then INSERT the outbox_event row at the same aggregate_version. The event goes to a table rather than a broker because no transaction spans both.
- Branch on the affected-row count before doing anything else. Zero has three causes — stale version, wrong tenant, row gone — so re-read once and map to 409 carrying the current version, or 404 for an id outside the caller's tenant, which also stops the endpoint confirming that another tenant's id exists.
- State the engine behaviour instead of assuming it. Under READ COMMITTED the second UPDATE blocks on the row lock, and when the first commits PostgreSQL re-evaluates the WHERE clause against the newly committed row, so the version predicate now fails and the statement reports zero rows. Under REPEATABLE READ the identical collision raises SQLSTATE 40001 instead, so the handler must fold both shapes into one conflict response.
- Keep UNIQUE (resource_id, version) even though the predicate already serialises writers. It is what makes a lost update unwritable if any other path ever reaches the revision table, and it converts a logic bug into 23505 rather than into a silently missing history row.
- Refuse to auto-retry the whole PUT. A retry re-reads the winner's state and reapplies an intent formed against data that no longer exists — the silent overwrite the version token was added to detect. Return the conflict; merge field-wise only if the patches are provably disjoint.
- Note that now() is the transaction timestamp in PostgreSQL, so resource.updated_at, the revision's created_at and the outbox row share one instant, which is what later makes reconciliation between the three tables unambiguous.
Follow-up
- A client sends the version it read ten minutes ago and the resource has moved three versions. What is in your 409 so it can resolve the conflict without a full re-fetch?
- Two editors, two disjoint fields, no overlap. Does your answer still refuse the second write, and should it?
- Every write now touches a second hot table. How do you keep the outbox insert and its partial index from becoming the write bottleneck at 1.2k writes/second?
Denormalise tenant onto revisions and backfill it live
Hard · denormalisation · online migration · backfill
resource_revision (revision_id, resource_id, version, actor_user_id, change_kind, patch, request_id, created_at) has 400M rows and no tenant column; tenant_id lives only on resource. Two reads need it: a tenant-scoped audit feed ordered by created_at DESC, and an offboarding purge. Both join back to resource today. Justify adding tenant_id to resource_revision against those two reads, name the anomaly the copy introduces and the constraint that prevents it, then give the ordered migration for a live table taking 1.2k writes/second — the lock each step takes, how the backfill is batched, and where each step stops being reversible. PostgreSQL 16.
Approach
- Justify from the access path rather than from taste. Without the column, the audit feed either scans resource_revision by created_at and discards other tenants' rows, or resolves the tenant's resource_ids first and probes with them — both proportional to the tenant's whole history rather than to one page. With (tenant_id, created_at DESC, revision_id DESC) it is a seek that stops at 50 rows, and the purge becomes a ranged delete instead of a join.
- Name the cost exactly: a second copy of a fact can disagree with the first. Make the disagreement unwritable rather than documented — add UNIQUE (resource_id, tenant_id) on resource so it can serve as a foreign-key target, then FOREIGN KEY (resource_id, tenant_id) REFERENCES resource (resource_id, tenant_id) on the revision table. A revision can then only ever carry its parent's tenant.
- Step one, expand: ALTER TABLE resource_revision ADD COLUMN tenant_id BIGINT NULL, with no default, so it is a catalogue change and no rewrite. It still needs ACCESS EXCLUSIVE for an instant, and that instant queues behind the longest open transaction on the table while every later query queues behind it — set lock_timeout to 2s and retry rather than wait.
- Step two, dual-write: deploy the writer that populates tenant_id on every new revision while reads still use the join. Reversible by redeploying the previous build, because nothing reads the column yet.
- Step three, backfill: batch by primary key rather than by created_at so the cursor is dense and resumable — UPDATE resource_revision rr SET tenant_id = r.tenant_id FROM resource r WHERE r.resource_id = rr.resource_id AND rr.revision_id > $1 AND rr.revision_id <= $1 + 5000 AND rr.tenant_id IS NULL — committing per batch and persisting the cursor. Throttle on replica replay lag and on dead-tuple count, since each batch writes 5,000 new row versions. Run the backfill before the index exists so those updates can stay HOT.
- Step four, index then enforce then contract: CREATE INDEX CONCURRENTLY (cannot run inside a transaction block, scans the table twice, waits on open transactions, and leaves an INVALID index to drop concurrently if it fails); ADD CONSTRAINT ... CHECK (tenant_id IS NOT NULL) NOT VALID, then VALIDATE CONSTRAINT, which takes only SHARE UPDATE EXCLUSIVE, after which SET NOT NULL uses the validated check instead of re-scanning on PostgreSQL 12 and later. Only then move the audit reads onto the column and, in a later deploy, delete the join path.
Worked solution (40 min)
- Write the five steps as separate scripts and state, for each, the lock mode it acquires and the deploy it pairs with.
- On a 20M-row copy, run the ADD COLUMN while a 30-second transaction holds a lock on the table, and record how long unrelated queries queue behind it.
- Run the batched backfill at 5,000 rows, kill it mid-run, restart from the persisted cursor, and confirm no row is processed twice and none is skipped.
- Build the index concurrently under concurrent write load, then add the CHECK ... NOT VALID, VALIDATE it and SET NOT NULL, timing each.
- Compare the audit-feed plan before and after: join-and-filter versus an index seek with no Sort.
Expected result: ADD COLUMN completes in milliseconds once it holds the lock, but with a 30-second transaction open it stalls all traffic on that table for the full 30 seconds unless lock_timeout cuts it off. The backfill resumes exactly at the persisted revision_id. After VALIDATE, SET NOT NULL returns without a table scan. The audit feed drops from scanning the tenant's history to reading about 50 rows.
Verify it yourself
- SELECT count(*) FROM resource_revision WHERE tenant_id IS NULL returns 0, re-checked after the dual-writer has been live for a full deploy cycle.
- A sampled join confirms rr.tenant_id = r.tenant_id across 100k random revisions; the composite foreign key makes a mismatch unwritable from then on.
- pg_stat_activity shows no transaction older than one batch's duration while the backfill runs.
- EXPLAIN of the audit feed shows an Index Scan with no Sort node.
Follow-up
- The backfill is half finished and a rollback is required. What state is the table in, and what does the previous build do with a half-populated column?
- How do you verify the backfill actually finished, given rows are still being inserted while it runs?
- A resource must now be movable between tenants. What does that do to the composite foreign key and to the revisions already written?
What were the primary technical challenges you faced during [Project Name]?
Medium
Approach
- State your assumptions explicitly before working the problem.
- Work from the requirement backwards to the design.
- Say what you would check first and why it is the highest-information step.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
How did you ensure the scalability of your solution?
Medium
Approach
- Work from the requirement backwards to the design.
- Say what you would check first and why it is the highest-information step.
- State your assumptions explicitly before working the problem.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Walk me through the architecture of your most significant project.
Medium
Approach
- Clarify what is being asked and what a complete answer contains.
- State your assumptions explicitly before working the problem.
- Say what you would check first and why it is the highest-information step.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Relay committed events to the log without gaps or reordering
Medium · transactional outbox · at-least-once · ordering · leases
outbox_event rows are written in the same transaction as the state change and carry aggregate_type, aggregate_id, aggregate_version, payload and status, with a partial index on (created_at, event_id) WHERE status = 'pending'. The relay publishes about 4k events/second to a partitioned append-only log keyed by aggregate_id, with one leader per partition range holding a lease. Consumers must never miss an event; they may see one twice. Design the claim-publish-mark loop, and state exactly what consumers observe when a leader's lease expires while it is mid-batch.
Approach
- Claim with SELECT ... WHERE status='pending' ORDER BY created_at, event_id LIMIT $batch FOR UPDATE SKIP LOCKED inside a transaction. SKIP LOCKED lets several relay workers share a range without serialising on each other's rows, and the partial index keeps the claim proportional to the backlog rather than to a table that is overwhelmingly published rows. At 4k/second a batch of 500 is eight claims per second, each an index scan of 500 entries.
- Publish before marking, never the reverse, and say why it is a choice. Marking first loses the event outright if the process dies in the gap, and the loss is silent - nothing remains to retry, and it surfaces later as a projection missing a row. Publishing first can repeat the event, and repetition is what every consumer is already built to survive. That single ordering is the whole at-least-once guarantee.
- Preserve the only ordering on offer. Partition by aggregate_id and never publish two events for one aggregate concurrently: claim in (created_at, event_id) order and publish sequentially within an aggregate, or hash aggregate_id to a worker slot. Order across aggregates is not available at any price here, which is why the event carries aggregate_version and the full fact rather than a delta - a consumer can then discard what it has already applied without coordinating with anyone.
- State the failover behaviour precisely, because it is the consistency-versus-availability decision in this design. A lease expires because the holder is slow, and no mechanism distinguishes that from dead, so for the length of the lease window two leaders can publish the same claimed batch. The system accepts duplicates to avoid stalling publication for every aggregate in the range whenever one worker pauses. Consumers deduplicate on (aggregate_id, aggregate_version) and drop anything at or below what they have applied.
- Bound the failure paths and pick the right alarm. A row that fails to publish increments attempts, records last_error, and moves to 'dead' after a limit so one poison payload cannot block the backlog behind it. Alert on the age of the oldest pending row, not on the relay's error rate: the failure worth catching is a relay reporting itself healthy while nothing is being published.
Worked solution (25 min)
- Write the claim statement and check it against the partial index: which columns it seeks on, how many entries it touches, and what two concurrent workers do to each other.
- Write both orderings of publish and mark, and for each state what exists after a crash at every point in the loop.
- Write the consumer's dedupe rule on (aggregate_id, aggregate_version) and test it against a replayed batch of 500.
- Compute the backlog after a 40-minute outage and the batch rate needed to drain it while 4k/second continues to arrive.
Expected result: A claim using FOR UPDATE SKIP LOCKED over the partial index in (created_at, event_id) order, publish to the aggregate_id partition, then mark published - in that order, accepting duplicates on lease expiry. Per-aggregate order is preserved by never publishing an aggregate concurrently; cross-aggregate order is explicitly not guaranteed. Failures increment attempts and terminate at 'dead'. The alarm is the age of the oldest pending row.
Verify it yourself
- Killing the relay mid-batch produces duplicates on restart and no gap.
- The claim query's plan uses the partial index and reads at most batch-size entries regardless of table size.
- Two relay workers on the same range never block on each other's rows.
- Events for one aggregate_id arrive in aggregate_version order, and no consumer relies on order across aggregates.
Follow-up
- The relay is down 40 minutes and 9.6 million rows are pending. What does catch-up do to the primary, and what changes in the claim loop to survive it?
- A consumer insists it never received an event. Which single query settles whether the relay lost it, and what does each answer look like?
- Delivery is at-least-once. What would exactly-once require end to end, and why is that a property of the consumer rather than of the relay?
Exports duplicate a row range about once a week
Hard · leases · clock skew · idempotency · fencing
Roughly once a week an export writes a file containing a duplicated range of rows. The affected job_run rows show attempt = 1, status = succeeded, one started_at, and a lease_owner naming a different host from the one whose logs show the job starting. Leases last 30 seconds and are heartbeated every 10 from inside the handler; lease_expires_at is computed on the worker and compared against the database's now(). Find the mechanism, and give a fix that holds even if you cannot fix the clocks.
Approach
- Start from the fact that eliminates the obvious answer. attempt = 1 means no retry was recorded, so this is not a re-run after failure; two workers ran the same row concurrently and the takeover path never touched the counter. lease_owner naming a host other than the one that started the job is the same statement from the other side.
- Enumerate the mechanisms that cause a premature takeover, then find the signal that separates them. Either the lease genuinely expired because the heartbeat did not fire, which is what happens when the heartbeat runs on the handler's own thread and the handler makes a long blocking call, or it only appeared expired because two clocks disagree, since lease_expires_at is written from the worker's clock and evaluated against the database's. The discriminator is the distribution: incidents clustered on the longest exports indict the heartbeat, incidents clustered on one host indict skew. Measure both, and measure each host's offset against the database directly.
- Read the reclaim query precisely. In PostgreSQL now() is transaction start time, not statement time, so a reclaimer holding a long transaction compares against an older timestamp than expected; clock_timestamp() is the statement-time function. This is worth ruling in or out before you redesign anything, because it changes which rows look expired.
- Remove the second clock rather than trying to synchronise it. Issue and extend the lease in the database, with lease_expires_at = now() + interval '30 seconds' in both the claim and the heartbeat, so exactly one clock is ever compared and worker skew stops mattering to this predicate.
- Accept that a lease can still expire under a slow worker, because a lease cannot distinguish slow from dead, and fence the work. Carry a monotonically increasing lease generation and make every write the handler performs conditional on still holding it, as UPDATE ... WHERE job_run_id = $1 AND lease_owner = $2 AND lease_generation = $3, so a displaced worker's writes affect zero rows and it aborts instead of duplicating.
- Make the handler's writes idempotent independently of all that: give each exported chunk a natural key of (job_run_id, batch_start) with a unique constraint so a second copy conflicts rather than appends, move the heartbeat off the handler's thread, and increment attempt on takeover so the event is visible in a metric.
Follow-up
- The displaced worker has already streamed half the file to object storage. What makes that side effect safe to repeat?
- You now count takeovers. What alert fires on that counter, and at what threshold?
- What breaks if you simply raise the lease to five minutes?
Suggested practice checklist
For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.
Day 1 — Numbers before diagrams
- Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
- Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
- Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.
Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.
Day 2 — One system, from requirements to schema
- Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
- Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
- Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.
Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.
Day 3 — The consistency you are actually buying
- Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
- Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
- Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.
Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.
Day 4 — Failure is the design
- For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
- Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
- Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
- Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.
Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.
Day 5 — Scaling the hot path
- Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
- Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
- Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
- Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.
Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.
Day 6 — Keep the coding hand in, at the bar that applies to you
- Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
- Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
- Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.
Deliverable: One problem hardened to review standard, with its test list and one written review comment.
Day 7 — Defend it while being interrupted
- Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
- Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
- Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.
Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.
The estimate and the day you knew
A slipped date is only a bad story if you sat on it. What matters is what you believed when you gave the number, the signal that told you it was wrong, how many days passed before you said so, and what you cut rather than asking for more time. Scope you defended counts as much as scope you dropped.
Reverse your own decision and price the reversal
Medium · reversibility · measurement · migrations
Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.
Approach
- State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
- Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
- Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
- Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.
- Finish on the process change: the smallest experiment that would have produced the same measurement in a day, and why you did not run it the first time.
Follow-up
- What in that decision was irreversible, and did you know it was irreversible when you made it?
- How did you tell the people who had already built on top of the original decision?
- What do you now measure before committing to a change of this size?
Estimate work you have never done and defend the range
Hard · estimation · backfills · expand-contract
You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.
Approach
- Decompose into independently deployable steps before estimating anything: add the column nullable, write both shapes, backfill in batches, verify, move reads, stop writing the old shape, drop it. That is four deploys spread over days, and the calendar estimate is dominated by them rather than by the loop's runtime.
- Do the arithmetic aloud for the part that has arithmetic in it: batch size times number of batches times per-batch duration, at a write rate the primary can absorb alongside roughly 1.2k writes per second of production traffic. The loop is throttled by replication lag and lock waits, not by how fast it can issue statements.
- Price the schema step by its lock rather than its statement duration. In PostgreSQL an ALTER TABLE taking ACCESS EXCLUSIVE waits for every open transaction on that table while later queries queue behind it, so a millisecond change issued during a thirty-second analytics query stalls that table for thirty seconds. Adding a nullable column with a non-volatile default avoids a rewrite from version 11; a new index wants CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an invalid index behind if it fails.
- Express the answer as a range whose endpoints each trace to a stated assumption, then name the cheapest experiment that collapses it, which is almost always running one real batch against the real table and multiplying.
- Commit to a checkpoint rather than a completion date: the day you report a measured number from that first batch. That is a promise you can keep under uncertainty, and it is what the asker actually needs in order to plan.
Follow-up
- How do you verify the backfill genuinely finished, given rows written by production traffic while it ran?
- Where does the backfill resume from after a worker is killed mid-batch, and what makes that resume point trustworthy?
- Your first batch comes back ten times slower than assumed. What do you tell the person waiting on the estimate, and when?
Turn a code review disagreement into a decision
Easy · code review · optimistic concurrency · communication
A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.
Approach
- Sort the disagreement before writing anything. A silently discarded write is a correctness claim about data; the choice between 409 and 412 is taste. Only the first justifies blocking a merge, and saying which one you are doing is most of the value of the comment.
- Make the claim reproducible in the comment itself with an interleaving rather than a principle: A reads version 7, B reads version 7, B commits version 8, A's predicate matches zero rows, A is told it succeeded and A's edit is gone.
- Offer the alternative with its cost attached: return 409 carrying the current version and the revision that won, so the client can re-read and re-apply. Note that automatic retry is not the fix, because a retry re-reads the winner's state and reapplies an intent formed against data that no longer exists.
- Apply an escalation rule you can state: two round trips on the thread, then a call, and the service's owner decides rather than the reviewer. A reviewer who cannot be overruled is a bottleneck with extra steps.
- Close in writing wherever the decision lands, so the next reader finds the reasoning in the code or the ticket instead of in a collapsed review thread.
Follow-up
- Where would you put the test that fails if someone reintroduces the swallowed zero rowcount?
- The author says clients cannot handle a 409. How do you check whether that is true?
- How do you handle the same review comment when the author is more senior than you and in a hurry?