Interview guides · Slalom
Slalom Machine Learning Engineer Interview Questions & Guide 2026
Slalom Machine Learning Engineer interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.
The 60-second brief
A Machine Learning Engineer at Slalom acts as a bridge between complex data science models and scalable, production-ready software solutions. In this role, you are not just building models; you are architecting the end-to-end pipelines that allow Slalom clients to derive real-world value from their data. You will operate at the intersection of software engineering and data science, ensuring that AI/ML initiatives are robust, maintainable, and aligned with strategic business goals.
Browser-facing seats are not covered by algorithm practice. Scope in state ownership, what the page does on a slow or failed request, and how you would diagnose something that renders correctly but feels slow.
Focus areas
- Paginate large result sets with keyset cursors
- Detect concurrent edits instead of losing writes
- Trace a symptom to a mechanism under load
Overview
This position is critical because Slalom prides itself on delivering human-centric, high-impact solutions. You will work within diverse, cross-functional teams to solve sophisticated business problems—ranging from automating operational workflows to building predictive analytics engines. The work is fast-paced, intellectually demanding, and requires a high degree of adaptability, as you will likely pivot between different industry verticals and client technology stacks.
Successful candidates at Slalom demonstrate a blend of technical depth and the consultative mindset required to translate complex ML concepts into clear, actionable business strategies for non-technical stakeholders.
The Slalom interview process
Slalom candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Recruiter Screen
Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.
What to demonstrate
- Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
- Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
- Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not
How to prepare
- Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
- Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
- If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window
Technical Deep Dives
Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.
What to demonstrate
- Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
- Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
- Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
- Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply
How to prepare
- For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
- For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
- Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last
Behavioral Interviews
Many of these questions are about something that went wrong, and the grading sits mostly in the hours after you knew. Who found out first, whether that was you or an alert or a user, how long it took you to say it out loud, and whether the people who needed the news got it while they could still act on it. Engineers under-tell this part because it feels like confessing. The pattern it is looking for is the opposite: the quiet fix, an incident absorbed without telling anyone, after which nothing changed and the same failure is still available.
What to demonstrate
- How the problem was found, and whether that route was one you had built or one that happened to you, since a user reporting it first means your instrumentation did not cover that failure
- Whether time-to-detect and time-to-tell are separate numbers in your account and whether you know both, because a fast fix that nobody heard about until the retro is a different answer from a slow one that was announced immediately
- Whether the resolution left something durable behind, a check that fires or a default that changed, rather than depending on people remembering to be careful
- Whether you can say what the failure cost without either inflating it or waving it away
How to prepare
- Reconstruct one incident you were part of as a timeline with clock times: first bad request, first signal, first person who knew, first message outside the team, mitigation, permanent fix. The gaps between those entries are what gets asked about
- Look up the configuration of the signal that caught it, including its evaluation window and threshold. An alert defined on a five-minute aggregate cannot fire until the condition holds across that window, which puts a floor under time-to-detect that has nothing to do with how severe the failure was. Be able to say what that floor was and whether anyone had chosen it deliberately
- Prepare one story where you escalated early and the severity turned out to be smaller than you thought, including what it cost the people you pulled in. Without it, every answer you give about raising alarms is unfalsifiable
Leadership Discussion
You cannot drill a format you do not know, so put the preparation into material that travels. Three pieces of your own work, each rehearsed until you can take a follow-up you did not anticipate, will carry a conversation or a code walkthrough equally well. Specificity is what separates that from filler. A number needs its definition before it means anything: a p99 is over some window and measured at some hop, and a server-side figure excludes the queueing and network time a client would see. The number you cannot qualify is the one to leave out.
What to demonstrate
- Whether your examples carry detail only someone who did the work would hold, such as what the binding constraint actually was, which alternative you rejected and why it was worse, and what you measured on each side of the change
- Whether a number survives one follow-up, meaning you can say what it was measured over and whether it moved because of your change or merely alongside it
- Whether a failure is described with the specific change that followed it, rather than a lesson stated in general terms
- Whether your part in a team effort is stated accurately, including what other people did
How to prepare
- Write a page on each of three projects covering the constraint, the option you rejected, the measurement before and after, and what went wrong. Cut any line you cannot take a follow-up on, since you are writing the parts you will be pressed on rather than a summary.
- Recover the real figures while you still have access: request volume, data size, latency with its percentile and window, team size, timeline. Note where each came from, whether a dashboard, a design document or memory, and mark the estimates so you can say which they are out loud.
- Take your weakest project story to someone who works in a different area and have them ask why four times in succession. The point where you run out of answer is the part to go and re-read before the round.
Where candidates lose points
Letting a slow dependency consume unbounded concurrency
The failure that takes a service down is usually not an error but a delay. A dependency answering in thirty seconds instead of fifty milliseconds holds each request's worker or connection six hundred times longer, and since required concurrency is arrival rate times latency, a fleet sized for sixty in-flight requests now needs thirty-six thousand to sustain the same rate - so it queues, and requests whose clients have already abandoned them still occupy resources. Retries make it precisely worse: a policy of three attempts triples the load on a dependency at the exact moment it is least able to serve, which is how one slow dependency becomes an outage of everything sharing that pool. Containment is four specific things - a timeout on every outbound call shorter than the caller's remaining budget, a bounded pool per dependency so one cannot starve the others, backoff with full jitter rather than a fixed delay so retries do not resynchronise, and a circuit that stops sending once the failure rate makes an attempt pointless.
Choosing an index from the columns a query mentions rather than from how it filters and orders
A composite B-tree index on (a, b, c) can be seeked only as a left prefix: equality on a, then equality on b, then a range or an ordering on c. A query that filters on b alone cannot seek into it at all and at best gets a full scan of the index; a query that filters a and ranges on b gets no benefit from c, because the index is only sorted by c within a fixed (a, b) pair. The practical consequence is that one index per column is close to useless for multi-predicate queries while a single correctly ordered composite index turns a scan into a lookup. The ordering half is what gets missed: if the index cannot satisfy the ORDER BY, the database must read every matching row and sort before the limit can apply, so a LIMIT 20 over a million matching rows still reads a million rows.
Sharing mutable state with no stated owner
Say which thread, request or task owns each mutable structure, and what protects it when the answer is more than one: a lock, a queue that hands ownership across, or an immutable copy per reader. A structure documented as safe for concurrent reads is usually not safe for a concurrent write alongside those reads.
Assuming the input fits in memory
Ask how large the input is in bytes before committing to an in-memory algorithm; beyond that point the options are a single streaming pass, an external sort with bounded buffers, or a sketch that trades exactness for constant memory. An algorithm that assumes random access to the whole input is a different algorithm from one that sees each element once.
Questions & practice
If a model is performing well in development but struggling in production, what is your diagnostic process?
Medium · machine learning fundamentals
Approach
- Name the simplest model that could work and what would make you move past it.
- State the learning problem: the label, the unit of prediction and how the model is used.
- Pick the metric from the cost of each error type, not from habit.
Follow-up
- How would you know the model is overfitting?
- What changes if the classes are heavily imbalanced?
Describe the most challenging model deployment you have managed and the tools used.
Medium · machine learning fundamentals
Approach
- Name the simplest model that could work and what would make you move past it.
- Say how you would validate it, and where leakage could enter the split.
- Pick the metric from the cost of each error type, not from habit.
Follow-up
- Where could label leakage enter this setup?
- What changes if the classes are heavily imbalanced?
How would you approach designing a recommendation system from scratch for a retail client?
Medium · coding and algorithms
Approach
- Choose the data structure from the access pattern, not from familiarity.
- State the target complexity and say which constraint rules the naive version out.
- Restate the input: its shape, its size, and what is guaranteed about it.
Follow-up
- Which test case would catch an off-by-one here?
- What is the worst case, and how likely is it on real data?
Collapse a redelivered event batch into per-aggregate high-water marks
Easy · hashing · at-least-once · aggregation
You drain a batch of up to 5,000,000 events, each (aggregate_id BIGINT, aggregate_version INT, event_type, payload). The log guarantees order within one aggregate only; the batch merges 64 partitions, and a relay failover has redelivered a range, so an older version for an aggregate can appear after a newer one. Given a map of last_applied_version per aggregate, produce the events worth applying, at most one per (aggregate_id, version), plus the count discarded. Target O(n) time. State the memory for 2,000,000 distinct aggregates and what you do when it does not fit.
Approach
- One pass, one hash map from aggregate_id to the highest version kept, and a discard counter. An event whose version is at or below last_applied_version for its aggregate is dropped without further work, which is the whole reason the event carries its version rather than a delta. O(n) expected time, O(d) space in distinct aggregates.
- Keep the maximum, never the last occurrence. The redelivered range means the final appearance of an aggregate in the batch can be an older version than one seen earlier in the same batch, so last-wins applies stale state over newer state and the projection regresses with no error anywhere.
- Cost the memory instead of calling it large: an 8-byte key plus a 4-byte version is 12 bytes of payload, and an open-addressed table held at a 0.7 load factor costs roughly 17 bytes per entry before per-slot metadata, so 2,000,000 aggregates is tens of megabytes in a native layout and several times that in a runtime that boxes both key and value.
- If the distinct set exceeds memory, partition on hash(aggregate_id) mod P and reduce each partition independently. Every event for one aggregate hashes to the same partition, so the per-partition result is exact and the merge is concatenation rather than a second reduction.
- Reject sorting the batch by (aggregate_id, version) as the default. It is O(n log n) and buys nothing, because max is associative and commutative and needs no ordering; sorting earns its cost only when the downstream consumer must receive the events in order rather than a per-aggregate winner.
- Separate the two mechanisms out loud: in-batch deduplication does not make the consumer idempotent, because the same event redelivered tomorrow arrives in a different batch entirely. The projection write itself still has to be keyed on (aggregate_id, version).
Worked solution (20 min)
- Write the pass: look up last_applied_version, skip if the event's version is not greater, otherwise upsert into the keep-map only when the incoming version exceeds the version already held, incrementing the discard counter on every skip.
- Hand-trace one aggregate whose events arrive as v5, v3, v4, v5 with last_applied_version = 2, and confirm the output holds v5 once while the counter reads 3.
- Compute the table footprint for 2,000,000 entries at 12 bytes of payload and a 0.7 load factor, then state the multiplier for a runtime that boxes keys and values.
- Add the hash-partitioning fallback and say in one sentence why the per-partition results need no cross-partition merge logic.
Expected result: A single-pass O(n) reduction keyed on aggregate_id that keeps the maximum version rather than the last occurrence, O(d) space with the byte cost stated for 2,000,000 aggregates, a hash-partitioning fallback keyed on aggregate_id, and an explicit statement that batch-local deduplication does not replace a projection write keyed on (aggregate_id, version).
Verify it yourself
- Shuffling the batch order changes nothing in the output, which is the test that separates max-wins from last-wins.
- Feeding the same batch twice produces an identical keep-set and only doubles the discard counter.
- Every emitted event has a version strictly greater than the supplied last_applied_version for its aggregate.
Follow-up
- The payload is a patch rather than a snapshot, so applying only the highest version loses the intermediate changes. What changes in your reduction?
- How do you detect that version 7 arrived while version 6 was never delivered, and what should the consumer do about the gap?
- Two events for one aggregate carry the same version with different payloads. Which one is wrong, and how would you find out?
Denormalise tenant onto revisions and backfill it live
Hard · denormalisation · online migration · backfill
resource_revision (revision_id, resource_id, version, actor_user_id, change_kind, patch, request_id, created_at) has 400M rows and no tenant column; tenant_id lives only on resource. Two reads need it: a tenant-scoped audit feed ordered by created_at DESC, and an offboarding purge. Both join back to resource today. Justify adding tenant_id to resource_revision against those two reads, name the anomaly the copy introduces and the constraint that prevents it, then give the ordered migration for a live table taking 1.2k writes/second — the lock each step takes, how the backfill is batched, and where each step stops being reversible. PostgreSQL 16.
Approach
- Justify from the access path rather than from taste. Without the column, the audit feed either scans resource_revision by created_at and discards other tenants' rows, or resolves the tenant's resource_ids first and probes with them — both proportional to the tenant's whole history rather than to one page. With (tenant_id, created_at DESC, revision_id DESC) it is a seek that stops at 50 rows, and the purge becomes a ranged delete instead of a join.
- Name the cost exactly: a second copy of a fact can disagree with the first. Make the disagreement unwritable rather than documented — add UNIQUE (resource_id, tenant_id) on resource so it can serve as a foreign-key target, then FOREIGN KEY (resource_id, tenant_id) REFERENCES resource (resource_id, tenant_id) on the revision table. A revision can then only ever carry its parent's tenant.
- Step one, expand: ALTER TABLE resource_revision ADD COLUMN tenant_id BIGINT NULL, with no default, so it is a catalogue change and no rewrite. It still needs ACCESS EXCLUSIVE for an instant, and that instant queues behind the longest open transaction on the table while every later query queues behind it — set lock_timeout to 2s and retry rather than wait.
- Step two, dual-write: deploy the writer that populates tenant_id on every new revision while reads still use the join. Reversible by redeploying the previous build, because nothing reads the column yet.
- Step three, backfill: batch by primary key rather than by created_at so the cursor is dense and resumable — UPDATE resource_revision rr SET tenant_id = r.tenant_id FROM resource r WHERE r.resource_id = rr.resource_id AND rr.revision_id > $1 AND rr.revision_id <= $1 + 5000 AND rr.tenant_id IS NULL — committing per batch and persisting the cursor. Throttle on replica replay lag and on dead-tuple count, since each batch writes 5,000 new row versions. Run the backfill before the index exists so those updates can stay HOT.
- Step four, index then enforce then contract: CREATE INDEX CONCURRENTLY (cannot run inside a transaction block, scans the table twice, waits on open transactions, and leaves an INVALID index to drop concurrently if it fails); ADD CONSTRAINT ... CHECK (tenant_id IS NOT NULL) NOT VALID, then VALIDATE CONSTRAINT, which takes only SHARE UPDATE EXCLUSIVE, after which SET NOT NULL uses the validated check instead of re-scanning on PostgreSQL 12 and later. Only then move the audit reads onto the column and, in a later deploy, delete the join path.
Worked solution (40 min)
- Write the five steps as separate scripts and state, for each, the lock mode it acquires and the deploy it pairs with.
- On a 20M-row copy, run the ADD COLUMN while a 30-second transaction holds a lock on the table, and record how long unrelated queries queue behind it.
- Run the batched backfill at 5,000 rows, kill it mid-run, restart from the persisted cursor, and confirm no row is processed twice and none is skipped.
- Build the index concurrently under concurrent write load, then add the CHECK ... NOT VALID, VALIDATE it and SET NOT NULL, timing each.
- Compare the audit-feed plan before and after: join-and-filter versus an index seek with no Sort.
Expected result: ADD COLUMN completes in milliseconds once it holds the lock, but with a 30-second transaction open it stalls all traffic on that table for the full 30 seconds unless lock_timeout cuts it off. The backfill resumes exactly at the persisted revision_id. After VALIDATE, SET NOT NULL returns without a table scan. The audit feed drops from scanning the tenant's history to reading about 50 rows.
Verify it yourself
- SELECT count(*) FROM resource_revision WHERE tenant_id IS NULL returns 0, re-checked after the dual-writer has been live for a full deploy cycle.
- A sampled join confirms rr.tenant_id = r.tenant_id across 100k random revisions; the composite foreign key makes a mismatch unwritable from then on.
- pg_stat_activity shows no transaction older than one batch's duration while the backfill runs.
- EXPLAIN of the audit feed shows an Index Scan with no Sort node.
Follow-up
- The backfill is half finished and a rollback is required. What state is the table in, and what does the previous build do with a half-populated column?
- How do you verify the backfill actually finished, given rows are still being inserted while it runs?
- A resource must now be movable between tenants. What does that do to the composite foreign key and to the revisions already written?
Write the update path that detects a concurrent edit
Medium · optimistic concurrency · transactional outbox · audit history
resource carries version INT NOT NULL DEFAULT 1. resource_revision holds revision_id, resource_id, version, actor_user_id, change_kind, patch JSONB, request_id, created_at with UNIQUE (resource_id, version). outbox_event holds aggregate_type, aggregate_id, aggregate_version, event_type, payload, status. A PUT carries the version the client read. Write the exact statements for the single transaction that applies the edit, records the revision and enqueues 'resource.updated', and give the handler's branch on zero affected rows. Then say what PostgreSQL 16 does under READ COMMITTED when two of these updates hit one row at once.
Approach
- One transaction, three writes, no network call inside it: UPDATE resource SET title = $3, version = version + 1, updated_at = now() WHERE resource_id = $1 AND tenant_id = $4 AND version = $2; then INSERT the resource_revision row at version $2 + 1; then INSERT the outbox_event row at the same aggregate_version. The event goes to a table rather than a broker because no transaction spans both.
- Branch on the affected-row count before doing anything else. Zero has three causes — stale version, wrong tenant, row gone — so re-read once and map to 409 carrying the current version, or 404 for an id outside the caller's tenant, which also stops the endpoint confirming that another tenant's id exists.
- State the engine behaviour instead of assuming it. Under READ COMMITTED the second UPDATE blocks on the row lock, and when the first commits PostgreSQL re-evaluates the WHERE clause against the newly committed row, so the version predicate now fails and the statement reports zero rows. Under REPEATABLE READ the identical collision raises SQLSTATE 40001 instead, so the handler must fold both shapes into one conflict response.
- Keep UNIQUE (resource_id, version) even though the predicate already serialises writers. It is what makes a lost update unwritable if any other path ever reaches the revision table, and it converts a logic bug into 23505 rather than into a silently missing history row.
- Refuse to auto-retry the whole PUT. A retry re-reads the winner's state and reapplies an intent formed against data that no longer exists — the silent overwrite the version token was added to detect. Return the conflict; merge field-wise only if the patches are provably disjoint.
- Note that now() is the transaction timestamp in PostgreSQL, so resource.updated_at, the revision's created_at and the outbox row share one instant, which is what later makes reconciliation between the three tables unambiguous.
Follow-up
- A client sends the version it read ten minutes ago and the resource has moved three versions. What is in your 409 so it can resolve the conflict without a full re-fetch?
- Two editors, two disjoint fields, no overlap. Does your answer still refuse the second write, and should it?
- Every write now touches a second hot table. How do you keep the outbox insert and its partial index from becoming the write bottleneck at 1.2k writes/second?
How do you handle data drift and model retraining in a production environment?
Medium · ml system design and production
Approach
- Separate the offline training path from the online serving path.
- Name what you would monitor after launch and what triggers a retrain.
- Say where features come from at serving time and how they match training.
Follow-up
- How would you roll the new model out safely?
- How would you detect drift before the metric drops?
What criteria do you use to select between different cloud-native ML services versus custom-built solutions?
Medium
Approach
- Say what you would check first and why it is the highest-information step.
- Work from the requirement backwards to the design.
- Clarify what is being asked and what a complete answer contains.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Specify the outbound webhook contract a customer endpoint can verify
Medium · webhooks · hmac · at-least-once · ordering
Egress delivery sends resource.created, resource.updated and resource.archived to roughly 40k customer endpoints at about 1.5k deliveries/second, with a per-destination concurrency cap of 4 and a 10 second connect-plus-read timeout, at-least-once from the relay. The receiver is code you do not control, often behind a proxy that may re-encode JSON. Specify the payload envelope, how the receiver verifies authenticity and rejects replays, what ordering it may assume, how it deduplicates, which status codes it should return and what each means to us, and the retry, circuit and replay behaviour.
Approach
- Send the fact, not a pointer: event_id, event_type, aggregate_type, aggregate_id, aggregate_version, occurred_at and the payload copied from outbox_event. A receiver told only that resource 42 changed has to call back, and the call back returns a newer state than the event describes - that is precisely how a consumer applies changes out of order while believing it is following them.
- Sign the exact bytes on the wire: HMAC-SHA256 over the timestamp, a separator, and the raw body, in a header, compared in constant time before the body is parsed. Instruct receivers to verify the raw bytes, never a re-serialised copy, since a proxy that reorders keys or changes whitespace invalidates a signature computed over re-encoded JSON. Reject a timestamp outside a tolerance - five minutes is typical - and keep a short seen-set of event_ids, because a capture replayed inside the tolerance still verifies.
- Support key rotation by sending several signature values in one header during the overlap window and publishing that any one that verifies is sufficient. Without that rule a key rotation is a synchronised outage across every destination at once.
- State the guarantee as it actually is: at-least-once and unordered, including within a single aggregate_id. The outbox is ordered per aggregate at the source, but the relay holds up to 4 requests in flight per destination and backs off each retry independently, so a version 6 delivery that needs one retry lands after a version 7 that succeeded on its first attempt. Publishing per-aggregate ordering we do not enforce is worse than publishing none, because a receiver that trusts it overwrites new state with old. Receivers deduplicate on event_id and apply conditionally - UPDATE ... WHERE aggregate_version < :incoming, or the same comparison under a row lock on the aggregate - which discards stale redeliveries and stays correct when two deliveries for one aggregate are processed at the same time. A duplicate and an out-of-order delivery then have the same outcome: no-op.
- Define the response contract in the receiver's terms. 2xx means durably accepted, and accepting means writing it down - a receiver that does its processing inline inside our 10 second timeout will be retried while it is still working. 4xx is permanent and we stop. 429, 5xx and timeouts are retried with capped exponential backoff and full jitter within the per-destination concurrency cap, and a circuit opens after a run of consecutive failures and half-opens on a schedule, so a permanently dead endpoint costs one probe per interval instead of a growing queue.
- Give them a recovery path that does not involve us holding a queue for everyone: a visible parked state, and a replay endpoint keyed by event_id or time range, so a receiver that was down for two hours asks for the gap on its own schedule.
Worked solution (30 min)
- Write the envelope fields and mark which are needed for deduplication and which for discarding a stale delivery.
- Write the signature string construction and the receiver's verification steps in order, including the constant-time compare and the tolerance check.
- Write the receiver's idempotent apply: dedupe on event_id, then the version-conditional write that discards stale and concurrently delivered redeliveries.
- Write the status-to-action table from our side: which codes retry, which stop, and what opens the circuit.
- Describe the parked state and the replay call, including what the customer must supply.
Expected result: A self-contained signed envelope, verification over raw bytes with a replay window and a seen-set, an at-least-once guarantee that states delivery is unordered even within one aggregate, a version-conditional apply, a status table that distinguishes permanent from transient, and a customer-driven replay path.
Verify it yourself
- Delivering the same event twice leaves the receiver's state identical to delivering it once.
- Delivering version 7 and then version 6 for one aggregate leaves version 7 applied.
- A destination that always times out costs one probe per circuit interval rather than a retry queue.
Follow-up
- A customer insists an event never arrived. What do you check, in order, and which table settles the argument?
- A receiver cannot store a version and asks us to deliver their events in order. What would the relay have to give up to offer that, and what does one failing event then do to the rest of their stream?
Every query on one table stalls for forty seconds mid-deploy
Medium · lock contention · ddl · migrations · postgres
During a release on PostgreSQL, every query touching resource times out for about 40 seconds and then recovers with no intervention. The release ran one migration, ALTER TABLE resource ADD COLUMN archived_reason TEXT, and the migration log shows it completing in 6 ms. Unrelated tables showed no change in error rate. Explain how a 6 ms statement caused a 40-second stall, give the ordered checks you would run on a live system to confirm it, and give the migration procedure that prevents a repeat.
Approach
- Separate the statement's duration from the lock's duration. ADD COLUMN with no default is a catalogue-only change and genuinely runs in milliseconds, but it requires ACCESS EXCLUSIVE, and it cannot acquire that until every transaction already touching the table has finished.
- Account for the queueing, which is the part that surprises people. A lock request that is waiting blocks later requests for conflicting modes behind it rather than letting them overtake, so one long-open transaction holds the DDL and the DDL holds all the traffic. The stall length is set by the longest open transaction, not by the size of the change.
- Confirm on a live system in this order: pg_stat_activity for that table ordered by xact_start, looking for the oldest transaction and specifically for state = idle in transaction; then pg_locks where granted = false to find the waiter; then join them on pid to name blocker and blocked. pg_blocking_pids() does that join for you and is the fastest single call.
- Prevent rather than merely time it better. Set lock_timeout to a second or two on the migration session so the DDL abandons the queue after a bounded wait and is retried, instead of holding it for as long as the oldest transaction lives. Be exact about what that buys: queries arriving during the wait still queue behind the pending ACCESS EXCLUSIVE request, so each attempt costs them up to one lock_timeout of added latency. The outage goes from 40 seconds to about one second per attempt, not to zero. Also run migrations away from deploy-time peaks, and put a statement timeout and an idle-in-transaction timeout on the analytics role that opens the long transactions.
- Know the lock each change takes, since the mitigation differs by change. A column with a non-volatile default is a metadata-only change from PostgreSQL 11 and still needs the brief ACCESS EXCLUSIVE; an index needs CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an INVALID index to drop if it fails; a check or foreign key is added NOT VALID and then VALIDATE CONSTRAINT as a separate statement under a weaker lock.
Follow-up
- The same release also wants NOT NULL on that column. What is the sequence that gets there without a long lock?
- Your lock_timeout retry fails ten times in a row because the analytics transaction is always open. What do you change?
- How does this differ on MySQL with InnoDB online DDL, and what is the equivalent of the waiting-lock queue there?
Suggested practice checklist
For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.
Day 1 — Rebuild the primitives by implementing them
- Implement a dynamic array with doubling growth and an operation counter, then change the growth rule to add a fixed sixteen slots instead, and time both for n of ten thousand, a hundred thousand and a million. The fixed-increment version resizes n/16 times at O(n) each, so its total work is quadratic; doubling is what makes append amortised constant.
- Implement a hash map with separate chaining and a load-factor resize, then insert ten thousand keys engineered to land in one bucket and record what happens to lookup time, so that average-case O(1) becomes a claim with a stated precondition rather than a reflex.
- For dynamic-array append and hash-map insert, write down which cost is amortised rather than worst-case, which single operation pays the whole bill, and what a system with a hard per-operation deadline would have to do instead.
Deliverable: Two working implementations plus a timing table showing the input at which each structure's advertised complexity stops holding.
Day 2 — Arrays under an invariant: two pointers, sliding window, binary search
- Solve longest-subarray-with-sum-at-most-K using a sliding window, then run it on an input containing negative numbers and watch it return the wrong answer: extending the window only moves the sum monotonically when every element is non-negative, and that precondition is the whole reason the technique works.
- Write the binary search that finds the first index satisfying a predicate rather than an exact value, put the loop invariant above the loop in a comment, and verify termination on the two inputs that break careless versions: the empty range, and a range where every element satisfies the predicate.
- Compute the midpoint as lo + (hi - lo) / 2 and write one line on why the obvious (lo + hi) / 2 is a genuine defect in a fixed-width integer type and a non-issue in a language with arbitrary-precision integers.
Deliverable: Three solved problems, each with its invariant written above the loop, plus one recorded input on which the sliding window is provably wrong.
Day 3 — Sorting, heaps, and the greedy argument that has to be proved
- Solve one top-k problem three ways, by full sort, by a size-k heap, and by quickselect, then write the values of n and k at which each becomes the right choice, along with quickselect's quadratic worst case and why a randomised pivot makes that unlikely rather than impossible.
- Implement bottom-up heapify and count sift-down steps to confirm it does linear work rather than n log n, because most nodes sit near the bottom of the tree and therefore move only a short distance.
- Take interval scheduling by earliest finishing time and write the exchange argument out in full: given any optimal schedule, swapping in the earliest-finishing interval keeps it feasible and no smaller. Then construct the weighted variant where that same greedy fails and name what has to replace it.
Deliverable: A three-way top-k comparison with measured crossover points, one written exchange argument, and one counterexample to a greedy rule that looks almost identical.
Day 4 — Recursion, memoisation, and the step to a table
- Take one problem with overlapping subproblems, such as edit distance or coin change, instrument the plain recursion with a call counter to show the blow-up, then add memoisation and re-count.
- Convert the memoised version to a bottom-up table and state the two properties you relied on: each subproblem's result depends only on its arguments, and the dependencies form a DAG you can enumerate in order.
- Rewrite one deep recursion with an explicit stack, then find the input length at which the original hits the interpreter's frame limit, which defaults to about a thousand frames in CPython, so you know when the rewrite is required rather than decorative.
Deliverable: One problem in three forms, naive, memoised and tabulated, with call counts for each and the input length at which recursion depth becomes the binding constraint.
Day 5 — Graphs, where most of the work is choosing the traversal
- Implement BFS and DFS over one adjacency list, then answer for each which finds a shortest path in an unweighted graph and which you would use to detect a cycle in a directed graph, including why the in-progress versus finished distinction matters for the second.
- Implement topological sort by in-degree, feed it a graph containing a cycle, and confirm the failure signature is that fewer than V nodes come out rather than an exception, then note that the order it produces is one of several valid ones.
- Run a shortest-path search on a graph with a single negative edge weight and show the wrong answer, then write the precondition Dijkstra actually needs, non-negative weights, because it finalises a node's distance the first time that node is popped, and name the algorithm you would switch to and its own limit.
Deliverable: A small graph library with BFS, DFS and topological sort, plus two inputs that produce documented wrong answers under the wrong algorithm choice.
Day 6 — One day for everything that is not an algorithm
- Sketch one system only to the depth a coding-heavy loop tends to reach: the endpoints, what the service stores, and the single query pattern that decides the schema. Stop at twenty-five minutes.
- Prepare the project answer for an interviewer who codes, which means rehearsing the two levels they push to: the specific thing you built, and why you chose that approach over the alternative they will name. Open with a number and be ready to say what it excludes.
- Prepare the answer to what you would do differently, choosing a real technical mistake with a specific fix rather than a complaint about process or staffing.
Deliverable: One design sketch at endpoint-and-schema depth, plus a project answer rehearsed to two levels of follow-up.
Day 7 — Solve out loud, under time
- Do three timed problems at twenty-five minutes each in a plain editor with no autocomplete and no execution until the end, then tally separately the failures that were syntax and the ones that were approach, because those two numbers call for different fixes.
- Narrate one solution from the first sentence, stating the approach and its complexity before writing any code, and rehearse the sentence you will use when you realise mid-solution that the approach is wrong.
- Re-solve from blank the two problems you were slowest on this week and compare the times against the day they first appeared.
Deliverable: A recording of one fully narrated solution and a tally that separates syntax failures from approach failures.
The failure that was yours
Keep one story where the bad call was yours rather than a dependency's or a manager's. Name the check that would have caught it, whether you added that check afterwards, and whether it has fired since. Answers that route blame outward end the conversation early; answers that end in a guardrail someone still relies on tend to open it up.
How do you handle disagreements within a project team regarding technical architecture?
Medium · behavioural and collaboration
Approach
- Name the disagreement and how you resolved it with evidence.
- Pick a story where you made the decision, not one where you watched it.
- State the situation in two sentences and spend the rest on the reasoning.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that again?
Describe a time you had to explain a complex technical trade-off to a non-technical client or stakeholder.
Medium · behavioural and collaboration
Approach
- Close with what you would do differently, concretely.
- State the situation in two sentences and spend the rest on the reasoning.
- Pick a story where you made the decision, not one where you watched it.
Follow-up
- What did you decide not to do, and why?
- How did you know your change caused the improvement?
Ship under a deadline and bound the debt you chose
Medium · pagination · technical debt · tradeoffs
You have four days to ship a tenant-facing listing endpoint. The version you would defend uses keyset pagination over (tenant_id, status, updated_at DESC, resource_id DESC); the version you can finish uses LIMIT/OFFSET with no matching index. Describe a deadline call you actually made of this shape: what you shipped, what you knowingly deferred, how you bounded the damage with a mechanism rather than an intention, and the specific numeric condition that would force the follow-up. Name who you told and where you wrote it down.
Approach
- Name the deferred failure precisely instead of calling it slow. OFFSET n makes the database produce and discard n rows, so cost grows with page depth; without an index matching the sort, every matching row is read and sorted before the limit applies; and rows inserted between two page fetches shift across the boundary so items are skipped or repeated with nothing in the response to signal it.
- Bound the blast radius with something mechanical rather than a promise: cap maximum page depth, cap page size, restrict the endpoint to one internal caller, or keep it behind a flag. State which failure each cap removes and which it leaves standing.
- Attach a number to the trigger and wire it to an alarm: the first tenant crossing N resources, or the endpoint's p99 crossing its share of the 400 ms budget, so the debt announces itself instead of waiting to be remembered.
- Write it where the next engineer looks, which is the code and the ticket, not a chat message: what was deferred, why, the cap, and the trigger.
- Report what actually happened in your real example, including the case where the trigger never fired and the debt was correctly never repaid.
Follow-up
- At what page depth does the offset version breach your latency budget, given your page size and row counts?
- What breaks first when you switch to keyset pagination later, and what does a client holding an old page token see?
- Who would have overruled you if you had asked for two more days, and did you ask?