Interview guides · Sprinklr
Sprinklr Machine Learning Engineer Interview Questions & Guide 2026
Sprinklr Machine Learning Engineer interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.
The 60-second brief
As a Machine Learning Engineer at Sprinklr, you are at the core of the company’s Unified Customer Experience Management (Unified-CXM) platform. Your work involves building sophisticated models that process massive volumes of unstructured data, enabling global enterprises to gain actionable insights from social media, customer support interactions, and marketing campaigns.
Getting the code to run is the floor. What usually separates answers is the case checked without prompting: empty input, a single element, duplicate keys, or a value that overflows the integer type you chose.
Focus areas
- Paginate large result sets with keyset cursors
- Detect concurrent edits instead of losing writes
- Make every write idempotent under retry
Overview
The role is both challenging and high-impact, requiring you to bridge the gap between complex research and scalable production systems. You will likely contribute to projects involving natural language processing, sentiment analysis, and predictive modeling, directly influencing how brands interact with their customers at scale. Success in this role requires a blend of rigorous technical expertise and a product-focused mindset, as your models are the engine behind the real-time intelligence that Sprinklr provides to its clients.
Candidates are encouraged to focus on both the theoretical foundations of their models and the practical engineering hurdles of deploying them in a distributed, high-throughput environment.
The Sprinklr interview process
Sprinklr candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Technical Screening
What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.
What to demonstrate
- Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
- Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
- Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly
How to prepare
- Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
- Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
- Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live
System Design Interview
You cannot drill a format you do not know, so put the preparation into material that travels. Three pieces of your own work, each rehearsed until you can take a follow-up you did not anticipate, will carry a conversation or a code walkthrough equally well. Specificity is what separates that from filler. A number needs its definition before it means anything: a p99 is over some window and measured at some hop, and a server-side figure excludes the queueing and network time a client would see. The number you cannot qualify is the one to leave out.
What to demonstrate
- Whether your examples carry detail only someone who did the work would hold, such as what the binding constraint actually was, which alternative you rejected and why it was worse, and what you measured on each side of the change
- Whether a number survives one follow-up, meaning you can say what it was measured over and whether it moved because of your change or merely alongside it
- Whether a failure is described with the specific change that followed it, rather than a lesson stated in general terms
- Whether your part in a team effort is stated accurately, including what other people did
How to prepare
- Write a page on each of three projects covering the constraint, the option you rejected, the measurement before and after, and what went wrong. Cut any line you cannot take a follow-up on, since you are writing the parts you will be pressed on rather than a summary.
- Recover the real figures while you still have access: request volume, data size, latency with its percentile and window, team size, timeline. Note where each came from, whether a dashboard, a design document or memory, and mark the estimates so you can say which they are out loud.
- Take your weakest project story to someone who works in a different area and have them ask why four times in succession. The point where you run out of answer is the part to go and re-read before the round.
Behavioral Interview
Your first answer is not really what is scored. It buys the follow-up questions, and those decide the round. An interviewer with fifteen minutes takes one thread and pushes on it four or five times, so a story you can only tell at a single level of detail collapses under the third why. That is an argument for fewer stories known deeply rather than one prepared per prompt. Four or five pieces of work you can still explain down to the code you changed and the argument you had about it will cover nearly anything asked in this round.
What to demonstrate
- Whether a story holds as the questioning moves from what you did to why that instead of the alternative, and then to what you would change knowing what you know now
- Whether you can re-cut a project to answer the question actually asked rather than delivering a rehearsed block that answers an adjacent one
- Whether your level of detail is chosen rather than habitual: going down to the schema when the question is about the data model, staying out of it when the question is about the person who disagreed with you
How to prepare
- Pick four projects and write the chain out four levels deep for each: what you did, why that, why not the alternative, and what would have to be true for the alternative to have won. Where you cannot reach the fourth level, you have a placeholder rather than a story
- Have someone ask why three times in a row on a single thread with nothing else added, and mark the point where you start repeating a sentence you already said. That point is where the interviewer stops learning anything
- Build a one-page index instead of an answer bank: the common prompts in this round (disagreement, a failure that was yours, thin requirements, a deadline you missed, work you inherited) mapped to which of your four projects you would use for each, so the choosing is done now rather than while an interviewer waits
Where candidates lose points
Assuming an isolation level prevents the anomaly you actually have
Isolation levels are named by the SQL standard but implemented differently, so any claim about one is only true of a named engine. PostgreSQL defaults to READ COMMITTED, where every statement takes a fresh snapshot, so two statements inside one transaction can legitimately disagree about the same row. Its REPEATABLE READ is snapshot isolation: it removes non-repeatable and phantom reads but permits write skew, where two transactions each read a set, each conclude their own write is safe, both commit, and the combined result violates a constraint that no single row expresses. Only SERIALIZABLE closes that, and it closes it by aborting a transaction with a serialization failure (SQLSTATE 40001), which means the guarantee is theoretical unless the application has a retry loop. InnoDB's REPEATABLE READ is a different mechanism again - plain SELECTs read a consistent snapshot while locking reads and writes see the latest committed row - so a read-modify-write inside one transaction can act on a value that the transaction's own earlier SELECT never returned.
Shipping a migration and the code that depends on it as a single change
During any rolling deploy, and for as long as a rollback remains possible, old and new code execute against the same schema at the same time. A migration that drops or renames a column breaks every instance that has not restarted yet, and code that requires a column the migration has not applied breaks every instance that restarted early. The discipline is expand then contract: add the new column nullable, write both shapes, backfill in batches, move reads across once the backfill is verified, and only then stop writing the old shape and drop it - four deploys, usually spread over days. It feels disproportionate until the first rollback, at which point it is the only reason the previous version still runs.
Reading the constraints as preamble rather than as part of the problem
The bounds are usually there to eliminate the obvious approach: n up to 10^5 makes an O(n^2) scan roughly 10^10 operations, far outside any per-test time budget, and an input larger than memory rules out loading it at all. When a bound is not given, ask for it, then say out loud which approach it kills.
Finishing a solution without stating its complexity
Give time and space in the same breath as the code, and define n explicitly when there are two sizes, since n nodes and m edges are not interchangeable. Space is the half that gets skipped: count the auxiliary structures you allocate and the recursion stack at its deepest, not only the answer you hand back.
Questions & practice
Describe the architecture and use cases for Transformers in NLP.
Medium · machine learning fundamentals
Approach
- Say how you would validate it, and where leakage could enter the split.
- Pick the metric from the cost of each error type, not from habit.
- State the learning problem: the label, the unit of prediction and how the model is used.
Follow-up
- How would you know the model is overfitting?
- What changes if the classes are heavily imbalanced?
How do you handle imbalanced datasets in a classification problem?
Medium · machine learning fundamentals
Approach
- Pick the metric from the cost of each error type, not from habit.
- State the learning problem: the label, the unit of prediction and how the model is used.
- Say how you would validate it, and where leakage could enter the split.
Follow-up
- Where could label leakage enter this setup?
- What changes if the classes are heavily imbalanced?
Explain the difference between bagging and boosting.
Medium · machine learning fundamentals
Approach
- Pick the metric from the cost of each error type, not from habit.
- Name the simplest model that could work and what would make you move past it.
- State the learning problem: the label, the unit of prediction and how the model is used.
Follow-up
- Where could label leakage enter this setup?
- How would you know the model is overfitting?
Archive a resource graph without breaking live references or recursing
Medium · graph traversal · topological order · tenant isolation
Resources reference other resources within a tenant; for the largest tenant the reference table holds up to 2,000,000 nodes and 8,000,000 edges. Archiving a resource must archive everything reachable from it that nothing outside the set still references, refuse when a live external referrer exists, and terminate when references form cycles, which they legitimately do. Produce the archive order and the refusal list, targeting O(V+E). Say what stops the traversal crossing a tenant boundary, and why recursion is the wrong control structure at this size.
Approach
- Load the subgraph with the tenant predicate on both endpoints of the edge, not only on the side you started from. Scoping the left table alone is the classic cross-tenant leak: one mis-entered edge then pulls another tenant's resources into the traversal and, worse, into the archive.
- Traverse iteratively with an explicit stack. A 2,000,000-node graph can hold a chain deep enough to exhaust a native stack in the low tens of thousands of frames, and that failure is a process crash rather than an error you can return.
- Treat cycles as data rather than corruption: compute strongly connected components with Tarjan in O(V+E) using its own explicit stack, then condense. The condensation is a DAG, so a topological order over it gives the archive order, and every member of a component archives in one transaction because no order within a cycle is valid.
- Decide refusals with reverse edges. A candidate is archivable only if every in-edge originates inside the candidate set, so build the transpose or count in-degrees restricted to the visited set, and emit each blocked resource with the id of the external referrer, which is the only part of the answer an operator can act on.
- Store the graph as CSR rather than a map of lists: an offsets array of V+1 8-byte entries plus E 8-byte targets is about 80 MB at this size, where boxed adjacency lists cost several times that and lose cache locality on every hop.
- Run Kahn over the condensation for the order in O(V+E). If the emitted count is short of the component count the condensation step itself is wrong, since a condensation cannot contain a cycle, which makes the check free.
Worked solution (30 min)
- Write the edge-loading query with the tenant predicate on both endpoints and state what it does with a cross-tenant edge.
- Implement iterative Tarjan with an explicit stack and confirm on a three-node cycle that it emits one component of size three.
- Build the transpose restricted to the visited set and mark every node with an in-edge from outside it as refused, carrying the referrer id.
- Run Kahn over the condensation and verify the emitted order against the referrer-before-referenced rule.
- Size the CSR arrays for 2,000,000 nodes and 8,000,000 edges and compare against a boxed adjacency map.
Expected result: An iterative O(V+E) traversal over a tenant-scoped CSR subgraph, SCC condensation so cycles archive atomically as one component, a transpose-based refusal list naming the external referrer for each blocked resource, and a Kahn topological order over the condensation, with recursion replaced by an explicit stack because of graph depth rather than style.
Verify it yourself
- A two-node cycle with no external referrers archives as one unit, and adding a single external referrer moves both nodes to the refusal list.
- No resource appears in the archive order before something that references it, except within its own component.
- A counter on edge visits matches the loaded edge count exactly, confirming each edge is traversed once.
Follow-up
- The graph is read in one query and the archive writes a minute later. What can change in between, and how do you make the write safe?
- The candidate set is 400,000 resources. Is that one transaction, and if not, what does a half-finished archive look like to a reader?
- An edge points at a resource in another tenant. Is that a refusal, an error, or an alert?
Hold a per-tenant active cap against concurrent creates
Hard · write skew · isolation levels · constraints
A tenant on the standard plan may hold at most 50 resources with status='active'. The create handler runs SELECT count(*) FROM resource WHERE tenant_id = $1 AND status = 'active', compares to 50, then inserts. Two creates arrive 3 ms apart on different instances and the tenant lands at 51. Name the anomaly, say whether PostgreSQL 16 READ COMMITTED or REPEATABLE READ prevents it and why, then give an implementation that holds the cap at READ COMMITTED with the exact statements. Finally, say what changes when the cap is 'at most one running export per tenant' on job_run.
Approach
- Name it: write skew. The two transactions read an overlapping set and write disjoint rows, so there is no row-level conflict for the engine to detect and each commit is individually legal.
- Rule out the levels precisely. READ COMMITTED takes a fresh snapshot per statement and takes no lock on the counted rows, so both see 49. PostgreSQL's REPEATABLE READ is snapshot isolation: it removes non-repeatable reads and phantoms within the snapshot but still admits write skew, because the anomaly is not a re-read of a changed row, it is a read of a set that a concurrent transaction invalidates. Only SERIALIZABLE closes it, by tracking the read dependency and aborting one transaction with SQLSTATE 40001 — a guarantee that exists only if the application re-runs the whole transaction from the read.
- Convert the set predicate into a single-row conflict: keep tenant.active_resource_count and run UPDATE tenant SET active_resource_count = active_resource_count + 1 WHERE tenant_id = $1 AND active_resource_count < 50 in the same transaction as the INSERT. Zero affected rows is the cap, returned as 409. The row lock serialises the decision at any isolation level, and contention is bounded to one tenant's row — which is also the fair-scheduling unit, unlike a global counter that would convoy every tenant behind one row.
- State the cost you just took on: a counter is a second source of truth that can drift, so every path that changes status must adjust it inside the same transaction, and a periodic reconciliation has to exist, with resource_revision as the authority for what the count should have been.
- For the job case the invariant is expressible per row, so let the database hold it: a partial unique index on job_run (tenant_id, job_type) WHERE status IN ('queued','running') makes a second running export unwritable and the loser takes 23505, mapped to 409. That is strictly better than a counter — no drift, no reconciliation — and it is available only because the cap is one rather than fifty.
- Add the retry discipline each route demands: under SERIALIZABLE both 40001 and deadlock 40P01 are retryable and the retry must re-execute the read, while under READ COMMITTED with the counter nothing retries, because the conflict is reported to the caller rather than raised as an error.
Follow-up
- A resource moves from archived back to active. Which statements change, and what breaks if the counter update and the status change land in different transactions?
- The cap becomes plan-dependent and a plan can change mid-month. Where does the number 50 live, and who reads it?
- How do you detect after the fact that the counter drifted, without locking the table?
Replace offset paging on the resource feed with keyset
Easy · keyset pagination · composite index · sort order
resource holds resource_id, tenant_id, owner_user_id, title, body_ref, version, status ('draft','active','archived','deleted'), created_at, updated_at, deleted_at, with an index on (tenant_id, status, updated_at DESC, resource_id DESC). The listing endpoint returns active resources for one tenant, newest update first, 50 per page, today with LIMIT 50 OFFSET n. Tenants reach page 400 and rows are created while they read. Write the keyset query, define what the cursor carries and how it is encoded, and say which part of the index each predicate uses. Assume PostgreSQL 16.
Approach
- Name the two failures separately. OFFSET 20000 makes the server produce and discard 20,000 rows, so page cost grows with depth rather than with page size. Independently, any write that changes how many rows sort above the offset moves the window between two fetches, and the direction decides which anomaly you get: an insert lands at the head of updated_at DESC and pushes already-returned rows down past the boundary, so they are returned a second time; a delete above the offset, or a row whose updated_at is bumped above the cursor, pulls rows up and one is never returned at all. Nothing in the response reveals either.
- Write the seek: WHERE tenant_id = $1 AND status = 'active' AND (updated_at, resource_id) < ($2, $3) ORDER BY updated_at DESC, resource_id DESC LIMIT 50. The row-value comparison is one index range rather than a disjunction, and both columns are NOT NULL, which is what makes that comparison well defined.
- Map each predicate onto the index: tenant_id and status are equality on the leading columns, (updated_at, resource_id) is the range, and the ORDER BY matches the index order so no Sort node appears and the scan stops after 50 rows. The DESC in the definition only matters for mixed directions — a plain ascending btree on the same columns is read backwards for this query.
- Put both sort columns in the cursor and nothing the client can tamper with into another tenant: base64 of (updated_at, resource_id), validated server-side, with tenant_id taken from the principal.
- State the residual honestly. Keyset is stable against concurrent inserts and deletes, but not against a row whose updated_at changes mid-scroll — that row moves in the ordering and can be seen twice. If the feed must be a snapshot, order by an immutable key or bound the page set with updated_at <= the cursor's start value.
- Keep a total out of the page path. A tenant-wide COUNT(*) is the scan keyset just removed; fetch LIMIT 51 and return has_more instead.
Worked solution (25 min)
- Seed one tenant with 500k active resources, 2% of them sharing an identical updated_at.
- Time LIMIT 50 OFFSET 0 against OFFSET 20000 and record rows-read from EXPLAIN (ANALYZE, BUFFERS) for each.
- Page the whole set with the keyset query while an insert-only writer adds 100 rows/second, collecting resource_ids, and repeat the run with OFFSET.
- Repeat both runs under a second writer profile that also deletes 20 rows/second from pages already returned and bumps updated_at on 20 more, and diff each collected id set against the rows that existed for the whole run.
- Remove resource_id from the cursor so the seek degrades to updated_at < $2, and re-run the tie-heavy section of the feed.
Expected result: Keyset reads about 50 index entries per page and its latency is flat from page 1 to page 10,000; under both writer profiles the collected ids contain no duplicate and omit no row that existed for the whole scroll. OFFSET at page 400 reads 20,050 rows to return 50. Under the insert-only writer it repeats rows and skips none — an insert can only push a row down past the boundary — and only the second profile, whose deletes and updated_at bumps move rows up past the cursor, also makes it skip. Without the tie-break, the seek is updated_at < $2, which drops every still-unread row sharing the boundary timestamp.
Verify it yourself
- EXPLAIN shows an Index Scan with no Sort node — a Sort means the ORDER BY is not satisfied by the index.
- Buffers read per page stays constant as page depth grows.
- Ids collected across the scroll are unique, and complete for rows untouched during the run.
Follow-up
- The client asks for 'jump to page 400'. What do you offer instead, and what does the honest version cost?
- Sort order becomes user-selectable across four columns. How many indexes is that, and which would you refuse to add?
- What does the cursor do when the row it points at has since been deleted?
Design a system to detect spam in real-time customer support chats.
Medium · ml system design and production
Approach
- Name what you would monitor after launch and what triggers a retrain.
- Fix the product goal and the online metric before choosing any model.
- Separate the offline training path from the online serving path.
Follow-up
- How would you detect drift before the metric drops?
- What happens when a feature is missing at serving time?
What are the common pitfalls when deploying a model to production?
Medium · ml system design and production
Approach
- Name what you would monitor after launch and what triggers a retrain.
- Fix the product goal and the online metric before choosing any model.
- Say where features come from at serving time and how they match training.
Follow-up
- How would you detect drift before the metric drops?
- How would you roll the new model out safely?
How would you measure the success of a new recommendation engine?
Medium
Approach
- Clarify what is being asked and what a complete answer contains.
- Say what you would check first and why it is the highest-information step.
- Work from the requirement backwards to the design.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Keep one unresponsive destination from stalling all webhook delivery
Medium · fan-out · circuit breaker · timeouts · bulkheads
Egress delivery sends about 1.5k webhooks/second to 40k destinations, with a per-destination concurrency cap of 4 and a 10-second connect-plus-read timeout. One destination begins accepting connections and never responding; within the hour 150 destinations behave the same way. Design the delivery path so unrelated destinations are unaffected: the pool structure, the timeouts, the retry policy, the per-destination circuit, and what is recorded so a retry is not a second effect at the receiver. State how many in-flight slots the degraded destinations hold and why that number decides the design.
Approach
- Start with the number, and with the law that produces it. In-flight work is arrival rate times time in service, so 1.5k/second against a healthy 200 ms response needs about 300 concurrent slots. Per destination the same product applies, ceilinged by the concurrency cap: at the fleet average of 0.0375 deliveries/second per destination (1.5k spread over 40k) a 10-second timeout is 0.375 slots. A destination that has queued retries behind it is a different regime - every slot refills the instant an attempt expires, so it sits pinned at its cap of 4 - and 150 of those hold 600 slots, more than a pool sized for healthy traffic, entirely consumed by endpoints that will never answer. The per-destination cap bounds one endpoint and says nothing about the aggregate, which is exactly why it alone is not containment.
- Contain with bulkheads and an admission bound rather than a larger pool. Cap total in-flight per pool and shard destinations across pools by a hash of destination id, so a correlated group - one provider, one region - cannot exceed its pool's share. A delivery refused admission and re-queued with backoff is strictly better than one holding a slot on behalf of a receiver that is not listening.
- Treat the timeout as two timeouts, and be exact about what shortening one buys. Connect and read are separate failures and both must be shorter than the budget of whatever is waiting. Occupancy is min(cap, arrival rate x timeout), so dropping the read ceiling from 10 seconds to 3 cuts a merely slow destination's occupancy proportionally, 0.375 slots to 0.11 at the fleet average. It does not cut the 4 slots held by one of the 150: a destination with a retry backlog arrives far above cap/timeout - 0.4/second at a 10-second timeout, 1.33/second at 3 - so it stays pinned at the cap either way and only the slot-seconds per attempt fall. What that does buy is detection rate: 3.3x more failures observed per second on the same four slots, which is how fast the circuit reaches its threshold. Pick the value from the measured latency distribution of successful deliveries, with their high percentile as the floor, not from a round number.
- Add a circuit per destination, counting a timeout as a failure. Once open, fail fast without taking a slot - that is the whole point, converting 4 held slots into zero. Half-open on a schedule with exactly one probe and close only if the probe succeeds, so a permanently dead endpoint costs one request per interval instead of a growing retry queue.
- Make retries safe and non-synchronising. Back off with full jitter, sleeping a random value in [0, min(cap, base x 2^attempt)], because a fixed delay re-synchronises every failed delivery to one destination into a simultaneous burst. Delivery is at-least-once, so the payload carries the event id under the signature and the receiver deduplicates on it; record the attempt against (destination, event id) rather than a bare success flag, so a lost response does not become a second business effect on the other side.
Worked solution (25 min)
- Compute healthy in-flight from rate times latency, then slots held by 150 destinations at the cap and the full timeout, and compare both against one pool size.
- Write the pool sharding rule and the admission bound, and state what a refused delivery does next.
- Pick connect and read timeouts from the success-latency distribution, then compute min(cap, arrival rate x timeout) for an average destination and for one with a retry backlog, and say which of the two the shorter timeout actually moves.
- Write the circuit's state machine with its open threshold, probe interval and close condition, and the backoff formula with full jitter.
Expected result: Destinations sharded across bounded pools with a total in-flight cap per pool, separate connect and read timeouts chosen from the success-latency distribution, a per-destination circuit that fails fast while open and probes once per interval, and full-jitter backoff on retry. A shorter read timeout lowers occupancy only for destinations arriving below cap/timeout; a dead destination with a retry backlog stays pinned at its cap of 4 whatever the timeout, and the circuit opening is what takes it to zero. Deliveries carry a signed event id the receiver deduplicates on, and attempts are recorded per (destination, event id). Degraded destinations hold at most their pool's share rather than 600 slots of a shared pool.
Verify it yourself
- Slots held by degraded destinations are bounded by the pool shard, not by how many destinations degrade.
- An open circuit reduces slots held by that destination to zero between probes.
- Lowering the read timeout does not reduce successful deliveries, verified against the measured success-latency distribution, and does not by itself reduce the slots a backlogged dead destination holds.
- A redelivered event is discarded by the receiver on its event id rather than applied twice.
Follow-up
- The destination is not dead - it answers in 9.5 seconds with a 200. Does a failure-rate circuit open? Should anything shed that traffic, and on what signal?
- One destination requires deliveries in order. What does a per-destination concurrency of 4 do to that guarantee, and what would you change to offer it?
- A destination has been parked six hours with 900k undelivered events. What does resuming look like, and is delivering the whole backlog the right call?
One customer endpoint stalls deliveries to every other destination
Medium · bulkhead · timeouts · saturation · retry amplification
The egress service delivers about 1.5k webhooks/second across 40,000 destinations, with a per-destination concurrency cap of 4 and a 10-second connect-plus-read timeout. Throughput falls to 300/second, queue depth climbs, and p99 delivery latency for unaffected destinations goes from 200 ms to minutes, while the error rate barely moves. One tenant holds 900 destination rows whose URLs share a hostname that now answers in 9.5 seconds. Explain the mechanism with the arithmetic, then give the containment in the order you would apply it.
Approach
- Look at saturation before errors. A flat error rate with collapsing throughput says nothing is failing, things are waiting, so the first signal to pull is in-flight request count or pool wait time rather than the error counter. This is the distinction that decides the whole investigation.
- Group in-flight work by resolved host, not by destination id. The cap is keyed per destination row, so 900 rows sharing one hostname buy 3,600 concurrent slots against a single host, each held for 9.5 seconds. The bulkhead was never a bulkhead for that host, and grouping by the wrong dimension is why the dashboard looked healthy.
- Do the arithmetic in both directions. Required concurrency is arrival rate times latency, so 1.5k/second at 200 ms needs about 300 in flight, which is entirely consumed by 3,600 slow slots; conversely whatever concurrency is left sustains rate equals concurrency divided by 9.5 seconds, which is the 300/second you are seeing. Matching both numbers is what promotes this from a plausible story to the mechanism.
- Explain why the circuit breaker never helped. It opens on consecutive failures, and a 9.5-second response inside a 10-second timeout is a success. Slow is not failing, so an error-rate breaker cannot see this; you need a slow-call ratio, a deadline propagated from the caller's remaining budget, or a concurrency limiter.
- Contain in order: park the offending host so the shared pool drains, add a per-resolved-host concurrency cap alongside the per-destination one, give slow hosts their own queue so they cannot occupy the general pool, derive the timeout from the delivery deadline rather than a round number, and check the retry policy is not tripling load on a host that is already slow. Use backoff with full jitter so retries do not resynchronise on recovery.
- State the invariant you are restoring: one tenant's endpoints degrade only that tenant's deliveries. That is a property to load-test for, not to assume from a config value.
Follow-up
- The host recovers to 80 ms. How long does the queue take to drain, and what does the drain do to the recovered host?
- Where should the 10-second timeout number actually come from?
- If that tenant had one destination row instead of 900, would the cap of 4 have saved you? What would you measure to be sure?
Suggested practice checklist
Four days sample coding, design, fundamentals and the practical rounds at deliberately shallow depth, which is enough to surface the topics you did not know were in scope. That map, rather than a guess made on day one, decides where the last three days go.
Day 1 — Coding, one pass at shallow depth
- Solve one problem from each of six families, an array with two pointers, hash counting, binary search, a tree traversal, a graph traversal and one dynamic program, under a hard twenty-minute cap with no extensions, marking each finished, late, or stalled.
- For every stall, write the exact move you could not make rather than the subject, so the note reads could not turn the recurrence into a loop rather than bad at dynamic programming.
- Fix nothing today. The value of the pass is the unfixed record.
Deliverable: Six timed attempts marked finished, late or stalled, each stall carrying a named blocking move.
Day 2 — Design, one pass at shallow depth
- Spend twenty minutes each on three different shapes, a read-heavy feed, a write-heavy ingest path, and something needing a transaction across two entities, stopping each at requirements, interface and data model.
- After each, write the first question you could not answer, which is usually a number you could not estimate or a failure mode you had no vocabulary for.
- Mark which of the three you would be most relieved not to be asked, and treat that as data rather than as a preference.
Deliverable: Three shallow designs, each with the first unanswerable question written at the bottom.
Day 3 — Fundamentals and the practical rounds
- Answer eight short questions in writing at four minutes each, covering the material that fills the gaps between the big rounds: what happens between a URL and a rendered page, what an index costs on write, when a process is preferable to a thread, and what conditions a deadlock requires.
- Do one thirty-minute practical task of the kind a take-home compresses: read an unfamiliar two-hundred-line file and write what it does, what you would change, and the one thing you remain unsure of.
- Score every answer fluent, correct but slow, or absent, and keep the absent ones visible.
Deliverable: Eight scored short answers and one written reading of unfamiliar code.
Day 4 — The rounds that are about you, and the map
- Deliver three behavioural answers aloud against a timer, a conflict, a failure you owned, and a decision made without enough information, marking any that ran past three minutes or contained no number.
- Assemble the map: every marked item from days one to three on a single page, sorted by how likely it is to appear in your loop rather than by how uncomfortable it felt.
- Choose exactly two areas for the remaining three days and write down what you are deliberately abandoning.
Deliverable: A one-page scored map of the whole surface area with two areas chosen and the rest explicitly abandoned.
Day 5 — First chosen area, to the depth you skipped
- Work the higher-ranked area in four focused blocks, choosing items one level above where you stalled rather than repeating what already works.
- After each block write the rule you extracted in one sentence with its precondition attached, since a rule carrying no precondition is exactly what fails under a variation.
- Re-attempt the day-one or day-two item that exposed this area and compare against the original timing.
Deliverable: Four worked blocks, a timed re-attempt against the original, and three one-sentence rules with preconditions.
Day 6 — Second chosen area, where the gap is coverage rather than speed
- Treat the second area differently from the first. Day five drilled something you could already half-do; this one is usually a topic you had simply never met, so build one worked reference example end to end and keep it, rather than attempting six problems badly.
- Write down the vocabulary you were missing on day two or three, five terms at most, each with the one sentence that makes it usable in an answer rather than the textbook definition.
- Redo the shallow attempt that exposed this area and note whether you now fail later in the problem, because moving the failure point is the realistic gain from a single day and is worth more than a score that did not change.
Deliverable: One worked reference example for the newly covered area, a five-term vocabulary list, and a note on where the failure point moved.
Day 7 — Reassemble the loop
- Sit two rounds back to back with no gap, ordering them so the area you chose second comes last, because the map was built from rested, isolated attempts and the loop will reach your weaker area when you are already spent.
- Write where the second round suffered from the first, which is normally the point at which structure collapses into narration.
- Reduce the week to one page holding only the rules you can state without reading them.
Deliverable: Mock notes on cross-round carryover plus a one-page card of rules you can recite from memory.
Know where your numbers came from
Bring the two or three numbers the story rests on and know how they were collected. A p99 whose timer starts inside your handler excludes the time a request spent queued, so it can sit flat while users wait longer. Give the window, the percentile and what the measurement left out, or drop the number.
How do you manage your time when working on multiple, competing priorities?
Medium · behavioural and collaboration
Approach
- Close with what you would do differently, concretely.
- Name the disagreement and how you resolved it with evidence.
- State the situation in two sentences and spend the rest on the reasoning.
Follow-up
- How did you know your change caused the improvement?
- What would you do differently if you ran that again?
Describe a time you disagreed with a teammate on a technical approach.
Medium · behavioural and collaboration
Approach
- Name the disagreement and how you resolved it with evidence.
- Give the blast radius: what could have broken, and what you measured.
- Close with what you would do differently, concretely.
Follow-up
- What would you do differently if you ran that again?
- What did you decide not to do, and why?
Estimate work you have never done and defend the range
Hard · estimation · backfills · expand-contract
You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.
Approach
- Decompose into independently deployable steps before estimating anything: add the column nullable, write both shapes, backfill in batches, verify, move reads, stop writing the old shape, drop it. That is four deploys spread over days, and the calendar estimate is dominated by them rather than by the loop's runtime.
- Do the arithmetic aloud for the part that has arithmetic in it: batch size times number of batches times per-batch duration, at a write rate the primary can absorb alongside roughly 1.2k writes per second of production traffic. The loop is throttled by replication lag and lock waits, not by how fast it can issue statements.
- Price the schema step by its lock rather than its statement duration. In PostgreSQL an ALTER TABLE taking ACCESS EXCLUSIVE waits for every open transaction on that table while later queries queue behind it, so a millisecond change issued during a thirty-second analytics query stalls that table for thirty seconds. Adding a nullable column with a non-volatile default avoids a rewrite from version 11; a new index wants CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an invalid index behind if it fails.
- Express the answer as a range whose endpoints each trace to a stated assumption, then name the cheapest experiment that collapses it, which is almost always running one real batch against the real table and multiplying.
- Commit to a checkpoint rather than a completion date: the day you report a measured number from that first batch. That is a promise you can keep under uncertainty, and it is what the asker actually needs in order to plan.
Follow-up
- How do you verify the backfill genuinely finished, given rows written by production traffic while it ran?
- Where does the backfill resume from after a worker is killed mid-batch, and what makes that resume point trustworthy?
- Your first batch comes back ten times slower than assumed. What do you tell the person waiting on the estimate, and when?