Interview guides · Diligent Tec

Diligent Tec Machine Learning Engineer Interview Questions & Guide 2026

Diligent Tec Machine Learning Engineer interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.

The 60-second brief

The role of a Machine Learning Engineer at Diligent Tec is pivotal in shaping the future of the company’s AI-driven products. As a Machine Learning Engineer, you will design and implement machine learning models that enhance the functionality and intelligence of various applications. Your work will directly influence user experience, making products smarter and more efficient. The impact of this role stretches across teams, driving innovation and contributing to the strategic objectives of the organization.

The shape of the workload matters more for your prep than the industry label does. Read-heavy serving, write-heavy ingestion and scheduled batch processing have different binding constraints and fail in different places, so find out which one the team lives in before picking design topics.

Focus areas

Trace a symptom to a mechanism under load
Detect concurrent edits instead of losing writes
Paginate large result sets with keyset cursors

Overview

You will work on complex problems that require not only deep technical knowledge but also creativity and critical thinking. From developing predictive algorithms that inform business decisions to optimizing data pipelines for machine learning applications, your contributions will be essential. Collaborating with cross-functional teams, you will help transform data into actionable insights that influence product direction and improve user satisfaction. This role not only demands technical proficiency but also offers the opportunity to work on cutting-edge technologies, making it an exciting and fulfilling career path.

The Diligent Tec interview process

Diligent Tec candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

The person on this call usually cannot evaluate your code and does not need to. They write a short paragraph, and that paragraph is what a hiring manager skims when deciding who to put on your loop. So the test is not whether your work was hard, it is whether a non-engineer can repeat it correctly. Name systems by what they did rather than by their internal codename, give each project a shape (what was breaking, what you changed, what happened after), and keep the whole walkthrough near ninety seconds. Depth that cannot survive a paraphrase reads as vagueness.

What to demonstrate

Whether a non-engineer can restate your projects without distorting them, since their paraphrase is what travels to the hiring manager, not your sentences
Whether each project has a shape rather than a stack list: the failure or constraint, the change you made, the result and how it was measured
Whether you can say what was yours inside a team project without either inflating it or disappearing into the plural

How to prepare

Rewrite each headline project as two sentences with no internal system names and no acronyms outside your company, then say them to someone outside engineering and have them repeat them back. Fix whatever came back wrong
Attach one measured number to each project: the baseline, the change, and the window it was measured over. Where nothing was ever measured, say that plainly rather than reaching for a plausible percentage
Time the background walkthrough against a clock. If it runs past two minutes, compress the earliest role to a single clause and spend the recovered time on the most recent one

Technical Interviews

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Behavioral Interviews

What you say here is written down by each interviewer and compared afterwards, so the unit of evaluation is a claim someone else could check, not a well-told narrative. Two things make a story checkable: detail only a participant would hold, and a clean line around which part was yours. Vague ownership is the usual failure and it is usually accidental, because engineers say we about the team's work and we about their own, so the thing they personally built disappears into the plural. Name the part you wrote, and name who did the rest.

What to demonstrate

Whether your details are ones a participant would hold and an observer would not: the constraint that ruled out the obvious approach, the first attempt that failed, the person who objected and on what grounds
Whether ownership survives a direct question, since a follow-up to we decided is routinely who decided, and an answer that stays plural at that point is read as the work belonging to someone else
Whether the numbers you quote are ones you would say identically to a former colleague with the dashboard open

How to prepare

Go through each story replacing every we with either I or a named role (the on-call engineer, the reviewer, the other team) and check the story still holds together. Wherever it stops making sense you have found a part you cannot actually speak to
Open the artefacts for two of your stories, the pull request, the design doc, the incident notes, and read them for dates and figures you have been rounding in the retelling. Correct your version to match
For each story write the single sentence you would least want repeated to a former teammate, then either make it accurate or take it out

Onsite Interviews

Nobody in the room with you decides this. Interviewers typically write their rounds up separately, often before seeing anyone else's, and the outcome is settled later from those write-ups. A split panel gets resolved by whichever note carries specific evidence, so what you want out of each room is one concrete thing that person could write down: a bug you caught yourself, a trade-off you named, a decision you owned. The rest is arithmetic. The project you describe in a behavioural conversation is often the same system you sketched an hour earlier, and the two accounts have to agree.

What to demonstrate

Whether the scale, team size and timeline you attach to a project hold steady when that project resurfaces in a different round
Whether each interviewer leaves with a specific thing to cite rather than a general impression of competence
Whether a trade-off you defended in one round survives a challenge in another, instead of being quietly swapped for the answer the new interviewer seemed to want
Whether a question you have already answered earlier in the day gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page sheet per project fixing the figures you will quote — request volume, data size, team size, elapsed time, what broke — and say them aloud from the sheet until they come out identical every time
For each round on the schedule, decide in advance the one sentence you want in that person's notes, then check in a mock that you said it outright instead of leaving it to be inferred
Have someone ask you the same project question twice, an hour apart, and diff the two answers for numbers that moved or a trade-off that reversed

Where candidates lose points

Letting a slow dependency consume unbounded concurrency

The failure that takes a service down is usually not an error but a delay. A dependency answering in thirty seconds instead of fifty milliseconds holds each request's worker or connection six hundred times longer, and since required concurrency is arrival rate times latency, a fleet sized for sixty in-flight requests now needs thirty-six thousand to sustain the same rate - so it queues, and requests whose clients have already abandoned them still occupy resources. Retries make it precisely worse: a policy of three attempts triples the load on a dependency at the exact moment it is least able to serve, which is how one slow dependency becomes an outage of everything sharing that pool. Containment is four specific things - a timeout on every outbound call shorter than the caller's remaining budget, a bounded pool per dependency so one cannot starve the others, backoff with full jitter rather than a fixed delay so retries do not resynchronise, and a circuit that stops sending once the failure rate makes an attempt pointless.

Shipping a migration and the code that depends on it as a single change

During any rolling deploy, and for as long as a rollback remains possible, old and new code execute against the same schema at the same time. A migration that drops or renames a column breaks every instance that has not restarted yet, and code that requires a column the migration has not applied breaks every instance that restarted early. The discipline is expand then contract: add the new column nullable, write both shapes, backfill in batches, move reads across once the backfill is verified, and only then stop writing the old shape and drop it - four deploys, usually spread over days. It feels disproportionate until the first rollback, at which point it is the only reason the previous version still runs.

Assuming the input fits in memory

Ask how large the input is in bytes before committing to an in-memory algorithm; beyond that point the options are a single streaming pass, an external sort with bounded buffers, or a sketch that trades exactness for constant memory. An algorithm that assumes random access to the whole input is a different algorithm from one that sees each element once.

Treating a network call as though it were a local function call

A remote call can be slow, fail, or return after you stopped waiting, so name the timeout, the retry policy, and what the caller sees while the dependency is down. A call with no timeout turns one slow dependency into an exhausted thread or connection pool in every service upstream of it.

Questions & practice

Given a specific business problem, how would you approach building a machine learning solution?

Medium · machine learning fundamentals

Approach

Pick the metric from the cost of each error type, not from habit.
Say how you would validate it, and where leakage could enter the split.
Name the simplest model that could work and what would make you move past it.

Follow-up

What changes if the classes are heavily imbalanced?
Where could label leakage enter this setup?

What considerations do you take into account when deploying a machine learning model to production?

Medium · machine learning fundamentals

Approach

Say how you would validate it, and where leakage could enter the split.
State the learning problem: the label, the unit of prediction and how the model is used.
Pick the metric from the cost of each error type, not from habit.

Follow-up

How would you know the model is overfitting?
What changes if the classes are heavily imbalanced?

Write a function to implement logistic regression from scratch.

Medium · machine learning fundamentals

Approach

Name the simplest model that could work and what would make you move past it.
Say how you would validate it, and where leakage could enter the split.
Pick the metric from the cost of each error type, not from habit.

Follow-up

How would you know the model is overfitting?
What changes if the classes are heavily imbalanced?

How would you architect a machine learning pipeline for real-time data processing?

Medium · machine learning fundamentals

Approach

State the learning problem: the label, the unit of prediction and how the model is used.
Pick the metric from the cost of each error type, not from habit.
Name the simplest model that could work and what would make you move past it.

Follow-up

How would you know the model is overfitting?
What changes if the classes are heavily imbalanced?

How would you optimize a function for speed and efficiency?

Medium · coding and algorithms

Approach

Walk one small example through your approach before writing the whole thing.
State the target complexity and say which constraint rules the naive version out.
Name the brute-force solution and its complexity before improving on it.

Follow-up

What is the worst case, and how likely is it on real data?
How does this change if the input no longer fits in memory?

What strategies do you use to ensure code quality and maintainability?

Medium · coding and algorithms

Approach

Restate the input: its shape, its size, and what is guaranteed about it.
State the target complexity and say which constraint rules the naive version out.
Choose the data structure from the access pattern, not from familiarity.

Follow-up

How does this change if the input no longer fits in memory?
What is the worst case, and how likely is it on real data?

Track a rolling failure rate per destination for circuit decisions

Easy · sliding window · ring buffer · circuit breaker

The egress service delivers about 1,500 webhooks per second across roughly 40,000 destinations, each call bounded by a 10 second timeout. Maintain, per destination, the failure rate over the trailing 60 seconds so a caller can ask before dispatch whether the circuit should open. Attempts arrive as (destination_id, finished_at_ms, outcome). Requirement: amortised O(1) per attempt, with total memory bounded by the destination count rather than by traffic. Give the structure, its exact memory, and the rule that stops a destination with three attempts from opening a circuit.

Approach

Name the exact-deque version and then reject it as the default. Holding timestamps and advancing a tail pointer past anything older than now minus 60 seconds is a correct two-pointer window at amortised O(1) per attempt, but its memory tracks in-window traffic, so one destination in a retry storm holds hundreds of thousands of entries while thousands of quiet destinations hold none.
Use a ring of 60 one-second buckets per destination, each bucket a pair of counters for attempts and failures. On an attempt, advance the ring by the elapsed whole seconds, zeroing at most min(elapsed, 60) buckets, then increment the head. That is amortised O(1) with a fixed footprint per destination.
State the footprint: 60 buckets times two 4-byte counters is 480 bytes of payload per destination, so 40,000 destinations is roughly 20 to 25 MB with per-entry overhead, bounded by the catalogue rather than by the rate. The cost is granularity, since the oldest bucket ages out in whole seconds, which is far tighter than the decision needs.
Require a minimum sample before the circuit may open. A destination with three attempts and three failures reads as 100 percent and is not evidence; a floor of roughly 20 attempts in the window makes the ratio meaningful, and below that floor use a run of consecutive failures as the trigger instead.
Expire idle destinations, or memory grows with every destination ever seen rather than with the live set. Hold the rings in a bounded LRU keyed on destination_id and treat a miss as no history, which is the correct default for an endpoint that has been silent for a minute.
Keep the half-open probe out of the window arithmetic. After the circuit opens, one probe per interval decides whether to close it, and folding that single success into a window that still holds a 100 percent failure history would reopen the destination on one data point.

Worked solution (20 min)

Define the bucket struct and the advance step: take floor(finished_at_ms / 1000), compare with the ring's current second, zero min(delta, 60) buckets forward, then write into the new head.
Trace a destination that receives 5 attempts, goes silent for 90 seconds, then receives one more, and confirm the rate is computed from one attempt rather than six.
Compute total memory for 40,000 destinations at 60 buckets of two 4-byte counters, and state what changes if the window widens to 300 seconds.
Write the open rule as a single predicate combining the minimum-attempt floor with the rate threshold.

Expected result: A per-destination ring of 60 one-second (attempts, failures) buckets advanced lazily for amortised O(1) cost, roughly 480 bytes of counters per destination and about 20 to 25 MB for 40,000 of them, an LRU bound on live rings, and an open rule requiring both a minimum attempt count in the window and a rate above threshold.

Verify it yourself

After 60 seconds of silence a destination's window holds zero attempts and cannot open a circuit.
Memory measured with one hot destination and 39,999 idle ones is indistinguishable from the reverse, which the deque version would fail.
Replaying the same attempt stream with the clock shifted by an hour yields the same final rate.

Follow-up

The fleet is 30 instances and each sees roughly a thirtieth of a destination's traffic. Where does the rate actually live, and what does a per-instance answer get wrong?
A destination answers in 9.5 seconds and succeeds. It is not failing but it is consuming your per-destination concurrency. What signal should open the circuit here?
How would you make the window survive a process restart, and is it worth the cost?

Find version gaps and relay lag with window functions

Medium · window functions · event tables · frame semantics

outbox_event holds event_id, aggregate_type, aggregate_id, aggregate_version, event_type, payload, status ('pending','published','dead'), attempts, created_at, published_at. A projection is missing rows and you must decide whether the relay skipped events or the consumer dropped them. Write three queries over the last seven days: one listing every aggregate_id whose published aggregate_version sequence has a hole, one giving per-day counts with a running total, and one returning the newest published event per aggregate. For each, say where the window function is evaluated relative to WHERE and LIMIT. PostgreSQL 16.

Approach

Gaps: compute lead(aggregate_version) OVER (PARTITION BY aggregate_id ORDER BY aggregate_version) in a subquery, then filter next_version <> aggregate_version + 1 in the outer query. Window functions are evaluated after WHERE, GROUP BY and HAVING and before the outer ORDER BY and LIMIT, so the predicate cannot sit in the same WHERE clause and PostgreSQL 16 has no QUALIFY.
Say what the seven-day filter does to the answer: it truncates every partition, so the first row per aggregate has no predecessor inside the window and a hole spanning the boundary is invisible. Widen the window, or join to resource.version as the authority for the true maximum.
Running total: SELECT date_trunc('day', created_at) AS d, count() AS n, sum(count()) OVER (ORDER BY date_trunc('day', created_at) ROWS UNBOUNDED PRECEDING). An aggregate inside a window call is legal because grouping runs before windowing. The grouping key is unique per row here so ROWS and RANGE agree, but write the frame anyway — over ungrouped rows with tied timestamps the default RANGE frame pulls in every peer row and the total jumps.
Newest per aggregate: DISTINCT ON (aggregate_id) ... ORDER BY aggregate_id, aggregate_version DESC is the cheap PostgreSQL-only form when an index matches that order; row_number() OVER (PARTITION BY aggregate_id ORDER BY aggregate_version DESC) = 1 is the portable form and needs a subquery for the same evaluation-order reason as the gap query.
Interpret rather than report: no gaps plus a normal p95 of published_at - created_at points at the consumer; gaps or a fat lag tail point at the relay; rows still 'pending' with attempts > 0 point at neither, because they never left the database.
Be explicit that the partial index on (created_at, event_id) WHERE status = 'pending' does not serve any of these — they read published rows. Name the index a recurring monitor would need, and say why a query run twice a year may not deserve one.

Worked solution (30 min)

Write the three queries against seven days of data and confirm each returns without error.
In a scratch copy, delete one middle event for a single aggregate and confirm the gap query names that aggregate and the versions either side.
Run a running total over ungrouped rows ordered by date_trunc('second', created_at), once with the default frame and once with ROWS, and record where the two series diverge.
Compare the DISTINCT ON and row_number() plans on the same data and record rows-read for each.

Expected result: The gap query returns one row per hole with the version on each side; the daily query returns seven rows whose running total ends at the seven-day count; the newest-per-aggregate query returns exactly one row per aggregate_id present in the window. The RANGE running total is higher than the ROWS one at every tied second.

Verify it yourself

The per-day counts sum to COUNT(*) over the same predicate.
Re-running the gap query after reinserting the deleted event returns zero rows.
count(*) of the newest-per-aggregate result equals count(DISTINCT aggregate_id) over the same filter.

Follow-up

Relay failover redelivers events. Does a duplicate break the gap query, and how would you detect one from this table alone?
Turn the gap check into a continuous monitor rather than a query someone runs after an incident. What does it watch?
The consumer claims it never received event 4,812,006. What do you look at, in what order?

Stop tag and share joins from fanning out a page

Medium · join fan-out · semi-joins · many-to-many

resource_tag is (resource_id, tag_id) with PK (resource_id, tag_id); resource_share is (resource_id, shared_with_user_id, permission). The tagged-and-shared listing inner-joins resource to both, filters tenant_id, tag_id = ANY($2) and shared_with_user_id = $3, orders by updated_at DESC and takes 50. Pages come back with fewer than 50 distinct resources and the total in the header is far too high. Explain the row multiplication, rewrite both the page query and the count query so each is correct, and name the index each one needs. PostgreSQL 16.

Approach

Do the arithmetic against the predicates that are actually there. An inner join emits one row per matching child row, and both joins are filtered: tag_id = ANY($2) admits only the requested tags, shared_with_user_id = $3 admits one user's share rows. So a resource holding three of the requested tags and shared with $3 once yields three rows, not one — the multiplier is its count of matching tags times its share rows for that single user, and that second factor is 1 unless the table admits duplicate (resource_id, shared_with_user_id) pairs. LIMIT 50 then limits rows rather than resources, and COUNT(*) counts pairs — the header is the product, not the population.
Reject DISTINCT as the fix. It deduplicates after the product has been built, so the planner must materialise and sort the fanned-out set before the LIMIT can apply, and it leaves any SUM or AVG in the same select list wrong.
Rewrite both filters as semi-joins, keeping resource as the only row source: AND EXISTS (SELECT 1 FROM resource_tag rt WHERE rt.resource_id = r.resource_id AND rt.tag_id = ANY($2)) and the same shape against resource_share. A semi-join stops at the first match per resource and preserves the driving index order, so ORDER BY updated_at DESC, resource_id DESC LIMIT 50 still stops after 50 rows.
Count with the same predicates and no join at all: SELECT count(*) FROM resource r WHERE r.tenant_id = $1 AND r.status = 'active' AND EXISTS (...) AND EXISTS (...). Nothing multiplies a resource, so the number is the population.
Attach the tags for display after the page has been cut — LEFT JOIN LATERAL (SELECT array_agg(rt.tag_id) FROM resource_tag rt WHERE rt.resource_id = p.resource_id) ON TRUE over the 50 returned rows. Aggregate over the page, never over the tenant.
Index both directions and say which query each serves: PK (resource_id, tag_id) serves the lateral lookup, (tag_id, resource_id) serves the EXISTS probe by tag, and resource_share needs (shared_with_user_id, resource_id) for the same reason. An index covering one direction only leaves the other as a scan.

Follow-up

The filter changes from 'any of these tags' to 'all of these tags'. Rewrite it and state what it costs relative to the ANY form.
A resource can be shared with the same user twice under different permissions. Does your count change, and should it?
Where does the correct total come from when the tenant holds 4M resources and the header must not cost 200 ms?

Discuss how you would implement a recommendation system.

Medium · system design

Approach

Fix the scope first: who calls this, how often, and what they do when it fails.
State the consistency you need, and where you are willing to be stale.
Choose a partition key and say what query it makes expensive.

Follow-up

How does this behave when that dependency is down for an hour?
What breaks first when traffic grows ten times?

Can you explain the differences between supervised and unsupervised learning?

Medium

Approach

Clarify what is being asked and what a complete answer contains.
State your assumptions explicitly before working the problem.
Work from the requirement backwards to the design.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Design the bulk write endpoint a migration script retries blindly

Medium · batch api · partial failure · idempotency

A customer's migration script pushes 2 million resources through POST /v1/resources:batch, up to 500 items per call, and retries any call that errors or times out. Within a call some items fail validation, some collide with rows that already exist, and some succeed. Specify the request and response shape, whether a batch is atomic or per-item, how idempotency works for the call and for each item, the status code for a mixed outcome, the size and item-count limits with their error codes, and exactly what the script does after a timeout mid-batch.

Approach

Choose atomicity deliberately and price it. All-or-nothing means one transaction holding locks for the whole batch on a primary already absorbing about 1.2k writes/second, which bounds batch size by lock duration, and it turns one bad row into 499 rejections the script must resend. Per-item partial success is the right default for a migration, and the contract's job is then to make a partial outcome impossible to miss.
Require a client-supplied id on every item and echo it in every result. Deriving a per-item key from the array index breaks the first time the script resends a batch with the failures removed: the indices shift, previously-succeeded items acquire new keys, and they are created a second time.
Key the effects at two levels. The call's Idempotency-Key covers an exact resend of the same bytes; per-item keys of (tenant_id, client_item_id) make a partially-applied batch safe to resend whole. Resending an identical batch must reproduce the same per-item results, not 500 conflicts the script has to interpret.
Answer a mixed outcome with one status plus per-item detail: 200, or 207 borrowed from WebDAV if you prefer it - document whichever you pick - carrying an array of client_item_id, per-item status, and either resource_id or an error code from the same taxonomy the single-item endpoint uses. Reserve 4xx for the request as a whole: unparseable body, too many items, payload over the limit (413). Put a failed count at the top level so that even a script checking only the cheapest thing cannot conclude success while rows were dropped.
Cap the request before doing any of it: item count, total bytes, and concurrent batches per tenant, since one tenant's migration otherwise consumes write capacity everyone shares. Anything that cannot finish inside the request deadline belongs on job_run behind a 202, not in a synchronous call that will time out halfway.
Script behaviour after a timeout: the outcome is unknown and no results were received, so resend the identical batch with the same keys and read the results. Never resend 'only the items I have no result for' - a timeout yields no results at all, and that rule silently means resend everything anyway.

Worked solution (30 min)

Write the request schema with the per-item client id, and the response schema with per-item status and a top-level failed count.
Write the atomicity decision and the sentence of justification that names the lock cost or the resend cost.
Define both key levels and trace a resend of a half-applied batch through them, item by item.
List the whole-request rejections with their status codes and limits.
Write the script's timeout rule and the pacing it should apply between calls.

Expected result: A per-item batch with client-supplied item ids, per-item idempotency plus a call-level key, a mixed-outcome response that a status-only check cannot misread, explicit request caps, and a resend rule that is correct when no results were received.

Verify it yourself

Resending an identical batch twice leaves exactly the same set of resource rows as sending it once.
Resending a batch with the successful items removed does not create duplicates of anything.
A caller that reads only the top-level fields can still tell that some items failed.

Follow-up

At 500 items per call, what is the wall-clock time for 2 million rows, and what pacing do you publish so the migration does not become an incident?
One item in every batch fails with the same code. How does the script discover that without a human reading logs?

Read latency spikes on a sixty-second sawtooth

Medium · cache stampede · single-flight · ttl · tail latency

The cached listing read path serves about 14k reads/second at an 85% hit rate. p99 sits at 35 ms for 57 seconds, jumps to 900 ms for 3, and repeats. During each spike the primary shows several hundred identical listing queries starting within the same millisecond, all carrying one large tenant's id. Cache entries use a 60-second TTL. Give the mechanism, the ordered checks, the fix, and the correctness hazard your fix must not introduce.

Approach

Match the period to a configured number before theorising about load. A spike every 60 seconds against a 60-second TTL is an entry expiring, and you confirm it by correlating spike timestamps with the entry's write time rather than with the traffic curve. If the period had matched a cron or a GC interval instead, this is a different investigation.
Establish the concurrency of the miss. Several hundred identical queries in one millisecond means the miss path has no coalescing: every request that arrives between expiry and repopulation recomputes. The herd size is that key's arrival rate times its recompute time, so at 1.2k reads/second for the hot key and a 250 ms recompute you expect about 300 concurrent misses, which matches what is observed.
Add single-flight on the miss path so one caller per key recomputes under a short-lived lock while the rest wait for its result. Prefer stale-while-revalidate where the read tolerates it: return the expired value immediately and refresh asynchronously, which removes the latency spike rather than serialising it into a queue of waiters.
De-synchronise the keys. Write TTLs with jitter, for example 60 seconds plus or minus 10%, so a deploy or a mass invalidation does not align every key on the same second and turn a per-key herd into a fleet-wide one.
Name the hazard the fix must not introduce. Serving a stale listing is acceptable only because the API reports the projection watermark, and a reader that loaded the old value before a write can repopulate the entry after the invalidation, so the bounded TTL is what actually caps staleness rather than the delete. Keep read-after-write pinned to the primary for the writing session regardless.
Verify on miss concurrency, not hit rate. The hit rate barely moves, because the herd is one miss multiplied; the number that must change is distinct origin queries per key per minute.

Follow-up

The same sawtooth appears on a key that is invalidated on write rather than expired. Is that the same bug?
How does your answer change if the recompute takes 4 seconds instead of 250 ms?
What exactly does a client see during a stale-while-revalidate window, and how does the watermark let them tell?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

The failure that was yours

Keep one story where the bad call was yours rather than a dependency's or a manager's. Name the check that would have caught it, whether you added that check afterwards, and whether it has fired since. Answers that route blame outward end the conversation early; answers that end in a guardrail someone still relies on tend to open it up.

Describe your experience with containerization and orchestration tools like Docker and Kubernetes.

Medium · behavioural and collaboration

Approach

Close with what you would do differently, concretely.
Give the blast radius: what could have broken, and what you measured.
State the situation in two sentences and spend the rest on the reasoning.

Follow-up

What did you decide not to do, and why?
How did you know your change caused the improvement?

Unblock an engineer without taking the keyboard

Easy · mentoring · leases · at-least-once

A teammate has spent two days on a job handler that occasionally writes duplicate rows. They are certain the queue is delivering twice by mistake. You suspect a lease expiring under a slow handler, so the job is running concurrently with itself. Describe how you have unblocked someone in this position: what you asked before offering a hypothesis, what you showed them rather than told them, and what you left them owning. Then say what you would do if their theory turned out to be the right one.

Approach

Ask before diagnosing, and ask for things answerable from data they already have: the attempt count on the job rows that produced duplicates, the handler's observed duration against its lease expiry, and whether the duplicate rows share a natural key that a unique constraint could have caught.
Teach the shape rather than the answer. A lease cannot distinguish a dead worker from a slow one, so a handler that outruns its lease is running twice by design, and deploys deliver the other half by killing handlers mid-run on every rollout. Both of their candidate theories produce identical duplicate rows, which is why the evidence has to come from timings rather than from argument.
Hand over a checklist they execute: a natural key on every write the handler performs so the second copy collides rather than appends, the record of intent written before any external effect, a lease heartbeat while running, and the metric that shows it working.
Keep ownership with them deliberately. Pair on the first write, then step back; if you finish it yourself you have closed one ticket and left the same person stuck on the next redelivery.
Close on the systemic gap that let two days pass, which is usually a missing dashboard for attempt counts or an undocumented at-least-once contract, and fix that rather than only the bug.

Follow-up

How would you distinguish a genuine double-delivery from a lease expiry using only the data already stored?
Their handler calls an external endpoint before recording that it did. What do you tell them to change first?
What do you do the third time the same person brings you the same class of bug?

Reverse your own decision and price the reversal

Medium · reversibility · measurement · migrations

Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.

Approach

State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.
Finish on the process change: the smallest experiment that would have produced the same measurement in a day, and why you did not run it the first time.

Follow-up

What in that decision was irreversible, and did you know it was irreversible when you made it?
How did you tell the people who had already built on top of the original decision?
What do you now measure before committing to a change of this size?