Interview guides · Evernorth Health Services

Evernorth Health Services Machine Learning Engineer Interview Questions & Guide 2026

Evernorth Health Services Machine Learning Engineer interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.

The 60-second brief

The Machine Learning Engineer at Evernorth Health Services plays a pivotal role in enhancing the company's healthcare solutions through the application of advanced machine learning techniques. This position is vital as it directly influences product development, operational efficiency, and the quality of care provided to users. By leveraging machine learning models, you will help drive data-driven decisions that can significantly improve patient outcomes and streamline healthcare services.

Scope in the operational half of the job when the seat carries on-call. Rollout, rollback and what you would put on a dashboard belong inside a design answer rather than after it, and an answer that never reaches them reads as someone who has built systems but not run them.

Focus areas

Detect concurrent edits instead of losing writes
Make every write idempotent under retry
Paginate large result sets with keyset cursors

Overview

In this role, you will contribute to various projects that may include predictive analytics for patient management, natural language processing for communication tools, and deep learning applications for medical imaging. The complexity and scale of the tasks you will engage in will not only challenge your technical skills but also offer you a unique opportunity to impact real-world healthcare challenges. Expect to collaborate with multidisciplinary teams to explore innovative ways to harness data for better health solutions, making this position both critical and rewarding.

The Evernorth Health Services interview process

Evernorth Health Services candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.

What to demonstrate

Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not

How to prepare

Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window

In-Depth Technical Interview

Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.

What to demonstrate

Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply

How to prepare

For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last

Behavioral Interview

This round is deciding whether a change you make without supervision can be allowed to reach production. It is scored on what you knew at the moment you decided, not on how it turned out, so a story that opens with the result and works backwards reads as luck retold as judgement. Say what the options were, what you did not know, what you did to shrink the unknown before committing, and what you accepted as the worst plausible case. The detail that separates answers is a bound: how many users, how much data, and for how long, if you had been wrong.

What to demonstrate

Whether the reasoning you give was available at the time you decided rather than after the result came in, since a story whose deciding evidence arrived later describes an outcome and not a judgement
Whether you can put units on the exposure (users, rows, minutes of degraded service) and whether the containment you chose actually bounded it: a canary bounds the request path it fronts, while a background job writing to a shared table reaches every user regardless of which version served their requests
Whether the reversal path existed before you shipped or was improvised during the incident, and whether it restores state or only stops further damage

How to prepare

For your three largest changes, write down the one thing you would have had to be wrong about for it to fail, and what your best estimate of it was on the day you shipped. If you never held an estimate, that is the gap the follow-up questions will find
Write the undo procedure for one of those changes as it existed at the time, then mark which steps restore data and which only stop new damage. Turning a flag off or reverting a deploy ends the new writes; rows already written come back only from a copy you kept, and a dropped column comes back empty unless something outside the schema holds the values
Rehearse one story from the decision point forward and stop before the outcome, then have someone ask what you would do next. If the story only works with the ending attached, it is an anecdote rather than a decision you can defend

Where candidates lose points

Letting a slow dependency consume unbounded concurrency

The failure that takes a service down is usually not an error but a delay. A dependency answering in thirty seconds instead of fifty milliseconds holds each request's worker or connection six hundred times longer, and since required concurrency is arrival rate times latency, a fleet sized for sixty in-flight requests now needs thirty-six thousand to sustain the same rate - so it queues, and requests whose clients have already abandoned them still occupy resources. Retries make it precisely worse: a policy of three attempts triples the load on a dependency at the exact moment it is least able to serve, which is how one slow dependency becomes an outage of everything sharing that pool. Containment is four specific things - a timeout on every outbound call shorter than the caller's remaining budget, a bounded pool per dependency so one cannot starve the others, backoff with full jitter rather than a fixed delay so retries do not resynchronise, and a circuit that stops sending once the failure rate makes an attempt pointless.

Assuming an isolation level prevents the anomaly you actually have

Isolation levels are named by the SQL standard but implemented differently, so any claim about one is only true of a named engine. PostgreSQL defaults to READ COMMITTED, where every statement takes a fresh snapshot, so two statements inside one transaction can legitimately disagree about the same row. Its REPEATABLE READ is snapshot isolation: it removes non-repeatable and phantom reads but permits write skew, where two transactions each read a set, each conclude their own write is safe, both commit, and the combined result violates a constraint that no single row expresses. Only SERIALIZABLE closes that, and it closes it by aborting a transaction with a serialization failure (SQLSTATE 40001), which means the guarantee is theoretical unless the application has a retry loop. InnoDB's REPEATABLE READ is a different mechanism again - plain SELECTs read a consistent snapshot while locking reads and writes see the latest committed row - so a read-modify-write inside one transaction can act on a value that the transaction's own earlier SELECT never returned.

Quoting amortised or average cost as if it were a worst-case guarantee

Appending to a dynamic array is amortised O(1), but the append that triggers a resize copies every element, and hash lookup is constant only while the hash spreads the actual keys. Say which guarantee you are offering when the caller cares about the latency of one call rather than the total over many.

Answering a debugging question with a guess instead of a bisection

Give a procedure that halves the search space at each step: confirm the symptom reproduces, establish the last known-good version, input or timestamp, then bisect over commits, over the data, or over the layers of the request path. A plausible cause with no way to confirm it is the same move whether it happens to be right or wrong, which is why it scores nothing.

Questions & practice

What considerations would you take into account when deploying a machine learning model in production?

Medium · machine learning fundamentals

Approach

Name the simplest model that could work and what would make you move past it.
Say how you would validate it, and where leakage could enter the split.
State the learning problem: the label, the unit of prediction and how the model is used.

Follow-up

How would you know the model is overfitting?
What changes if the classes are heavily imbalanced?

Discuss a machine learning project you have worked on and the challenges you faced.

Medium · machine learning fundamentals

Approach

Pick the metric from the cost of each error type, not from habit.
State the learning problem: the label, the unit of prediction and how the model is used.
Name the simplest model that could work and what would make you move past it.

Follow-up

Where could label leakage enter this setup?
What changes if the classes are heavily imbalanced?

Discuss trade-offs between batch processing and real-time processing in machine learning applications.

Medium · machine learning fundamentals

Approach

Pick the metric from the cost of each error type, not from habit.
State the learning problem: the label, the unit of prediction and how the model is used.
Name the simplest model that could work and what would make you move past it.

Follow-up

Where could label leakage enter this setup?
How would you know the model is overfitting?

How do you stay updated with the latest advancements in machine learning?

Medium · machine learning fundamentals

Approach

Say how you would validate it, and where leakage could enter the split.
Name the simplest model that could work and what would make you move past it.
State the learning problem: the label, the unit of prediction and how the model is used.

Follow-up

Where could label leakage enter this setup?
What changes if the classes are heavily imbalanced?

Solve a common algorithmic problem and explain your thought process.

Medium · coding and algorithms

Approach

Restate the input: its shape, its size, and what is guaranteed about it.
State the target complexity and say which constraint rules the naive version out.
Walk one small example through your approach before writing the whole thing.

Follow-up

How does this change if the input no longer fits in memory?
What is the worst case, and how likely is it on real data?

Collapse a redelivered event batch into per-aggregate high-water marks

Easy · hashing · at-least-once · aggregation

You drain a batch of up to 5,000,000 events, each (aggregate_id BIGINT, aggregate_version INT, event_type, payload). The log guarantees order within one aggregate only; the batch merges 64 partitions, and a relay failover has redelivered a range, so an older version for an aggregate can appear after a newer one. Given a map of last_applied_version per aggregate, produce the events worth applying, at most one per (aggregate_id, version), plus the count discarded. Target O(n) time. State the memory for 2,000,000 distinct aggregates and what you do when it does not fit.

Approach

One pass, one hash map from aggregate_id to the highest version kept, and a discard counter. An event whose version is at or below last_applied_version for its aggregate is dropped without further work, which is the whole reason the event carries its version rather than a delta. O(n) expected time, O(d) space in distinct aggregates.
Keep the maximum, never the last occurrence. The redelivered range means the final appearance of an aggregate in the batch can be an older version than one seen earlier in the same batch, so last-wins applies stale state over newer state and the projection regresses with no error anywhere.
Cost the memory instead of calling it large: an 8-byte key plus a 4-byte version is 12 bytes of payload, and an open-addressed table held at a 0.7 load factor costs roughly 17 bytes per entry before per-slot metadata, so 2,000,000 aggregates is tens of megabytes in a native layout and several times that in a runtime that boxes both key and value.
If the distinct set exceeds memory, partition on hash(aggregate_id) mod P and reduce each partition independently. Every event for one aggregate hashes to the same partition, so the per-partition result is exact and the merge is concatenation rather than a second reduction.
Reject sorting the batch by (aggregate_id, version) as the default. It is O(n log n) and buys nothing, because max is associative and commutative and needs no ordering; sorting earns its cost only when the downstream consumer must receive the events in order rather than a per-aggregate winner.
Separate the two mechanisms out loud: in-batch deduplication does not make the consumer idempotent, because the same event redelivered tomorrow arrives in a different batch entirely. The projection write itself still has to be keyed on (aggregate_id, version).

Worked solution (20 min)

Write the pass: look up last_applied_version, skip if the event's version is not greater, otherwise upsert into the keep-map only when the incoming version exceeds the version already held, incrementing the discard counter on every skip.
Hand-trace one aggregate whose events arrive as v5, v3, v4, v5 with last_applied_version = 2, and confirm the output holds v5 once while the counter reads 3.
Compute the table footprint for 2,000,000 entries at 12 bytes of payload and a 0.7 load factor, then state the multiplier for a runtime that boxes keys and values.
Add the hash-partitioning fallback and say in one sentence why the per-partition results need no cross-partition merge logic.

Expected result: A single-pass O(n) reduction keyed on aggregate_id that keeps the maximum version rather than the last occurrence, O(d) space with the byte cost stated for 2,000,000 aggregates, a hash-partitioning fallback keyed on aggregate_id, and an explicit statement that batch-local deduplication does not replace a projection write keyed on (aggregate_id, version).

Verify it yourself

Shuffling the batch order changes nothing in the output, which is the test that separates max-wins from last-wins.
Feeding the same batch twice produces an identical keep-set and only doubles the discard counter.
Every emitted event has a version strictly greater than the supplied last_applied_version for its aggregate.

Follow-up

The payload is a patch rather than a snapshot, so applying only the highest version loses the intermediate changes. What changes in your reduction?
How do you detect that version 7 arrived while version 6 was never delivered, and what should the consumer do about the gap?
Two events for one aggregate carry the same version with different payloads. Which one is wrong, and how would you find out?

Replace offset paging on the resource feed with keyset

Easy · keyset pagination · composite index · sort order

resource holds resource_id, tenant_id, owner_user_id, title, body_ref, version, status ('draft','active','archived','deleted'), created_at, updated_at, deleted_at, with an index on (tenant_id, status, updated_at DESC, resource_id DESC). The listing endpoint returns active resources for one tenant, newest update first, 50 per page, today with LIMIT 50 OFFSET n. Tenants reach page 400 and rows are created while they read. Write the keyset query, define what the cursor carries and how it is encoded, and say which part of the index each predicate uses. Assume PostgreSQL 16.

Approach

Name the two failures separately. OFFSET 20000 makes the server produce and discard 20,000 rows, so page cost grows with depth rather than with page size. Independently, any write that changes how many rows sort above the offset moves the window between two fetches, and the direction decides which anomaly you get: an insert lands at the head of updated_at DESC and pushes already-returned rows down past the boundary, so they are returned a second time; a delete above the offset, or a row whose updated_at is bumped above the cursor, pulls rows up and one is never returned at all. Nothing in the response reveals either.
Write the seek: WHERE tenant_id = $1 AND status = 'active' AND (updated_at, resource_id) < ($2, $3) ORDER BY updated_at DESC, resource_id DESC LIMIT 50. The row-value comparison is one index range rather than a disjunction, and both columns are NOT NULL, which is what makes that comparison well defined.
Map each predicate onto the index: tenant_id and status are equality on the leading columns, (updated_at, resource_id) is the range, and the ORDER BY matches the index order so no Sort node appears and the scan stops after 50 rows. The DESC in the definition only matters for mixed directions — a plain ascending btree on the same columns is read backwards for this query.
Put both sort columns in the cursor and nothing the client can tamper with into another tenant: base64 of (updated_at, resource_id), validated server-side, with tenant_id taken from the principal.
State the residual honestly. Keyset is stable against concurrent inserts and deletes, but not against a row whose updated_at changes mid-scroll — that row moves in the ordering and can be seen twice. If the feed must be a snapshot, order by an immutable key or bound the page set with updated_at <= the cursor's start value.
Keep a total out of the page path. A tenant-wide COUNT(*) is the scan keyset just removed; fetch LIMIT 51 and return has_more instead.

Worked solution (25 min)

Seed one tenant with 500k active resources, 2% of them sharing an identical updated_at.
Time LIMIT 50 OFFSET 0 against OFFSET 20000 and record rows-read from EXPLAIN (ANALYZE, BUFFERS) for each.
Page the whole set with the keyset query while an insert-only writer adds 100 rows/second, collecting resource_ids, and repeat the run with OFFSET.
Repeat both runs under a second writer profile that also deletes 20 rows/second from pages already returned and bumps updated_at on 20 more, and diff each collected id set against the rows that existed for the whole run.
Remove resource_id from the cursor so the seek degrades to updated_at < $2, and re-run the tie-heavy section of the feed.

Expected result: Keyset reads about 50 index entries per page and its latency is flat from page 1 to page 10,000; under both writer profiles the collected ids contain no duplicate and omit no row that existed for the whole scroll. OFFSET at page 400 reads 20,050 rows to return 50. Under the insert-only writer it repeats rows and skips none — an insert can only push a row down past the boundary — and only the second profile, whose deletes and updated_at bumps move rows up past the cursor, also makes it skip. Without the tie-break, the seek is updated_at < $2, which drops every still-unread row sharing the boundary timestamp.

Verify it yourself

EXPLAIN shows an Index Scan with no Sort node — a Sort means the ORDER BY is not satisfied by the index.
Buffers read per page stays constant as page depth grows.
Ids collected across the scroll are unique, and complete for rows untouched during the run.

Follow-up

The client asks for 'jump to page 400'. What do you offer instead, and what does the honest version cost?
Sort order becomes user-selectable across four columns. How many indexes is that, and which would you refuse to add?
What does the cursor do when the row it points at has since been deleted?

Hold a per-tenant active cap against concurrent creates

Hard · write skew · isolation levels · constraints

A tenant on the standard plan may hold at most 50 resources with status='active'. The create handler runs SELECT count(*) FROM resource WHERE tenant_id = $1 AND status = 'active', compares to 50, then inserts. Two creates arrive 3 ms apart on different instances and the tenant lands at 51. Name the anomaly, say whether PostgreSQL 16 READ COMMITTED or REPEATABLE READ prevents it and why, then give an implementation that holds the cap at READ COMMITTED with the exact statements. Finally, say what changes when the cap is 'at most one running export per tenant' on job_run.

Approach

Name it: write skew. The two transactions read an overlapping set and write disjoint rows, so there is no row-level conflict for the engine to detect and each commit is individually legal.
Rule out the levels precisely. READ COMMITTED takes a fresh snapshot per statement and takes no lock on the counted rows, so both see 49. PostgreSQL's REPEATABLE READ is snapshot isolation: it removes non-repeatable reads and phantoms within the snapshot but still admits write skew, because the anomaly is not a re-read of a changed row, it is a read of a set that a concurrent transaction invalidates. Only SERIALIZABLE closes it, by tracking the read dependency and aborting one transaction with SQLSTATE 40001 — a guarantee that exists only if the application re-runs the whole transaction from the read.
Convert the set predicate into a single-row conflict: keep tenant.active_resource_count and run UPDATE tenant SET active_resource_count = active_resource_count + 1 WHERE tenant_id = $1 AND active_resource_count < 50 in the same transaction as the INSERT. Zero affected rows is the cap, returned as 409. The row lock serialises the decision at any isolation level, and contention is bounded to one tenant's row — which is also the fair-scheduling unit, unlike a global counter that would convoy every tenant behind one row.
State the cost you just took on: a counter is a second source of truth that can drift, so every path that changes status must adjust it inside the same transaction, and a periodic reconciliation has to exist, with resource_revision as the authority for what the count should have been.
For the job case the invariant is expressible per row, so let the database hold it: a partial unique index on job_run (tenant_id, job_type) WHERE status IN ('queued','running') makes a second running export unwritable and the loser takes 23505, mapped to 409. That is strictly better than a counter — no drift, no reconciliation — and it is available only because the cap is one rather than fifty.
Add the retry discipline each route demands: under SERIALIZABLE both 40001 and deadlock 40P01 are retryable and the retry must re-execute the read, while under READ COMMITTED with the counter nothing retries, because the conflict is reported to the caller rather than raised as an error.

Follow-up

A resource moves from archived back to active. Which statements change, and what breaks if the counter update and the status change land in different transactions?
The cap becomes plan-dependent and a plan can change mid-month. Where does the number 50 live, and who reads it?
How do you detect after the fact that the counter drifted, without locking the table?

How would you design a recommendation system for a healthcare provider?

Medium · ml system design and production

Approach

Fix the product goal and the online metric before choosing any model.
Separate the offline training path from the online serving path.
Say where features come from at serving time and how they match training.

Follow-up

How would you detect drift before the metric drops?
What happens when a feature is missing at serving time?

How would you design a system to monitor the performance of machine learning models post-deployment?

Medium · ml system design and production

Approach

Fix the product goal and the online metric before choosing any model.
Say where features come from at serving time and how they match training.
Name what you would monitor after launch and what triggers a retrain.

Follow-up

What happens when a feature is missing at serving time?
How would you detect drift before the metric drops?

Describe the architecture of a scalable machine learning pipeline.

Medium · system design

Approach

Choose a partition key and say what query it makes expensive.
Name the read and write paths separately; they rarely have the same bottleneck.
Name the failure you are designing for, then the recovery path.

Follow-up

What would you drop to keep the system up under load?
What breaks first when traffic grows ten times?

How would you approach a project where the requirements are not clearly defined?

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer contains.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Publish rate-limit and deadline semantics the edge actually enforces

Easy · rate limiting · backpressure · retries

The edge API serves about 3k requests/second steady and 9k at peak against a 400 ms p99 budget, with an explicit bounded concurrency limit per instance. Limits exist per principal and per tenant. Callers are a partner integration running nightly bulk loads and a browser app. Specify the counting algorithm and window, which limit a request is charged against, the headers a well-behaved client reads, the status and body when a limit is hit, how that differs from the response when an instance is shedding load, and what each caller does with each.

Approach

Choose the counter and name its failure mode. Fixed windows admit nearly twice the limit across a boundary - a full burst at the end of one window and another at the start of the next. A token bucket states sustained rate and burst separately, which is exactly what a nightly bulk load needs. A sliding-window counter is more faithful and costs more state per key. State the choice and the burst it permits.
Charge each request against both keys and reject on the stricter. The tenant limit protects the shared primary, which absorbs roughly 1.2k writes/second in total; the per-principal limit stops one credential inside a tenant from consuming that tenant's whole allowance. The tenant is the fairness unit for the same reason it is the leading column of every index.
Advertise limit, remaining and reset for the binding key on every response, not only on rejections, so a client can pace before it is refused. Pick one naming scheme - the RateLimit-* draft fields or an X-prefixed set - document the units, and never change them afterwards.
Separate two rejections that look identical to a naive client. 429 means this caller exceeded its own share and Retry-After is a real schedule it should obey. 503 means the instance is at its concurrency bound and shedding, which is a statement about the server; a fleet-wide 503 retried on a fixed delay resynchronises every client into one stampede, so full jitter is mandatory there and the delay is the client's guess, not ours.
Make shedding cheap and early - before the token is verified against the database, before any downstream call - because a rejection that costs as much as the work relieves nothing. Drop requests whose client deadline has already elapsed rather than serving them; the caller has stopped listening and the work is pure cost.
Write the caller behaviours down: the bulk loader paces against remaining and treats a 429 as a defect in its own pacing; the browser surfaces the wait and must never retry a 429 inside a render loop, which turns one limited user into a self-inflicted flood.

Worked solution (20 min)

Write the bucket parameters for both keys: sustained rate, burst size, and the refill interval, with the arithmetic that ties them to the 3k/9k figures.
Draft the three response headers and one example 429 body carrying a code, the limit that bound, and Retry-After.
Write the 429-versus-503 decision as a two-line rule an on-call engineer can apply to a log line.
State where in the request pipeline the rejection happens and which work it skips.

Expected result: A token bucket per principal and per tenant with named burst, headers on every response, 429 with Retry-After for share exhaustion, 503 with jittered client backoff for concurrency shedding, and rejection placed before authentication's database read.

Verify it yourself

A client that respects remaining never sees a 429 under steady traffic at its configured rate.
The cost of a shed request is bounded and does not include a database round trip.
Peak throughput times the limiter's per-request state cost is affordable at 9k requests/second.

Follow-up

One tenant stays under its limit and still degrades everyone else during a backfill. What changes - the limiter, the worker concurrency caps, or both?
How are counters kept correct across 20 to 40 stateless instances, and what does your answer cost per request?

Read latency spikes on a sixty-second sawtooth

Medium · cache stampede · single-flight · ttl · tail latency

The cached listing read path serves about 14k reads/second at an 85% hit rate. p99 sits at 35 ms for 57 seconds, jumps to 900 ms for 3, and repeats. During each spike the primary shows several hundred identical listing queries starting within the same millisecond, all carrying one large tenant's id. Cache entries use a 60-second TTL. Give the mechanism, the ordered checks, the fix, and the correctness hazard your fix must not introduce.

Approach

Match the period to a configured number before theorising about load. A spike every 60 seconds against a 60-second TTL is an entry expiring, and you confirm it by correlating spike timestamps with the entry's write time rather than with the traffic curve. If the period had matched a cron or a GC interval instead, this is a different investigation.
Establish the concurrency of the miss. Several hundred identical queries in one millisecond means the miss path has no coalescing: every request that arrives between expiry and repopulation recomputes. The herd size is that key's arrival rate times its recompute time, so at 1.2k reads/second for the hot key and a 250 ms recompute you expect about 300 concurrent misses, which matches what is observed.
Add single-flight on the miss path so one caller per key recomputes under a short-lived lock while the rest wait for its result. Prefer stale-while-revalidate where the read tolerates it: return the expired value immediately and refresh asynchronously, which removes the latency spike rather than serialising it into a queue of waiters.
De-synchronise the keys. Write TTLs with jitter, for example 60 seconds plus or minus 10%, so a deploy or a mass invalidation does not align every key on the same second and turn a per-key herd into a fleet-wide one.
Name the hazard the fix must not introduce. Serving a stale listing is acceptable only because the API reports the projection watermark, and a reader that loaded the old value before a write can repopulate the entry after the invalidation, so the bounded TTL is what actually caps staleness rather than the delete. Keep read-after-write pinned to the primary for the writing session regardless.
Verify on miss concurrency, not hit rate. The hit rate barely moves, because the herd is one miss multiplied; the number that must change is distinct origin queries per key per minute.

Follow-up

The same sawtooth appears on a key that is invalidated on write rather than expired. Is that the same bug?
How does your answer change if the recompute takes 4 seconds instead of 250 ms?
What exactly does a client see during a stale-while-revalidate window, and how does the watermark let them tell?

Suggested practice checklist

For someone fluent in a dynamic language who has shipped real work but has never had to say what the runtime is doing underneath. The week is built on measuring and deliberately breaking things, because the questions that expose this background are the ones where the interviewer asks why a second time.

Day 1 — Measure before reasoning

Take a slow piece of your own code, write down in advance where you believe the time goes, then profile it and record how wrong the guess was. The cost is usually an allocation you did not notice or an accidental quadratic membership test.
Replace one list membership test inside a loop with a set and measure at a thousand, ten thousand and a hundred thousand elements, confirming the shape of the curve rather than only that it got faster.
Write down the three quantities you can now measure instead of assert: wall time, peak memory, and call count for the function you suspected.

Deliverable: A before-and-after profile of real code plus a written note on the size of the gap between the guess and the measurement.

Day 2 — References, copies, and the bugs they produce

Write the function with a mutable default argument, call it three times, and explain the accumulating result: the default is evaluated once when the function is defined, so every call shares one object.
Build a nested structure, take a shallow copy, mutate an inner element, and show that both views changed, because a shallow copy duplicates the container and not the elements. Then fix it with a deep copy and state the cost you just accepted.
Write two functions, one mutating its argument in place and one rebinding the local name, and predict the caller's view of each before running it. That single distinction produces most of the bugs that pass their tests.

Deliverable: Three small programs whose output you predicted correctly before running, each with a one-line statement of the rule underneath.

Day 3 — Types, once, in a language that checks them

Port one module you have already written, roughly a hundred lines, into a statically typed language, and record every place the compiler demanded an answer your original had left implicit: a value that can be absent, a numeric width, a case never handled.
Write the same signature in both languages and state what the static one guarantees before the program runs and what it does not, since it will not save you from a wrong algorithm or an index out of range.
Write the difference between an interface satisfied by declaration and one satisfied structurally, with one case each where the other approach would miss the mistake.

Deliverable: One module in two languages plus a list of the questions the type checker forced you to answer.

Day 4 — Concurrency, starting with what actually runs at the same time

Run the same CPU-bound function across four threads and four processes and measure both. Under the default CPython build the threaded version will not speed up, because only one thread executes bytecode at a time; the process version will. Check which build you are on first, since free-threaded builds remove that lock and change the result.
Then run a blocking I/O workload across four threads and measure it speeding up, because the interpreter releases that lock around blocking calls, which is why treating threads as useless is wrong as a general claim.
Build the lost update: two threads each incrementing a shared counter a hundred thousand times, and show a final value below the expected sum, because an increment is a load, an add and a store and the thread can be suspended between them. Fix it with a lock and then measure what the lock costs.

Deliverable: Three measurements, threads against processes on CPU work, threads on I/O work, and a demonstrated lost update, each with the mechanism written underneath.

Day 5 — Debugging as a procedure rather than an instinct

Work one real failure as a bisection: find a revision or an input size where it is good and one where it is bad, halve repeatedly, and state the two assumptions bisection needs, that the property changes exactly once across the range and that the test is reliable.
Minimise one failing input to the smallest version that still fails, and record how many rounds it took.
Keep a hypothesis log for one bug in three columns, what I believe, what would disprove it, what I observed, and stop yourself the first time you are about to change two things at once.

Deliverable: One bug worked to root cause with a written hypothesis log and a minimised reproducing input.

Day 6 — Tests that catch the bug you are about to write

Implement an LRU cache with a capacity bound, then write the three test cases that would catch an off-by-one in eviction: insert exactly capacity items and assert nothing was evicted, insert one more and assert the least recently used key is the one gone, and read an old key just before that insert so the eviction victim changes.
Add a property test comparing your implementation against a deliberately slow reference, an ordered list scanned linearly, over a few thousand random operation sequences, because a slow reference finds the cases you would not have thought to write.
Write one numeric test that fails under exact equality and passes with a tolerance, and state why the tolerance has to be relative rather than absolute once the magnitudes grow.

Deliverable: An LRU implementation with three boundary tests, one property test against a slow reference, and one tolerance-based numeric test.

Day 7 — Debug something broken, out loud

Have someone plant three defects in a two-hundred-line program, an off-by-one, a shared mutable state bug, and a wrong error-handling path, then find them while narrating, under a fixed rule: state the hypothesis before touching anything.
Time each one and record which tool found it, reading, a printed value, a debugger, or a test, because the question asked in interviews is how you would find it rather than what it was.
Write the sentence you will use when you do not yet know the cause, one that names the next measurement instead of offering a guess.

Deliverable: A recorded debugging session with time-to-find per defect and the method that found each.

What the behavioural round is actually measuring

Every story you tell gets read for blast radius and judgement: what could have broken, who else it touched, what you knew at the moment you decided. Nobody can audit your code in an hour, so they audit your reasoning instead. Pick work where the call was genuinely yours and the consequences were real enough to remember.

What do you do when you encounter a problem you cannot solve immediately?

Medium · behavioural and collaboration

Approach

State the situation in two sentences and spend the rest on the reasoning.
Close with what you would do differently, concretely.
Name the disagreement and how you resolved it with evidence.

Follow-up

How did you know your change caused the improvement?
What did you decide not to do, and why?

Estimate work you have never done and defend the range

Hard · estimation · backfills · expand-contract

You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.

Approach

Decompose into independently deployable steps before estimating anything: add the column nullable, write both shapes, backfill in batches, verify, move reads, stop writing the old shape, drop it. That is four deploys spread over days, and the calendar estimate is dominated by them rather than by the loop's runtime.
Do the arithmetic aloud for the part that has arithmetic in it: batch size times number of batches times per-batch duration, at a write rate the primary can absorb alongside roughly 1.2k writes per second of production traffic. The loop is throttled by replication lag and lock waits, not by how fast it can issue statements.
Price the schema step by its lock rather than its statement duration. In PostgreSQL an ALTER TABLE taking ACCESS EXCLUSIVE waits for every open transaction on that table while later queries queue behind it, so a millisecond change issued during a thirty-second analytics query stalls that table for thirty seconds. Adding a nullable column with a non-volatile default avoids a rewrite from version 11; a new index wants CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an invalid index behind if it fails.
Express the answer as a range whose endpoints each trace to a stated assumption, then name the cheapest experiment that collapses it, which is almost always running one real batch against the real table and multiplying.
Commit to a checkpoint rather than a completion date: the day you report a measured number from that first batch. That is a promise you can keep under uncertainty, and it is what the asker actually needs in order to plan.

Follow-up

How do you verify the backfill genuinely finished, given rows written by production traffic while it ran?
Where does the backfill resume from after a worker is killed mid-batch, and what makes that resume point trustworthy?
Your first batch comes back ten times slower than assumed. What do you tell the person waiting on the estimate, and when?

Ship under a deadline and bound the debt you chose

Medium · pagination · technical debt · tradeoffs

You have four days to ship a tenant-facing listing endpoint. The version you would defend uses keyset pagination over (tenant_id, status, updated_at DESC, resource_id DESC); the version you can finish uses LIMIT/OFFSET with no matching index. Describe a deadline call you actually made of this shape: what you shipped, what you knowingly deferred, how you bounded the damage with a mechanism rather than an intention, and the specific numeric condition that would force the follow-up. Name who you told and where you wrote it down.

Approach

Name the deferred failure precisely instead of calling it slow. OFFSET n makes the database produce and discard n rows, so cost grows with page depth; without an index matching the sort, every matching row is read and sorted before the limit applies; and rows inserted between two page fetches shift across the boundary so items are skipped or repeated with nothing in the response to signal it.
Bound the blast radius with something mechanical rather than a promise: cap maximum page depth, cap page size, restrict the endpoint to one internal caller, or keep it behind a flag. State which failure each cap removes and which it leaves standing.
Attach a number to the trigger and wire it to an alarm: the first tenant crossing N resources, or the endpoint's p99 crossing its share of the 400 ms budget, so the debt announces itself instead of waiting to be remembered.
Write it where the next engineer looks, which is the code and the ticket, not a chat message: what was deferred, why, the cap, and the trigger.
Report what actually happened in your real example, including the case where the trigger never fired and the debt was correctly never repaid.

Follow-up

At what page depth does the offset version breach your latency budget, given your page size and row counts?
What breaks first when you switch to keyset pagination later, and what does a client holding an old page token see?
Who would have overruled you if you had asked for two more days, and did you ask?