Interview guides · Experian

Experian Software Engineer Interview Guide

API design practice for Experian software engineer candidates: compatibility, versioning, and changes you cannot take back.

The 60-second brief

A Software Engineer at Experian plays a central role in driving the technology behind one of the world's leading global information services companies. Engineers at Experian build, maintain, and scale high-throughput data processing platforms, consumer credit decision engines, microservices architectures, and client-facing web applications. Operating at the intersection of finance and technology, your work directly affects millions of individuals and thousands of businesses relying on accurate, real-time data for critical financial decisions.

The title spans product, platform and infrastructure work, and which of those the seat actually is decides whether design or algorithms carries more weight in your preparation. The posting rarely settles it; what the team is on call for usually does.

Focus areas

Bound blast radius with per-tenant concurrency limits
Make every write idempotent under client retries
Scope every query and cache key by tenant

Overview

The modern tech stack across Experian engineering teams relies heavily on robust backend services written in Java or Python, event-driven microservices, cloud-native deployments, and responsive frontend frameworks like React. Engineers solve complex engineering challenges surrounding massive data volume, low-latency API communication, strict security standards, and high-availability system design. Whether you are modernizing core data pipelines, building analytics platforms, or implementing enterprise customer portals, your solutions must balance velocity with strict precision.

Joining Experian offers the opportunity to work across global engineering hubs—including major teams in North America, the UK, Malaysia, and India—while solving high-impact engineering problems. Candidates entering this role can expect a collaborative, agile culture that values foundational computer science knowledge, clean code, database proficiency, and effective cross-functional communication.

The Experian interview process

Experian candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screening Call

The title covers product work, platform work, infrastructure, mobile and frontend, and those are different jobs with different loops behind them. A screening call is the cheapest place to find out which one the seat is, and asking reads as experienced rather than fussy. The questions that separate them: what the team is on call for, what the last three projects were, and whether any round happens inside an existing repository instead of a blank file. Then say which of that you have done and which you have not. Claiming the whole posting is the fastest way to be found out one round later.

What to demonstrate

Whether you can locate your experience inside one flavour of the role honestly instead of claiming the entire requirements list
Whether you name what you have not done, which an experienced screener reads as a level signal and can plan the loop around
Whether what you want next matches what the seat is: someone who wants greenfield work landing on a team that mostly operates an existing system is a hire that leaves within the year

How to prepare

Mark every line of the posting as done, adjacent or new, and write one sentence for each adjacent line naming the closest thing you actually built
Split your last two years into rough percentages across feature work, operating and debugging live systems, and design or review, so a question about scope gets numbers rather than adjectives
Bring three questions that discriminate between seats: what the team is paged for, how much of the work is changing existing code versus standing up something new, and what shipped in the last quarter

Pre-recorded Video Interview

An unlabelled round is first an information problem, and the cheapest information is free. Whoever schedules it can usually tell you how long it runs, who will be in the room and what they work on, whether you will be writing code and in what environment, and whether anything is being sent beforehand. Ask in writing so the answer is on record, then prepare for the two or three formats those answers still leave open instead of betting on one. What separates a strong candidate is not guessing right; it is having an opening that works whichever one it turns out to be.

What to demonstrate

Whether you can start work from an ambiguous brief, since tolerating a vague scope without stalling is the same thing the job asks for
Whether the questions you asked beforehand were ones that change your preparation, such as duration, medium and who is joining, rather than ones whose answers you could not have acted on
Whether you adapt when the round turns out to be something other than what you were told, instead of spending the first ten minutes visibly recalibrating

How to prepare

Send one short scheduling message asking four things: how long, who is joining and what they work on, whether you will be writing code and where, and whether to prepare anything in advance. Treat a vague reply as real information, since it means the round is loosely structured and you will be shaping it yourself.
Write one opening that works in any of the formats still open: restate in your own words what you have been asked to do, then ask which of two directions is more useful to them. Say it aloud until it stops sounding recited.
Set up for the two most likely formats before the call starts, with a blank editor in the language you would choose and a shared document you can type into, so a format surprise costs you nothing in the first minutes

Live Technical Interviews

What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.

What to demonstrate

Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly

How to prepare

Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live

Managerial Fit Round

When a round has no standard shape, it is often there because something is still open: an area no earlier conversation reached, a round where the signal came out mixed, or a decision someone is not ready to make alone. Work out which by going back over what each earlier round actually covered rather than how it felt, and arrive able to give evidence on that point without being asked twice. Weak answers replay the loop's earlier material at the same depth. Strong ones go a level deeper and stay consistent with what you already said.

What to demonstrate

Whether your account of a project matches the one you gave earlier in the loop, since what you said before may be available to whoever runs this round
Whether you can go a level deeper on something already covered, reaching the decision and its alternatives rather than repeating the summary
Whether you state your own uncertainty accurately, including parts of a system you did not build and decisions you inherited, instead of claiming even ownership across all of it
Whether you can answer a question you handled poorly earlier by naming what you missed, rather than delivering a polished second version as if the first had not happened

How to prepare

Reconstruct the loop on one page: for each round, the questions you were asked and the answer you actually gave, not the better one you thought of afterwards. The gaps on that page are your best available guess at why this round exists.
Take the two claims you made earlier that carry the most weight and assemble the backing for each: the measurement, the date, what broke, the decision you would make differently now.
Write down the three facts about your work that must not drift between tellings, such as team size, timeline and your own role, and check your stories against that list rather than trusting recall under pressure

Where candidates lose points

Serialising a tenant's writes through select ... for update on a single counter row

It is the first change that makes a counter correct, and it caps that tenant's write throughput at roughly one divided by the lock hold time. A transaction that takes the lock, makes a network call and then commits holds it for the entire round trip: at 2 ms that is about 500 writes per second for the whole tenant, and the largest tenants are exactly the ones that exceed it. The damage then spreads, because every waiter holds a database connection while it queues, so one hot tenant drains the shared pool and the symptom presents as a site-wide latency incident rather than as a lock problem. The repairs are to shrink the critical section to a single statement, to shard the counter into per-(tenant, hour) or per-(tenant, bucket) rows and sum on read, or to batch in memory and flush periodically while accepting the bounded loss that batching implies.

Paginating a growing table with limit and offset

Two unrelated defects share the idiom. Correctness: rows inserted or deleted between page requests shift the window, so a consumer walking an export skips rows and sees others twice, which for a customer-facing sync is silent data loss rather than an error anyone notices. Cost: the database still produces and discards the skipped rows, so page N costs time proportional to N times the page size and a deep page on a large table degrades from milliseconds to seconds. Keyset pagination over a stable, unique, indexed ordering -- where (created_at, id) < ($1, $2) order by created_at desc, id desc limit $3 -- is constant-cost per page and immune to shifting, on the precondition that the cursor columns never change value for a row, which disqualifies updated_at as a cursor.

Designing for a scale nobody asked for

Ask for request rate, data size, read-to-write ratio and expected growth, then size the simplest option first; one relational instance on current hardware covers a large share of real workloads. Reaching for shards, queues and a cache tier before any number has been quoted reads as pattern-matching rather than judgement.

Trusting input because it came from your own front end

Anything crossing a trust boundary is hostile: parameterise queries instead of building SQL by concatenation, validate against an allow-list rather than a deny-list, and bound the size of anything you allocate from a request. Raising this unprompted in an API or design question is a cheap and unusually strong signal.

Questions & practice

Demonstrate manipulation of core data structures, such as reversing or merging a linked list and managing key-value lookups with hash maps.

Medium · data structures and algorithms

Approach

Name the brute-force solution and its complexity before improving on it.
Restate the input: its shape, its size, and what is guaranteed about it.
Choose the data structure from the access pattern, not from familiarity.

Follow-up

Which test case would catch an off-by-one here?
What is the worst case, and how likely is it on real data?

Solve array manipulation problems requiring sliding window or two-pointer techniques to handle continuous data streams.

Medium · data structures and algorithms

Approach

Walk one small example through your approach before writing the whole thing.
Name the brute-force solution and its complexity before improving on it.
Choose the data structure from the access pattern, not from familiarity.

Follow-up

What is the worst case, and how likely is it on real data?
How does this change if the input no longer fits in memory?

Hold a tenant to a trailing sixty-second request limit

Medium · sliding-window · two-pointer · rate-limiting · tenant-skew

The gateway must hold each tenant to R requests in any trailing 60 seconds, in aggregate across three regions and every pod, within a budget of under 10 ms added p99. Peak is 30,000 requests/second across 200,000 active tenants, and traffic is heavily skewed toward a handful of them. Give an exact single-process algorithm with its amortised per-request cost and its memory per tenant, then a bounded-memory approximation and the worst-case overshoot it actually admits. Say what the distributed version does when the counter store is unreachable.

Approach

Exact, single process: a per-tenant deque of request timestamps. On arrival, pop from the front while front <= now - 60s, then admit if the remaining length is below R and push. Each timestamp is pushed once and popped once, so the cost is O(1) amortised. The O(R) version is the one that re-filters the whole deque on every request.
Quote the memory. R = 1,000 across 200,000 active tenants is up to 2 x 10^8 timestamps at 8 bytes, about 1.6 GB, and that is the worst case rather than the mean, because the long tail of small tenants holds almost nothing. Skew helps you here and hurts you in the sharding decision.
Bounded alternative, with its real bound stated: a fixed 60-second counter is O(1) memory but admits close to 2R across a 60-second span straddling a boundary. The weighted two-bucket estimate, prev * (60 - elapsed)/60 + cur, is better on smooth traffic but assumes the previous window's arrivals were uniform; an adversary packing them at the end of that window is undercounted and can still approach 2R. Say that rather than calling it exact.
Token bucket is the usual gateway answer and a different contract: O(1) state per tenant (tokens, last_refill), a sustained rate, and a deliberate burst allowance equal to the bucket size. Choose it when a burst is acceptable and the log when the limit is contractual.
Distributed: the limit is per tenant in aggregate, so a local bucket of R/N per pod is wrong in both directions under skew. A tenant landing on one pod is throttled at R/N, and a tenant spread evenly across pods exceeds R. The shared check must be a single atomic round trip, one script or one increment-and-compare, never read-then-write, and it must fit inside the 10 ms p99 budget.
Decide the unavailable case in advance and write it down. Failing open keeps the product up and lets a tenant exceed its limit for the duration; failing closed converts a counter-store outage into a full outage. Most gateways fail open on rate limits and closed on authorisation, and those are two separate decisions made separately.

Worked solution (25 min)

Implement the deque version and instrument the per-request pop count, then confirm total pops equal total pushes over a run.
Generate a burst that places R requests in the last 100 ms of one minute and R more in the first 100 ms of the next.
Run that burst through the exact deque, a fixed 60-second counter, and the weighted two-bucket estimate, recording admissions in the trailing 60 seconds at every instant.
Size the memory as R x active tenants x 8 bytes at R = 1,000 and 200,000 tenants, and compare it against what a token bucket would need.

Expected result: The exact deque never admits more than R in any trailing 60-second window. The fixed counter admits close to 2R across the boundary. The weighted estimate lands between the two on this burst and approaches 2R once the previous window's requests are packed at its end.

Verify it yourself

Total pops equal total pushes, which is what makes the amortised O(1) claim true rather than merely asserted.
Sliding the whole burst by 30 seconds changes the fixed counter's result and leaves the deque's unchanged.
Running two processes each limiting to R/2, against a tenant that sends everything to one of them, shows that tenant throttled at half its contractual limit.

Follow-up

One tenant sends 40% of all traffic. What does that do to a single counter key, and what do you shard on instead?
Quotas rather than rate limits: the check is select used; if used < limit then insert. Name the isolation level that still permits the overshoot, and the two fixes.
How do you return an accurate Retry-After from the exact algorithm without a second scan?

Migrate a live partitioned event table without blocking ingest

Hard · online migration · ddl locks · expand and contract · partitioning

usage_event is range-partitioned daily on ingested_at, holds roughly 250M rows per day across 400 live partitions, and is written at 10-40k rows/second. Two changes are required: quantity must move from double precision to numeric(20,6), and a new environment column must become NOT NULL with a default of 'production'. Ingest cannot stop. Give the ordered plan, naming for each step the lock it takes, what that lock blocks, and roughly how long it is held. Identify the one step that cannot be rolled back cleanly once traffic depends on it.

Approach

Classify the two changes before planning anything. Adding a column with a non-volatile default has been metadata-only since PostgreSQL 11, so it is cheap. Changing double precision to numeric is not binary-coercible, so alter column ... type rewrites every partition under ACCESS EXCLUSIVE and rebuilds its indexes; on this volume that is hours of blocked ingest and is simply not an option, which is why the plan is expand-and-contract rather than one statement.
Expand: add quantity_numeric numeric(20,6) and environment with its default on the parent. Both are catalogue-only but both take a brief ACCESS EXCLUSIVE that cascades to partitions, so run each with lock_timeout set to a second or two and retry on failure. A queued ACCESS EXCLUSIVE request blocks every reader behind it, which is how a metadata-only change turns into an outage.
Dual-write: deploy producer code that populates both columns on every insert, and leave it running before anything reads the new column. This is the step that cannot be reverted cleanly. Once readers depend on quantity_numeric, reverting the writer leaves rows with a null there, and the gap is only discoverable by re-reading the old column, which the readers have stopped doing.
Backfill older partitions in batches keyed on the primary key, oldest first, committing every few thousand rows with a pause between batches, and skipping the partition still receiving writes until it rotates. Each batch is an ordinary UPDATE taking row locks only. The cost is bloat and WAL rather than blocking, so watch dead tuples and let autovacuum keep pace instead of wrapping 400 partitions in one transaction.
Make NOT NULL cheap with the three-step form: add constraint ... check (environment is not null) not valid (brief ACCESS EXCLUSIVE, no scan), then validate constraint (SHARE UPDATE EXCLUSIVE, scans while reads and writes continue), then set not null, which from PostgreSQL 12 uses the validated check and skips its own full scan. Do this per partition, then on the parent.
Switch and contract: move reads to the new column behind a flag, verify over a full period that both columns agree on freshly written rows, drop the old column (metadata-only), and only then remove the dual-write. Any index on the new column goes on with CREATE INDEX CONCURRENTLY per partition, since CIC is not supported on a partitioned parent: create the parent index with ONLY, build each child concurrently, then ALTER INDEX ... ATTACH PARTITION until the parent index becomes valid.

Worked solution (45 min)

On a scratch cluster, build 10 partitions of 2M rows each and run a writer at a few thousand inserts/second.
Run the naive type change and measure how long writes stall and how far ingest lag grows before killing it.
Run the expand step with lock_timeout = '2s' while the writer runs, and observe a clean lock timeout and retry instead of a pile-up of blocked readers.
Backfill in 5k-row batches and chart dead tuples and WAL generated per batch.
Run the not-valid, validate, set-not-null sequence and confirm from pg_stat_activity and timings that nothing held an exclusive lock through a full scan.
Add an index with CIC per partition plus ATTACH PARTITION and confirm the parent index reports valid only after the last attach.

Expected result: An ordered expand, dual-write, backfill, switch, contract plan with the lock named per step: metadata-only ADD COLUMN under a brief ACCESS EXCLUSIVE taken with lock_timeout; batched UPDATE backfill under row locks; CHECK NOT VALID, then VALIDATE under SHARE UPDATE EXCLUSIVE, then SET NOT NULL; CREATE INDEX CONCURRENTLY per partition with ATTACH PARTITION; and the dual-write-to-read switch identified as the step that cannot be reverted cleanly.

Verify it yourself

No step in the plan holds ACCESS EXCLUSIVE for longer than the configured lock_timeout.
The writer's error rate stays at zero through the rehearsal apart from the deliberate lock timeout.
After VALIDATE, SET NOT NULL completes in milliseconds on a table where the naive form scanned every row.
Freshly written rows agree on both columns for a full verification period before the old column is dropped.

Follow-up

A CREATE INDEX CONCURRENTLY fails halfway through the partition list. What state is the table in, how do you detect it, and what do you run?
The producer computes quantity itself. What happens to a request already in flight when the dual-write deploy lands, and does it matter?
Give two queries that prove the backfill is complete: one cheap enough to run every minute, one authoritative.

Decide which facts an invoice line copies instead of joining

Medium · denormalisation · immutable records · foreign keys · rounding

invoice_line_item already denormalises tenant_id, which is reachable through invoice_id, and stores amount_minor even though quantity times unit_price_micros would recompute it. A reviewer asks you to normalise both away, and separately asks whether the tenant's legal name and billing address should be copied onto the invoice header. Decide each case. For every field you keep denormalised, name the read pattern or the invariant that justifies it, the anomaly the copy can develop, and the mechanism that prevents that anomaly here.

Approach

Split the question into two kinds of copy, because they fail differently. A copy of a currently mutable fact is a cache: it drifts and needs invalidation. A copy of a fact frozen at write time is not a cache at all, it is the record of what happened, and normalising it away destroys information the source no longer holds.
Keep tenant_id on the line. It costs 8 bytes, it leads every index on the table so no read is ever accidentally cross-tenant, and it turns a wrong join into an empty result rather than another tenant's money. Prevent the drift structurally: a unique constraint on invoice (invoice_id, tenant_id) plus a composite foreign key from the line on (invoice_id, tenant_id) makes a mismatched pair impossible, so the database enforces agreement instead of a code review.
Keep amount_minor. Rounding must happen exactly once, at a named site, with a stated mode (half-even here). If readers recompute from quantity and unit_price_micros, every reader owns a rounding decision, and half-up and half-even diverge systematically across thousands of lines rather than cancelling out. A check constraint can bound the stored value but deliberately cannot re-derive it.
Copy the legal name and billing address onto the invoice header, written once and never updated. The statement must show what was true when it was sealed, and the tenant record will change afterwards. This is a snapshot for the same reason source_rollup_watermark is stored per line: without it, nobody can reconstruct what the customer was told.
Name the read pattern that pays for all of it. Rendering, dispute response and export are per-tenant, per-period reads over thousands of lines that would otherwise join back to slowly changing dimensions that no longer hold the historical value. The write side is a once-per-period batch, so the extra columns cost nothing that matters.
Concede the case where the reviewer is right: a mutable operational attribute such as the tenant's current plan name has no business on a line. If a report wants it, join. If a statement needs the plan as of the period, that is another snapshot and it belongs on the header with the rest.

Follow-up

Write the composite foreign key and the unique constraint it requires on the parent. What does it cost on every line insert, and what does it do to a bulk load?
A tenant is renamed after being invoiced. Which rows change, and what does the customer see on last quarter's PDF?
Where does currency live, and what breaks if a tenant's billing currency changes between two periods?

Compare REST and GraphQL paradigms, detailing the specific performance or architectural problems GraphQL helps solve.

Medium · system design and architecture

Approach

State the consistency you need, and where you are willing to be stale.
Fix the scope first: who calls this, how often, and what they do when it fails.
Name the failure you are designing for, then the recovery path.

Follow-up

What would you drop to keep the system up under load?
What breaks first when traffic grows ten times?

Explain dependency injection and autowiring mechanisms in enterprise frameworks like Spring Boot.

Medium · system design and architecture

Approach

Name the failure you are designing for, then the recovery path.
Name the read and write paths separately; they rarely have the same bottleneck.
Choose a partition key and say what query it makes expensive.

Follow-up

How does this behave when that dependency is down for an hour?
What breaks first when traffic grows ten times?

How do you manage data consistency across distributed microservices using event-driven communication or saga patterns?

Medium · system design and architecture

Approach

Fix the scope first: who calls this, how often, and what they do when it fails.
Choose a partition key and say what query it makes expensive.
Name the failure you are designing for, then the recovery path.

Follow-up

What would you drop to keep the system up under load?
What breaks first when traffic grows ten times?

What are SOLID principles, and how do you apply them to build modular, maintainable software?

Medium

Approach

Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.
Work from the requirement backwards to the design.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Design the batch ingest endpoint metering agents retry into

Hard · batch api · partial failure · deduplication

A customer-run agent posts usage events in batches of up to 1,000 to metering-ingest with a 30-second timeout and at-least-once retry of the whole batch. Each event carries event_id, idempotency_key, sku, quantity and occurred_at; the server adds ingested_at, and usage_event is partitioned daily on ingested_at with unique (ingested_day, tenant_id, idempotency_key). Design the endpoint: the request shape, what the response says when 900 events are new, 90 are duplicates and 10 are malformed, the status code, and the agent's algorithm on timeout. Then state the deduplication horizon and justify it against that unique constraint.

Approach

Fix the per-item outcome taxonomy first, because the status code follows from it: accepted, duplicate, and rejected with a permanent code. A duplicate is a success; reporting it as an error makes the agent either re-send revenue it already delivered or drop it.
Allow only permanent failures per item. A transient per-item failure inside a 200 invites the agent to discard that event, so anything transient escalates to a 5xx for the entire batch. A 200 is then a promise that every event not marked rejected is committed and durable.
Return 200 with a results array aligned by index and carrying the event id, so the agent can retry precisely the subset that needs it and quarantine the ten malformed events instead of hard-looping a poison batch forever. Cap the batch at 1,000 items and a byte size, with 413 beyond it and 429 with Retry-After for backpressure.
Deduplicate per event, never per batch: the agent may split, merge or reorder a retried batch, so a batch-level key matches nothing on the second attempt. The key is (tenant_id, idempotency_key), and the tenant comes from the resolved credential; a tenant id present in the body is compared against it, never trusted.
Size the horizon as a correctness parameter. The unique index includes the partition key, so it deduplicates only within one day: a retry that crosses midnight, or a replay run a week later, passes straight through it. A separate dedup store keyed (tenant_id, idempotency_key) with a TTL exceeding the agent's maximum retry window plus the longest replay you intend to support is what actually enforces the invariant, which makes its retention a correctness setting rather than a cost knob.
Order the commit against the response and the acknowledgement: commit then respond at the endpoint, and downstream commit the fold then acknowledge the message. Acknowledging first turns a crash into silently lost revenue with no error raised anywhere.

Worked solution (40 min)

Write the request body schema with the batch envelope and one event, and state which fields the server assigns rather than accepts.
Write the 200 response for the 900/90/10 case, showing three result entries, one of each outcome, with the rejected one carrying a permanent code.
Write the rule separating per-item rejection from whole-batch failure, and list which conditions fall on each side.
Compute the dedup horizon from the agent's retry window plus the replay window you support, and say where the dedup state lives and how it ages out.
Write the agent's pseudocode for timeout, 5xx, 429 and 200-with-rejections, four branches, and mark which branch may drop an event.
Trace the crash between commit and response, and between fold and acknowledgement, and say what each produces.

Expected result: A 200 with an index-aligned results array where duplicates count as success and rejections are permanent-only, transient problems escalated to a whole-batch 5xx, per-event deduplication keyed on (tenant_id, idempotency_key), and a published horizon larger than retry plus replay because the partitioned unique index only covers one day.

Verify it yourself

The agent can distinguish 'already counted' from 'never counted' for every event in the batch.
Exactly one branch of the agent's algorithm ever discards an event, and it is the permanent-rejection branch.
A retry crossing midnight is still deduplicated, and the mechanism named is not the unique index.
No crash point in the pipeline both commits work and loses it without an error.

Follow-up

The agent times out at 30 seconds having received nothing. What exactly does it do next, and what in your design makes that safe?
Ten events are rejected every hour for a week and nobody notices. What does the endpoint owe the customer beyond a per-item 4xx code?
A replay pushes 40 million events through this endpoint in an hour. Which part of your design degrades first?

Webhook workers leak until OOM and drop in-flight deliveries

Medium · memory leak · eviction · leases

webhook-delivery workers grow from 400 MB to a 2 GB limit over about 36 hours, are OOM-killed, restart, and repeat. Each restart abandons in-flight attempts, so webhook_delivery rows sit in in_flight until their leases expire and the backlog spikes. The live set measured after a forced full collection also grows. The fleet serves tens of thousands of subscriptions, several thousand of which have been failing for weeks. Give an ordered checklist, the measurement separating retention from fragmentation, and the fix.

Approach

Separate the two failure shapes with one measurement: track resident set size against the live set after a forced full collection. A live set that climbs monotonically is retention; a flat live set under a rising RSS is fragmentation, off-heap or native allocation, or an allocator that never returns pages. The stated symptom puts this in the first category, which rules out allocator tuning as a fix.
Characterise the curve rather than the total. Growth linear in uptime implies an unbounded structure keyed by something that keeps arriving; step growth implies buffering a large object. Correlate the slope against event rate and separately against the count of distinct subscriptions seen, because those two diverge and only one of them will fit.
Diff two heap snapshots an hour apart by retained size grouped by dominant root, not by allocation count, which is dominated by short-lived objects and will point at the wrong thing.
Expect a per-subscription map with no eviction: circuit-breaker or backoff state created on first failure and never removed, so the retained set grows with endpoints that have ever failed, and the several thousand permanently dead endpoints hold theirs forever.
Fix in two places. Bound the in-memory structure with a size-capped LRU or a TTL keyed on last use, and move state that must survive a restart onto the subscription or webhook_delivery row, since the worker holding it in memory is exactly why a restart loses it.
Repair the second-order damage separately, because it will outlive the leak: workers claim by compare-and-set with leased_until, so a bounded lease returns in_flight rows to pending on a known schedule, and a graceful shutdown releases leases instead of waiting them out.

Follow-up

The backlog spike after a restart is itself a thundering herd against customer endpoints. What stops the recovery from becoming a second incident?
Suppose the live set had been flat while RSS still climbed. Name two causes and the measurement that separates them.
How would you size the LRU, and what does a miss on an evicted circuit-breaker entry cost a customer whose endpoint is down?

Suggested practice checklist

Day one measures instead of guessing, under a fixed rubric, and the remaining hours are allocated in proportion to the gaps before any studying begins. The allocation is deliberately not renegotiated midweek, because the area that feels worst on day three is usually the one that is moving.

Day 1 — Diagnostic, scored before you study anything

Sit a 110-minute diagnostic in four blocks: forty-five minutes on two coding problems, twenty-five on one design prompt taken to interface and data model, twenty of short-answer fundamentals, and twenty delivering two behavioural answers aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct and 0 is stuck, grading the artifact rather than how the attempt felt.
Allocate days two to five in proportion to 3 minus each block's score, write the allocation down, and commit to leaving it alone.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Split the weakest area into named sub-skills and rate each separately. For coding those are restating the problem, choosing the structure, stating the invariant, turning the invariant into loop bounds, handling empty and single-element input, and accounting for complexity out loud.
Attempt three items positioned just above where the rating drops off, and for each write the first move you failed to make.
Re-attempt one of them from blank four hours later with nothing open.

Deliverable: A sub-skill map with the two blocking sub-skills circled.

Day 3 — Drill the blocking sub-skill by repeating the shape

Do eight short repetitions of the same shape rather than eight different problems, so what gets practised is the pattern and not the puzzle.
State the rule you now hold in one sentence, then test it against a case built to break it, a sliding window over an array containing negative values, or a cache-aside read path whose invalidation message is dropped.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on the strongest area

Run the same sub-skill decomposition on the second-largest gap in half the time.
Spend twenty-five timed minutes on the block you scored highest, choosing the hardest item you can still finish rather than a warm-up.
Write whether each area fails you on recall, on setup, or on execution, and set the fix accordingly: repetition for recall, a written checklist for setup, timed work for execution.

Deliverable: A second sub-skill map plus a one-line failure diagnosis for each area.

Day 5 — The gap that is not a skill

Record one technical and one behavioural answer, then count two things in the playback: seconds before your first clarifying question, and sentences you began without knowing where they would end.
Practise saying that you do not know, followed by how you would find out, without letting it soften into a guess, and practise stating a complexity or an estimate before being asked for it.
Redeliver one answer under a hard ninety-second cap, which forces structure ahead of detail.

Deliverable: Two recordings with a counted reduction in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 110-minute structure with new prompts of comparable difficulty and score it on the identical rubric.
For any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write down which single block you would still lose the offer on.

Deliverable: A second scored rubric placed beside the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a sixty-minute mock over the two blocks that moved least, with an interviewer briefed to interrupt and change direction mid-answer.
Write the recovery script for going blank: restate the question, state your assumption, name the first thing you would check.
Say every rule from the week aloud without reading it, and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive an interruption.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Know where your numbers came from

Bring the two or three numbers the story rests on and know how they were collected. A p99 whose timer starts inside your handler excludes the time a request spent queued, so it can sit flat while users wait longer. Give the window, the percentile and what the measurement left out, or drop the number.

Describe a complex technical challenge you faced on a prior project, how you approached it, and what you learned from the resolution.

Medium · behavioural and engineering judgement

Approach

Give the blast radius: what could have broken, and what you measured.
State the situation in two sentences and spend the rest on the reasoning.
Name the disagreement and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that again?
How did you know your change caused the improvement?

How do you approach code reviews, handling constructive feedback or resolving technical disagreements with team members?

Medium · behavioural and engineering judgement

Approach

Name the disagreement and how you resolved it with evidence.
Pick a story where you made the decision, not one where you watched it.
State the situation in two sentences and spend the rest on the reasoning.

Follow-up

What would you do differently if you ran that again?
What did you decide not to do, and why?

Estimate a tenant-leading index migration you have never run

Hard · estimation · online migration · index builds · uncertainty

Someone needs a date. usage_event carries an index on (occurred_at) and needs (tenant_id, occurred_at); the largest tenant holds roughly a hundred times the median tenant's rows, the table is partitioned daily with years of retention, and you have never run a migration on a table this large. Give an estimate you would defend: how you decompose the work, the two or three numbers you would go and measure first, the range and confidence you state, and what you commit to when the person asking needs a single date today.

Approach

Refuse the bare number and then give one anyway, in the form that is actually useful: a range plus the measurement that collapses it. 'Four to eleven days; one afternoon building this index on a restored copy of the largest partition takes that to within a day' is an answer, while 'it depends' is not.
Decompose by failure mode rather than into equal chunks, because that is where estimates go wrong. On a partitioned parent you create the index ON ONLY the parent, build each partition's index with CREATE INDEX CONCURRENTLY, then ALTER INDEX ... ATTACH PARTITION, at which point the parent index becomes valid. CONCURRENTLY does not block writes but scans each partition twice, waits out older transactions, cannot run inside a transaction block, and on failure leaves an invalid index you must drop concurrently and retry.
Name the two unknowns that dominate and price them: build time on one restored partition of realistic size, and whether the planner actually chooses the new index for the skewed tenant, since selectivity for a tenant holding most of the rows is a different question from selectivity for the median tenant. Both are half-day measurements against a replica, and both are cheaper than being wrong by a week.
State the assumptions the range is conditional on, because that is what makes a slip a re-estimate instead of a credibility event: no partition above a stated row count, one concurrent build at a time so it does not compete with ingest for I/O, and an ingest backlog that can absorb the added write amplification while both indexes exist.
Budget the step nobody budgets: verification and the old index's removal. Dropping the old index is fast, but deciding it is safe to drop means confirming no plan still uses it, and that confirmation waits on real traffic across a full weekly cycle rather than on your patience.
Answer the single-date request honestly. Commit to a date for the first checkpoint — the measured build number from the replica — and to re-estimating on that date, and say plainly what you are not committing to yet. A date with a scheduled re-estimate is worth more to the asker than a confident wrong one, and you should say why in those words.

Follow-up

The concurrent build fails half way through the largest partition. What is the state of the database and what do you do next?
Your estimate slips by sixty percent. Which assumption broke, and at what point would you have known?
The person asking needs the date for a customer commitment. Does your answer change?