cswork / libraryOpen practice workspace

Interview guides · Adaptive Biotechnologies

Adaptive Biotechnologies Software Engineer Interview Questions & Guide 2026

Adaptive Biotechnologies software engineer prep on operations: decide what you would need to see to know a release is going wrong.

The 60-second brief

As a Software Engineer at Adaptive Biotechnologies, you are at the intersection of cutting-edge immunology and massive-scale data processing. Your work directly enables the company's mission to decode the human immune system, translating complex biological data into actionable insights for disease detection and treatment. This is not just traditional software development; it is a role where your code has a tangible, life-altering impact on patients and clinical research.

If the seat owns a service boundary, scope your preparation toward failure behaviour rather than topology. Retrying over an at-least-once channel produces duplicates by construction, so a retry policy is only as safe as the idempotency key underneath it.

Focus areas

  • Authorise every identified read against current consent
  • Make HL7 and X12 ingestion idempotent under replay
  • Version clinical results instead of updating rows in place

Overview

You will contribute to the development of robust, scalable platforms that handle high-throughput genomic data. Whether you are working on bioinformatics pipelines, cloud infrastructure, or internal tooling, you will be expected to balance technical rigor with the agility required in a fast-paced biotech environment. Successful engineers here are those who are not only proficient in their craft but are also deeply curious about the biological problems their software helps solve.

The Adaptive Biotechnologies interview process

Adaptive Biotechnologies candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

The title covers product work, platform work, infrastructure, mobile and frontend, and those are different jobs with different loops behind them. A screening call is the cheapest place to find out which one the seat is, and asking reads as experienced rather than fussy. The questions that separate them: what the team is on call for, what the last three projects were, and whether any round happens inside an existing repository instead of a blank file. Then say which of that you have done and which you have not. Claiming the whole posting is the fastest way to be found out one round later.

What to demonstrate

  • Whether you can locate your experience inside one flavour of the role honestly instead of claiming the entire requirements list
  • Whether you name what you have not done, which an experienced screener reads as a level signal and can plan the loop around
  • Whether what you want next matches what the seat is: someone who wants greenfield work landing on a team that mostly operates an existing system is a hire that leaves within the year

How to prepare

  • Mark every line of the posting as done, adjacent or new, and write one sentence for each adjacent line naming the closest thing you actually built
  • Split your last two years into rough percentages across feature work, operating and debugging live systems, and design or review, so a question about scope gets numbers rather than adjectives
  • Bring three questions that discriminate between seats: what the team is paged for, how much of the work is changing existing code versus standing up something new, and what shipped in the last quarter

Technical Assessment

What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.

What to demonstrate

  • Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
  • Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
  • Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly

How to prepare

  • Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
  • Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
  • Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live

Virtual or Onsite Interviews

Where the day includes a partner from product, design or data, that conversation is weighted like the technical ones and prepared for least. They are deciding one thing: whether having you in the room makes their decisions cheaper. That means options with costs attached, not implementation detail and not "it depends". An estimate someone can plan against — a range, the assumption that would push it to the high end, and what you would drop to hit the low one — is worth more than a confident single number, which everyone present already knows is wrong.

What to demonstrate

  • Whether an estimate comes as a range with the assumption most likely to break it, and states what a specific scope cut would actually buy
  • Whether a technical constraint is handed over as a choice with consequences on their side, rather than as a verdict they have no standing to argue with
  • Whether you establish what decision is on the table before proposing anything
  • Whether risk is raised while it can still change the plan, with the trigger that would confirm it, instead of reported afterwards as a slip

How to prepare

  • Take a project that shipped late and write the two-sentence warning you could have given three weeks earlier, naming what you would have needed decided at that point
  • Rehearse one estimate out loud until it arrives in three parts: the range, the single assumption that would blow it, and the smallest thing you would cut to protect the date
  • Rewrite an objection you have actually made — the "we can't do that" version — as two options with their costs, so the choice ends up with the person who owns it

Where candidates lose points

Writing the access audit record inside the read transaction, or firing it off after the response with no durability.

Inside the transaction, an audit-store outage blocks clinical reads and turns a logging dependency into a care outage. Fire-and-forget afterwards means the audit trail is incomplete during precisely the incidents it exists to reconstruct, and the gaps are invisible until someone asks for the log. The usual resolution is committing the access decision and its audit row together to a local outbox and shipping asynchronously, which keeps the read path available while preserving durability.

Upserting on the order or result identifier, so a correction overwrites the original row.

It makes the question 'what did the clinician see at 14:02' unanswerable, which is exactly what an incident review or a legal hold asks. It also leaves downstream consumers that already acted on the preliminary value with no correction event to react to, because the state transition was collapsed into a single mutated row and never emitted.

Reading the constraints as preamble rather than as part of the problem

The bounds are usually there to eliminate the obvious approach: n up to 10^5 makes an O(n^2) scan roughly 10^10 operations, far outside any per-test time budget, and an input larger than memory rules out loading it at all. When a bound is not given, ask for it, then say out loud which approach it kills.

Assuming fixed-width integer arithmetic cannot overflow

In languages with fixed-width integers, including C, C++, Java, Go and Rust, computing a midpoint as (lo + hi) / 2 overflows once the sum passes the type's maximum, so write lo + (hi - lo) / 2 instead. Say which language you are in: arbitrary-precision integers, as in Python or Ruby, remove this specific hazard and none of the others.

Questions & practice

Find the next available meeting time slot where all participants are available.

Medium · data structures and algorithms

Approach

  1. State the target complexity and say which constraint rules the naive version out.
  2. Walk one small example through your approach before writing the whole thing.
  3. Choose the data structure from the access pattern, not from familiarity.

Follow-up

  • Which test case would catch an off-by-one here?
  • What is the worst case, and how likely is it on real data?

Write a function to generate all possible permutations of the characters in a given string.

Medium · data structures and algorithms

Approach

  1. Walk one small example through your approach before writing the whole thing.
  2. Restate the input: its shape, its size, and what is guaranteed about it.
  3. Name the brute-force solution and its complexity before improving on it.

Follow-up

  • Which test case would catch an off-by-one here?
  • How does this change if the input no longer fits in memory?

Find which sets of two numbers in an array add up to seven.

Medium · data structures and algorithms

Approach

  1. Restate the input: its shape, its size, and what is guaranteed about it.
  2. Choose the data structure from the access pattern, not from familiarity.
  3. State the target complexity and say which constraint rules the naive version out.

Follow-up

  • Which test case would catch an off-by-one here?
  • What is the worst case, and how likely is it on real data?

Update a binary search tree to keep track of each node’s grand-parent node.

Medium · data structures and algorithms

Approach

  1. State the target complexity and say which constraint rules the naive version out.
  2. Choose the data structure from the access pattern, not from familiarity.
  3. Restate the input: its shape, its size, and what is guaranteed about it.

Follow-up

  • What is the worst case, and how likely is it on real data?
  • How does this change if the input no longer fits in memory?

Resolve enterprise identity from a link log that supports unmerge

Hard · union-find · rollback · identity resolution

person_identity_link holds link_id, enterprise_person_id, assigning_authority, source_person_id, match_score, link_status (auto_linked, manual_linked, potential_duplicate, unlinked, rejected), version, superseded_by_link_id, decided_by, decided_at. You have up to 40 million source identities and 60 million decisions applied in decided_at order. Build a structure answering which enterprise identity a given (assigning_authority, source_person_id) resolves to after any prefix of the log, and supporting a revert of the k most recent merges at O(1) each. State the query complexity, and say why path compression is unavailable to you.

Approach

  1. Intern the node key as the pair (assigning_authority, source_person_id) into a dense integer index. Never key on source_person_id alone: two facilities in one network routinely issue the same medical record number to different people, so a bare-value key merges two patients before any matching logic has run, and a single-source test fixture will never show it.
  2. Classify the log rows before touching the structure. auto_linked and manual_linked are unions; potential_duplicate and rejected are recorded decisions that must not union anything; unlinked is a revert of the link it supersedes, not a new edge.
  3. Use union by size with an explicit undo stack, pushing (attached_root, its previous parent, the previous size of the absorbing root) on every union. Find walks parent pointers to the root: union by size bounds tree height at log2(n), so a query is O(log n) worst case, a union is O(log n), and a revert pops two words and is O(1).
  4. Path compression is the thing you have to give up, and the reason is specific: it rewrites parent pointers of nodes that were never named in the union being recorded, so the undo entry no longer describes the mutation that happened and a revert restores a forest that is quietly wrong. Near-constant find is only available to an append-only structure.
  5. Hold enterprise_person_id as a property of the root slot rather than a value copied onto every member. A merge then relabels one slot instead of n rows, and an unmerge restores two labels instead of reconstructing n.
  6. State the price plainly: every read path gains a resolution hop and no downstream table can carry a plain patient_id column. That cost is what buys reversibility, and it belongs in the design discussion rather than being discovered later by whoever writes the first join.

Worked solution (40 min)

  1. Intern three nodes: A is (AUTH1, MRN-77), B is (AUTH2, MRN-77), C is (AUTH1, MRN-90).
  2. Apply union(A, C) from a manual_linked row, pushing its undo entry.
  3. Apply union(A, B) from an auto_linked row with a high match score, pushing its undo entry.
  4. Query all three, then pop one undo entry and query all three again.
  5. Assert the parent and size arrays element-wise against a snapshot taken before the second union.

Expected result: After the first union, A and C share a root and B is its own root. After the second, all three share a root. After reverting one union, B is its own root again and A and C remain merged, with the first union's record untouched. Note what the key bought you: A and B collide on MRN-77 and would have been one identity from the first insert under a bare-value key, with no decision row ever recorded.

Verify it yourself

  • After reverting all k unions, the parent and size arrays are identical element by element to the pre-union snapshot, not merely equal in their query answers.
  • Tree height never exceeds log2(n). Assert it, because one union that attaches by arbitrary root instead of by size breaks the find bound without any visible symptom.
  • Applying a potential_duplicate or rejected row changes no query answer and pushes nothing onto the undo stack.
  • Querying (AUTH1, MRN-77) and (AUTH2, MRN-77) before any link row returns two distinct identities.

Follow-up

  • A merge is found wrong three weeks and 200,000 unions later, and your stack only reverts a suffix. What do you do instead, and what does that cost?
  • A claim was posted under the losing identity before the merge. After the unmerge, which identity owns it, and what in your design let you answer that?
  • Two matcher instances submit unions concurrently. What is the smallest change that keeps both the structure and the undo stack consistent?

Diagnose why an eligibility lookup ignores its index

Medium · indexing · query planning · partial indexes

A 40-million-row coverage_span serves eligibility lookups with: SELECT coverage_id, plan_id, coverage_order FROM coverage_span WHERE enterprise_person_id = $1 AND valid_to = 'infinity' AND effective_date <= $2 AND (termination_date IS NULL OR termination_date >= $2). An index exists on (effective_date). EXPLAIN (ANALYZE, BUFFERS) shows a sequential scan reading 1.8 million buffers at 2.4 seconds. Explain why that index cannot help, propose the index plus any schema change that makes the predicate index-friendly, and state what you expect the new plan and buffer count to be.

Approach

  1. Read the predicate for selectivity before touching the index. effective_date <= $2 matches most of a 40-million-row history, so a leading range scan on that column returns a large fraction of the table and the planner correctly prefers a sequential scan over millions of random heap fetches. The index is not being ignored; it is being rejected on cost.
  2. Put the equality column first. enterprise_person_id = $1 selects a handful of rows out of 40 million, so it must lead the composite index; a b-tree can only use columns after the first range predicate as filters, not as search bounds.
  3. Turn the open-ended termination into something indexable. The OR ... IS NULL branch forces either a BitmapOr or a post-index filter. Add coverage_end date GENERATED ALWAYS AS (COALESCE(termination_date, DATE '9999-12-31')) STORED and rewrite the predicate to coverage_end >= $2; both COALESCE over a column and a date literal are immutable, so the generated column is legal.
  4. Make the index partial on WHERE valid_to = 'infinity'. Most rows in a bitemporal table are superseded beliefs, so the partial index is a fraction of the full one, stays in cache, and encodes the predicate for free rather than re-checking it per row.
  5. Consider the range alternative honestly: a daterange(effective_date, termination_date + 1, '[)') column with a GiST index and the && operator handles overlap queries and supports an EXCLUDE constraint against overlapping active coverage, but GiST lookups are slower than b-tree for this point-in-time pattern. Pick b-tree for the read path and keep GiST only if you also need the overlap constraint.
  6. Verify rather than assert: re-run EXPLAIN (ANALYZE, BUFFERS) and read Rows Removed by Filter and the shared hit/read split, not just the total time.

Follow-up

  • After the change the plan is still a sequential scan for one particular member. What single query tells you whether that is stale statistics, a genuinely huge row count for that person, or parameter-dependent plan caching?
  • Your generated column needs backfilling on a live table. Does adding a STORED generated column rewrite it, and what does that mean for your maintenance window?
  • How would you serve the same lookup when the answer must be as of a past system-time instant rather than the current belief, given your index is partial on valid_to = 'infinity'?

Reproduce and fix a lost update on a deductible accumulator

Medium · isolation levels · lost update · lock contention

An accumulator row holds deductible_applied_cents bigint and plan_deductible_cents bigint, keyed by (enterprise_person_id, plan_id, benefit_year). The adjudicator opens a transaction, SELECTs the row, computes the member's share in application code, then UPDATEs the row to the absolute new total it computed, all under PostgreSQL's default READ COMMITTED. Two claim lines for one member adjudicate concurrently: both charge the member deductible, but the accumulator advances by only one of the two amounts, so the member is billed deductible again on a later line after the plan deductible has already been met. Write the exact two-session interleaving that produces it. Then give three fixes, each naming the lock or isolation level, the SQLSTATE you must handle, and the throughput cost.

Approach

  1. State the anomaly precisely. READ COMMITTED takes a fresh snapshot per statement, so it prevents dirty reads but permits a lost update when a transaction reads a value, computes outside the database, and writes back an absolute result. The vulnerable window is the round trip through application code between two statements, not the transaction boundary — the same logic expressed as one relative UPDATE is safe at this very isolation level.
  2. Write the interleaving as an ordered script both sessions can be replayed from, with the commit points marked, and make both writes absolute (SET deductible_applied_cents = :computed_total). The point is not that two writes happen — it is that the second write stores a total computed from a value that had already been superseded by the time it landed.
  3. Fix one, SELECT ... FOR UPDATE on the read: the second session blocks on the row lock and, under READ COMMITTED, re-reads the newest committed version when it unblocks, so its computation starts from the winner's total. No serialization error to handle. Cost is serialised throughput per member and a lock held for the transaction's whole duration, so nothing inside may call an external payer.
  4. Fix two, REPEATABLE READ or SERIALIZABLE with a retry loop: in PostgreSQL, REPEATABLE READ aborts the second writer of a row with SQLSTATE 40001, 'could not serialize access due to concurrent update'; SERIALIZABLE additionally aborts on read/write dependencies detected by SSI, also under 40001. The retry re-runs the whole transaction, so the claim application must be idempotent or keyed by claim_line_id, or a retry double-applies.
  5. Fix three, single-writer partitioning: route by hash(enterprise_person_id) to one consumer per partition, so no two transactions ever touch one accumulator. No locks, no retries; the cost is head-of-line blocking behind a slow claim, rebalancing during deploys, and losing the ability to scale a hot member.
  6. Name the cheapest option and its real catch. A single UPDATE ... SET deductible_applied_cents = LEAST(plan_deductible_cents, deductible_applied_cents + :amt) ... RETURNING deductible_applied_cents does not lose the update: under READ COMMITTED an UPDATE that hits a concurrently updated row blocks, then re-evaluates its expressions against the newly committed version, so both increments land. The catch is that the member's share is the delta this statement actually applied, which is the returned value minus the row's pre-image — and RETURNING does not expose the pre-image before PostgreSQL 18's OLD/NEW aliases. On earlier versions you must read that pre-image under the same row lock, which puts you back in fix one with a shorter critical section. The one-statement form is a clean fix only when the caller does not need the delta.

Worked solution (35 min)

  1. Open two psql sessions against a scratch database and seed one accumulator at deductible_applied_cents = 40000, plan_deductible_cents = 100000.
  2. Run the interleaving by hand with both writes absolute, and confirm the row ends at 70000 while the two lines between them charged the member 55000 of deductible — the serial answer is 95000, so 25000 of applied deductible has been lost.
  3. Run the control: repeat the identical interleaving with both UPDATEs written as SET deductible_applied_cents = deductible_applied_cents + :amt, and confirm the row ends at 95000. That form does not lose the update at READ COMMITTED, which is how you know the defect is the read-modify-write in application code rather than the isolation level by itself.
  4. Re-run the absolute version with FOR UPDATE on both SELECTs and confirm session two blocks, re-reads 65000, and writes 95000.
  5. Re-run both sessions at REPEATABLE READ and capture the exact error text and SQLSTATE from the losing session.
  6. Write the retry wrapper and show what makes the retry safe: a uniqueness key on (accumulator_key, claim_line_id) in an application ledger, so a re-run of an already-applied line is a no-op rather than a second application.
  7. Write the one-statement LEAST form with RETURNING and say where the member's share now comes from, given RETURNING carries no pre-image before PostgreSQL 18.

Expected result: Seed: deductible_applied_cents = 40000, plan_deductible_cents = 100000. Interleaving: T1 BEGIN; SELECT deductible_applied_cents -> 40000; application computes line A's deductible share as 25000 and the new total as 65000. T2 BEGIN; SELECT -> 40000, because T1 has not committed and READ COMMITTED shows only committed rows; application computes line B's share as 30000 and the new total as 70000. T1 UPDATE accumulator SET deductible_applied_cents = 65000 WHERE key...; COMMIT. T2 UPDATE accumulator SET deductible_applied_cents = 70000 WHERE key...; it blocks on T1's row lock, and on unblocking writes its absolute 70000 over the committed 65000; COMMIT. The row reads 70000 while the two lines charged the member 55000, so the serial answer is 95000 and T1's 25000 is lost; the accumulator understates by 25000 and the member is billed deductible again after the 100000 is already met. The write must be absolute for this to happen — SET deductible_applied_cents = deductible_applied_cents + :amt blocks, re-reads the committed row and applies both increments. Fixes: (1) SELECT ... FOR UPDATE on the read, no SQLSTATE to handle, serialises per member for the transaction's whole duration and therefore forbids external calls inside it; (2) REPEATABLE READ or SERIALIZABLE plus retry on SQLSTATE 40001 ('could not serialize access due to concurrent update' under REPEATABLE READ; SERIALIZABLE raises the same SQLSTATE for read/write dependencies), which needs an idempotency key because the whole transaction re-runs; (3) hash-partition claims by enterprise_person_id to one writer per partition, zero contention at the cost of head-of-line blocking and rebalancing.

Verify it yourself

  • The unfixed script reproduces 70000 on a real database rather than on paper, and replaying the same two lines serially reproduces 95000.
  • The control run with in-SQL relative increments, under the identical interleaving, ends at 95000 — that isolates the defect to the application-side read-modify-write and not to READ COMMITTED itself.
  • With FOR UPDATE, session two's SELECT visibly blocks until session one commits, and the final value is exactly 95000.
  • At REPEATABLE READ the losing session reports SQLSTATE 40001 with 'could not serialize access due to concurrent update', and its transaction is left in an aborted state requiring ROLLBACK.
  • Applying the same claim_line_id twice through the retry path leaves the accumulator unchanged on the second attempt.

Follow-up

  • A claim is reversed six months later under a plan design that has since changed. What amount does the reversal subtract, and where is that number stored?
  • Your retry loop hits 40001 repeatedly for one member during a batch window. What is the backoff, and at what point do you stop retrying and pend the claim?
  • Which of the three fixes survives a retroactive eligibility change that invalidates everything applied in the last month, and what does the rebuild look like?

What technologies are you most familiar with and why?

Medium

Approach

  1. Clarify what is being asked and what a complete answer contains.
  2. Work from the requirement backwards to the design.
  3. State your assumptions explicitly before working the problem.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

How would you design a car reservation system?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Say what you would check first and why it is the highest-information step.
  3. Clarify what is being asked and what a complete answer contains.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Shape a bulk enrolment endpoint where rows fail independently

Medium · bulk operations · partial failure · idempotency

A payer partner submits weekly enrolment as one call carrying 10^4 to 2x10^6 member rows that become coverage_span (coverage_id, enterprise_person_id, payer_id, plan_id, subscriber_id, effective_date, termination_date, coverage_order, status, valid_from, valid_to, source_file_id). Some rows fail identity resolution, some carry a termination date before their effective date, and the rest apply cleanly. The partner's loader must be able to retry only what failed, and cannot fix the file until next week. Design the submission contract: synchronous or asynchronous, the response shape, how a row is addressed, and what retrying the failed subset does to rows that already applied.

Approach

  1. Size the call before shaping it. 2x10^6 rows will not survive a request/response cycle under any sane timeout, so the contract is asynchronous: upload, 202 with a job resource, poll or subscribe. All-or-nothing is equally wrong, because one malformed row would reject two million good ones and the partner cannot resubmit for a week.
  2. Give every row a caller-assigned stable identity - the partner's file id plus line number - so a result addresses back to the input without positional matching. Positional indexes break the instant the partner resubmits a filtered file, which is exactly what the retry is.
  3. Make the results a separate paginated resource, not an inline array: GET /enrolment-jobs/{id}/results?status=failed&cursor= returns one entry per failed row with a reason code and the offending field. An inline error array is unbounded, and two million failures is not a response body.
  4. Classify per row into applied, rejected and needs_review, and keep the last two addressable rather than dropping them. A row that failed identity resolution is not invalid data, it is an unresolved person, and it must be re-drivable after a manual link without the partner resubmitting anything.
  5. Make the row the unit of idempotency: UNIQUE (source_file_id, partner_line_number), so a resubmitted overlapping file is absorbed. State the one case that is not a duplicate - the same key with changed content must supersede, which in a bitemporal table means closing the prior version's valid_to and inserting a new row with valid_from set, leaving effective_date and termination_date as the partner asserted them rather than rewriting business time.

Worked solution (30 min)

  1. Decide the transport and show the arithmetic that rules out a synchronous body at two million rows.
  2. Write the job resource schema: state, counts by outcome, started and finished timestamps, and links to the result pages.
  3. Write the per-row result schema with the caller's row identifier, the outcome, the reason code and the offending field.
  4. Write the row-level idempotency key and its constraint, then trace a retry of the failed subset against a job where 90 percent applied.
  5. Write what happens to a row whose content changed since it applied: the valid_to close-out and the new version, with business dates untouched.

Expected result: POST /enrolment-files returns 202 with Location: /enrolment-jobs/{id}. GET on the job returns {state, counts:{applied, rejected, needs_review}, started_at, finished_at}. GET /enrolment-jobs/{id}/results?status=failed is cursor-paginated, one entry per input line keyed by (file_id, line_number) with reason_code and field. Row idempotency on UNIQUE (source_file_id, partner_line_number), so a retried subset re-applies nothing already applied. A changed row closes the prior version's valid_to and inserts a new version with valid_from, leaving effective_date and termination_date as asserted.

Verify it yourself

  • No response in the design grows with the number of input rows.
  • Every failed row is addressable by an identifier the partner assigned, not by position.
  • Retrying the failed subset twice leaves the same coverage_span state as retrying it once.
  • A row rejected for unresolved identity can be re-driven after a manual link without a partner resubmission.

Follow-up

  • Midway the job hits a run of failures caused by our own dependency being down. Does the job fail, pause or keep going, and what does the partner see?
  • The partner resubmits the entire file instead of the failed subset. What happens?
  • How does the partner learn that a row applied and was later superseded by a newer file, without diffing two full exports?

Gateway instances OOM every five days with flat traffic

Medium · memory leak · heap profiling · idempotency

Interoperability gateway instances are OOM-killed with exit 137 every five to six days. Resident memory grows about 400MB a day per instance and never falls; restarting resets it. Message throughput, connection count and partner mix are unchanged over that period. The gateway keeps an in-process set of recently seen idempotency keys — (sending facility, sending application, message control ID, event timestamp) — in front of the durable ledger. Give the ordered investigation and say what evidence would rule the idempotency cache out.

Approach

  1. Establish that this is a retained-object leak before profiling. Compare resident memory with live heap measured immediately after a forced collection, sampled daily. Resident memory rising while post-collection live heap stays flat points at allocator fragmentation, off-heap or native buffers, or thread stacks, and a heap dump will implicate the wrong object. Rising post-collection live heap is a genuine leak.
  2. Find what the growth is linear in. Plot daily growth against messages processed, connections accepted and wall-clock hours across instances carrying different traffic. Proportional to messages means something is retained per message; proportional to connections means per-connection state is not released on abnormal close, such as a half-open socket whose FIN never arrives.
  3. Diff two heap dumps taken six hours apart on the same instance, sorted by retained size, and identify the dominator. An unbounded set or map shows up as one root holding millions of small entries. This is a five-minute answer once you have the dumps, which is precisely why the first two steps exist: they tell you whether the dump can answer the question at all.
  4. Check the idempotency cache directly against arithmetic rather than suspicion. At single-digit millions of messages a day and a key of roughly 80 bytes plus per-entry overhead, an unevicted set grows by hundreds of megabytes a day, which matches the observed 400MB. A live gauge of its entry count that stays flat is the evidence that rules it out; a gauge tracking messages processed convicts it.
  5. Fix and restore the invariant: bound the cache by size or by a time window, and keep the durable unique constraint as the authority. An in-process set is a latency optimisation and cannot be the correctness mechanism, because it does not survive a restart and is not shared across instances — so a replay right after a deploy would bypass it entirely.
  6. Verify over 48 hours that post-collection live heap is flat, and alert on the cache's entry-count gauge rather than on resident memory, which only tells you after the growth has already happened.

Follow-up

  • How long must the time window be? Tie the number to the sender's replay behaviour after an interface restart and to how far back the broker redelivers.
  • If growth had tracked connections instead of messages, what would you look at first, and how would a half-open MLLP connection show up in the socket table?

Suggested practice checklist

For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.

Day 1 — Rebuild the primitives by implementing them

  • Implement a dynamic array with doubling growth and an operation counter, then change the growth rule to add a fixed sixteen slots instead, and time both for n of ten thousand, a hundred thousand and a million. The fixed-increment version resizes n/16 times at O(n) each, so its total work is quadratic; doubling is what makes append amortised constant.
  • Implement a hash map with separate chaining and a load-factor resize, then insert ten thousand keys engineered to land in one bucket and record what happens to lookup time, so that average-case O(1) becomes a claim with a stated precondition rather than a reflex.
  • For dynamic-array append and hash-map insert, write down which cost is amortised rather than worst-case, which single operation pays the whole bill, and what a system with a hard per-operation deadline would have to do instead.

Deliverable: Two working implementations plus a timing table showing the input at which each structure's advertised complexity stops holding.

Day 2 — Arrays under an invariant: two pointers, sliding window, binary search

  • Solve longest-subarray-with-sum-at-most-K using a sliding window, then run it on an input containing negative numbers and watch it return the wrong answer: extending the window only moves the sum monotonically when every element is non-negative, and that precondition is the whole reason the technique works.
  • Write the binary search that finds the first index satisfying a predicate rather than an exact value, put the loop invariant above the loop in a comment, and verify termination on the two inputs that break careless versions: the empty range, and a range where every element satisfies the predicate.
  • Compute the midpoint as lo + (hi - lo) / 2 and write one line on why the obvious (lo + hi) / 2 is a genuine defect in a fixed-width integer type and a non-issue in a language with arbitrary-precision integers.

Deliverable: Three solved problems, each with its invariant written above the loop, plus one recorded input on which the sliding window is provably wrong.

Day 3 — Sorting, heaps, and the greedy argument that has to be proved

  • Solve one top-k problem three ways, by full sort, by a size-k heap, and by quickselect, then write the values of n and k at which each becomes the right choice, along with quickselect's quadratic worst case and why a randomised pivot makes that unlikely rather than impossible.
  • Implement bottom-up heapify and count sift-down steps to confirm it does linear work rather than n log n, because most nodes sit near the bottom of the tree and therefore move only a short distance.
  • Take interval scheduling by earliest finishing time and write the exchange argument out in full: given any optimal schedule, swapping in the earliest-finishing interval keeps it feasible and no smaller. Then construct the weighted variant where that same greedy fails and name what has to replace it.

Deliverable: A three-way top-k comparison with measured crossover points, one written exchange argument, and one counterexample to a greedy rule that looks almost identical.

Day 4 — Recursion, memoisation, and the step to a table

  • Take one problem with overlapping subproblems, such as edit distance or coin change, instrument the plain recursion with a call counter to show the blow-up, then add memoisation and re-count.
  • Convert the memoised version to a bottom-up table and state the two properties you relied on: each subproblem's result depends only on its arguments, and the dependencies form a DAG you can enumerate in order.
  • Rewrite one deep recursion with an explicit stack, then find the input length at which the original hits the interpreter's frame limit, which defaults to about a thousand frames in CPython, so you know when the rewrite is required rather than decorative.

Deliverable: One problem in three forms, naive, memoised and tabulated, with call counts for each and the input length at which recursion depth becomes the binding constraint.

Day 5 — Graphs, where most of the work is choosing the traversal

  • Implement BFS and DFS over one adjacency list, then answer for each which finds a shortest path in an unweighted graph and which you would use to detect a cycle in a directed graph, including why the in-progress versus finished distinction matters for the second.
  • Implement topological sort by in-degree, feed it a graph containing a cycle, and confirm the failure signature is that fewer than V nodes come out rather than an exception, then note that the order it produces is one of several valid ones.
  • Run a shortest-path search on a graph with a single negative edge weight and show the wrong answer, then write the precondition Dijkstra actually needs, non-negative weights, because it finalises a node's distance the first time that node is popped, and name the algorithm you would switch to and its own limit.

Deliverable: A small graph library with BFS, DFS and topological sort, plus two inputs that produce documented wrong answers under the wrong algorithm choice.

Day 6 — One day for everything that is not an algorithm

  • Sketch one system only to the depth a coding-heavy loop tends to reach: the endpoints, what the service stores, and the single query pattern that decides the schema. Stop at twenty-five minutes.
  • Prepare the project answer for an interviewer who codes, which means rehearsing the two levels they push to: the specific thing you built, and why you chose that approach over the alternative they will name. Open with a number and be ready to say what it excludes.
  • Prepare the answer to what you would do differently, choosing a real technical mistake with a specific fix rather than a complaint about process or staffing.

Deliverable: One design sketch at endpoint-and-schema depth, plus a project answer rehearsed to two levels of follow-up.

Day 7 — Solve out loud, under time

  • Do three timed problems at twenty-five minutes each in a plain editor with no autocomplete and no execution until the end, then tally separately the failures that were syntax and the ones that were approach, because those two numbers call for different fixes.
  • Narrate one solution from the first sentence, stating the approach and its complexity before writing any code, and rehearse the sentence you will use when you realise mid-solution that the approach is wrong.
  • Re-solve from blank the two problems you were slowest on this week and compare the times against the day they first appeared.

Deliverable: A recording of one fully narrated solution and a tally that separates syntax failures from approach failures.

The failure that was yours

Keep one story where the bad call was yours rather than a dependency's or a manager's. Name the check that would have caught it, whether you added that check afterwards, and whether it has fired since. Answers that route blame outward end the conversation early; answers that end in a guardrail someone still relies on tend to open it up.

Describe a time you had to resolve a conflict within a team.

Medium · behavioural and engineering judgement

Approach

  1. Pick a story where you made the decision, not one where you watched it.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • How did you know your change caused the improvement?
  • What would you do differently if you ran that again?

Tell me about a previous difficult experience you have had at work.

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement and how you resolved it with evidence.
  3. Give the blast radius: what could have broken, and what you measured.

Follow-up

  • What did you decide not to do, and why?
  • What would you do differently if you ran that again?

Ship an eligibility cache you already know is too stale

Medium · technical debt · caching · eligibility · trade-offs

You shipped the eligibility service against a fixed go-live with a 24-hour cache keyed on person and plan, no service date in the key and no computed-as-of field in the response, because the real-time payer inquiry path was two sprints behind. Enrolment files routinely terminate coverage retroactively — a file received on the fifth can terminate coverage effective the first. Describe shipping a compromise of this shape: what you wrote down, who accepted the risk, the bound you could honestly state, the measurement that sized the debt in money, and whether it was ever paid.

Approach

  1. The probe is whether you can state a compromise as a bounded risk rather than as a known limitation. Start by separating two staleness sources, because they have different fixes: cache staleness is bounded by the TTL, while business-time staleness is not bounded by anything in your system, since the source itself asserts changes retroactively.
  2. Follow that distinction to its uncomfortable conclusion — shortening the TTL to five minutes leaves the second unchanged. A zero-TTL cache still serves an answer that was false when computed, so anyone who proposes TTL tuning as the fix has misread which of the two is biting.
  3. Write the debt as a sentence with numbers: the service can answer 'covered' for a person whose coverage terminated up to the enrolment cadence plus its retroactive window earlier, with up to 24 hours of cache on top. That sentence is what a reviewer can accept or refuse; a ticket title is not.
  4. Name the mitigation that costs days rather than sprints: put service_date in the cache key and computed_as_of in the response body so consumers can see an answer's age, and re-check eligibility at claim submission rather than trusting the encounter-time answer.
  5. Give the measurement that sizes it in money — the weekly count and dollar value of claim_line rows denied for terminated coverage where the answer served at encounter time said active, filtered to the surviving claim version. That number is what buys the sprint later; without it the fix competes against features on opinion.
  6. Be specific about the acceptance. Name the artefact the decision lives in and the person with budget authority who signed it, because a compromise nobody accountable acknowledged is a decision you made alone and later described as a trade-off.

Follow-up

  • The payer inquiry path ships and the cache stays. Which of the two staleness bounds has actually moved?
  • A service was delivered against a stale 'covered' answer. Who absorbs the cost, and does your design record enough to answer that?
  • Registration wants a 50ms p99 on this call. What do you give up, and how do you say that to them in one sentence?