cswork / libraryOpen practice workspace

Interview guides · Assured Consulting Solutions

Assured Consulting Solutions Software Engineer Interview Questions & Guide 2026

Testing practice for Assured Consulting Solutions software engineer candidates: the boundaries worth covering, and why flakiness is a real defect.

The 60-second brief

A Software Engineer at Assured Consulting Solutions plays a pivotal role in delivering high-impact technical solutions, often within the defense and intelligence sectors. The work is deeply mission-focused, requiring engineers to translate complex requirements into robust, reliable software that supports critical national security and geospatial initiatives. Whether you are working on AI/ML integration, Cloud infrastructure, or Space Systems, your contributions directly influence the operational efficiency and success of Assured Consulting Solutions' clients.

Most rounds test how you handle an under-specified problem more than what you recall. A wrong first approach you correct out loud is recoverable; a constraint you assumed silently and were never given is the expensive mistake.

Focus areas

  • Model money movement as balanced double-entry postings
  • Make every money-moving endpoint idempotent by key
  • Consume webhooks duplicated, delayed and out of order

Overview

What makes this role particularly compelling is the emphasis on long-term growth and mission alignment. Unlike companies that rely heavily on automated coding tests, Assured Consulting Solutions prioritizes identifying candidates who possess the right mindset, cultural alignment, and foundational curiosity to tackle domain-specific challenges. You will be expected to collaborate closely with cross-functional teams, including project managers and chief engineers, to build systems that are not only technically sound but also strategically vital.

The Assured Consulting Solutions interview process

Assured Consulting Solutions candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Before anything technical happens, someone has to decide which rung of the ladder your loop is calibrated to, and that decision sets the bar for every round after it. It comes from how you describe scope, not from your title, because titles do not convert cleanly between companies. The weak version of the answer is team size and years. The strong version names the largest change you shipped where nobody reviewed the design, what would have broken if you had been wrong, and what you were paged for. Get the level said out loud on this call, because the range and the loop both follow from it.

What to demonstrate

  • Whether the scope in your own account maps onto a level the team actually has an opening at, so a mismatch ends the process cheaply rather than after four interviewers have spent a day
  • Whether your title needs re-mapping: the same word describes very different amounts of independent decision-making at a twenty-person company and a ten-thousand-person one
  • Whether your compensation expectation can be filled at that level in the structure the role pays in, which is why the number gets asked for before any engineer is scheduled

How to prepare

  • Write down two changes from the last two years: the largest one you designed with nobody reviewing the design, and the largest one where someone more senior did. Lead with the first when scope comes up, and be ready to say which parts of the second were yours
  • Ask which level the loop is calibrated to and what changes at the level above it, then plan your weeks from that answer rather than from the posting
  • Settle a total-compensation range beforehand with the split named, base against bonus against equity and its vesting period, so a question about numbers gets a number instead of the word market

Technical Discussion

Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.

What to demonstrate

  • Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
  • Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
  • Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
  • Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply

How to prepare

  • For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
  • For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
  • Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last

Leadership Interaction

Because the format is not fixed, the first job in the room is classification. Listen to the opening question and decide what it is: a probe into work you have already described, a fresh problem to solve now, or a conversation about how you operate. Each wants a different register, and the common failure is forcing a rehearsed structure onto a question that did not ask for it. Running a full design ritual on a ten-minute debugging question reads as not listening. When you cannot tell which it is, ask how long they want to spend and answer at that depth.

What to demonstrate

  • Whether the shape of your answer matches the question, so a yes-or-no gets answered before it is justified and an open prompt gets a direction before a detour
  • Whether you check how much depth is wanted instead of deciding for them, and whether you stop when the answer is complete rather than continuing until someone interrupts
  • Whether you can be redirected in the middle of an answer without restarting it from the beginning
  • Whether a question outside your experience gets an honest boundary followed by reasoning from what you do know, instead of a confident answer with nothing behind it

How to prepare

  • Rehearse one project at three lengths, roughly thirty seconds, three minutes, and a full walkthrough at the depth of a design review, and practise switching between them when someone interrupts mid-telling
  • Have someone ask you five questions of deliberately mixed type in one sitting without telling you the types, and score only whether you identified each one correctly before you started answering
  • Draft the sentence you will use to check depth, along the lines of asking whether the short version is useful here or they want the detail, and use it in a real conversation this week so the day of the round is not its first outing

Cultural Fit Assessment

What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.

What to demonstrate

  • Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
  • Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
  • Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly

How to prepare

  • Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
  • Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
  • Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live

Where candidates lose points

Writing the state change to the database and publishing the event in the same code path

No transaction spans a relational database and a message broker, so a crash between the two leaves one done and the other not, and the failure is asymmetric in both orderings: publish-then-commit invents events for state that never existed, while commit-then-publish silently loses events for state that does. Retrying the publish after the commit is not a fix, because the process can die before the retry runs. The working shape is an outbox row written inside the same transaction plus a relay that publishes it at least once, which makes consumer-side idempotency mandatory rather than optional. Note also that 'exactly-once' in a stream processor means exactly-once processing within that system's own read-process-write transaction, and says nothing at all about an external side effect such as charging a card.

Deriving the business date from the UTC timestamp

Posting date, value date and the processor's settlement date are three different dates, determined by cutoff times, business-day calendars and holidays rather than by midnight UTC. A movement recorded at 23:50 on one side of a cutoff belongs to the next business date, so computing business_date as created_at::date makes daily totals disagree with every statement and every settlement file. The signature is a reconciliation break that resolves itself the following day and then reopens, which reads like a flaky job and is actually a data model that is missing a column: business_date has to be stored explicitly and set from the cutoff rule, with the timestamptz kept separately for ordering.

Reading the constraints as preamble rather than as part of the problem

The bounds are usually there to eliminate the obvious approach: n up to 10^5 makes an O(n^2) scan roughly 10^10 operations, far outside any per-test time budget, and an input larger than memory rules out loading it at all. When a bound is not given, ask for it, then say out loud which approach it kills.

Arguing past a hint

When the interviewer asks what happens for a particular input or floats a different data structure, stop and take it seriously; it is almost always a correction rather than idle curiosity. Talking over it converts a recoverable wrong turn into a data point about how you handle review.

Questions & practice

Detect duplicate-charge bursts in an out-of-order authorisation stream

Hard · sliding window · event time · watermarks · deduplication

Authorisations arrive as (instrument_token_id, amount_minor, currency, event_time, arrival_time) at roughly 3,000 per second, up to 60 seconds late and out of order. Flag any token with three or more authorisations of identical (amount_minor, currency) inside any 10-minute window of event time. Report each flag once, as early as correctness allows. State memory per key and in total, how late an event you will accept, and what you do with one that arrives after you have already reported — or already declined to report — that window.

Approach

  1. Key state by (instrument_token_id, amount_minor, currency), not by token: the predicate is about identical amounts, so the window belongs to the triple. Each key holds its own event-time-ordered deque.
  2. In-order, this is two pointers: on insert, pop from the front while front <= new - 10 min, then flag if the deque reaches length 3. Amortised O(1) per event, memory proportional to that key's window occupancy.
  3. Out-of-order arrival breaks append-only monotonicity, so insert in position instead. With lateness bounded at 60 seconds the insertion point is always near the tail, so a short sorted vector or a 600-bucket per-second ring keeps it O(w) with tiny w; a balanced tree per key is correct but over-built for a one-minute reorder.
  4. Drive decisions off a watermark of max(event_time seen) - 60 s, never off wall clock. Anything older than the watermark is too late to change an answer and is counted in a late_dropped metric. Without an explicit watermark you have still chosen a lateness policy — you just cannot state it or test it.
  5. Report-once needs its own state: per key, a set of already-flagged 10-minute window ids. A later event inside an already-flagged window must not re-flag, and a late event that completes a window you never flagged must flag — which is why state is retired at window close plus the lateness bound, not at window close.
  6. Size it: state lives 660 seconds, so at 3,000 events per second there are about 1.98 million in flight, plus one flagged-window set per active key. Bound total memory explicitly and shed by key age, and say what shedding costs — a shed key can miss a flag, making the threshold a product decision rather than a tuning knob.

Worked solution (40 min)

  1. Write the key, the window, the watermark and the report-once rule in four lines before any code; every later bug is one of these left implicit.
  2. Implement per-key state as a sorted deque of event times plus a set of flagged window ids, both retired at watermark - 660 s.
  3. Fixture 1, in order: identical amounts at t, t+1 min and t+9 min produce one flag. A fourth at t+11 min evicts t+1 exactly under the half-open rule pop while front <= new - 10 min, leaving two events and no second flag; change that comparison to strict < and the same input flags twice. Pick one and write it in the spec.
  4. Fixture 2, out of order: deliver the same three events as t+9, t, t+1 and assert the flag fires on the third arrival with the same window id as fixture 1.
  5. Fixture 3, too late: the t event arrives 90 seconds after the watermark passed it, so no flag fires and late_dropped increments by one.
  6. Replay all three fixtures under twenty random arrival orders and assert the flag set is identical whenever every event is inside the lateness bound.

Expected result: Fixture 1 flags exactly once, at the t+9 event; the t+11 event does not flag under the half-open rule and does under the closed one, which is the boundary the fixture exists to pin. Fixture 2 yields the same single flag with the same window id. Fixture 3 yields no flag and late_dropped == 1. All twenty shuffles produce an identical flag set.

Verify it yourself

  • Replaying the same events in any arrival order within the lateness bound produces identical flags — the only property that makes this job re-runnable.
  • Per-key state never exceeds that key's events within 660 seconds; assert it rather than hope.
  • Every flag's window id derives from event time alone; arrival_time appears only in the watermark computation and the lateness metric.

Follow-up

  • Make the threshold and window configurable without rebuilding all in-flight state on every change. What does that constrain in the data structure?
  • Two stream partitions hold events for the same token. What does that force the partitioning key to be?
  • An event arrives three hours late. Does any answer change, and who is told?

Derive per-account balances and catch unbalanced transactions

Easy · aggregation · double-entry · streaming

You are given ledger_entry rows streamed in entry_id order: transaction_id, account_id, direction (debit or credit), amount_minor (a positive int64), currency, business_date. Up to 500 million rows, at most 20 million distinct (account_id, currency) pairs, and the entries of one transaction are contiguous in the stream. In a single pass with no re-reads, return the closing balance per (account_id, currency) and the transaction_id of every transaction whose entries do not sum to zero within each currency. State your time and space bounds.

Approach

  1. Normalise the sign at read time from direction, not from the amount: signed = +amount_minor for debit, -amount_minor for credit (state which convention you picked). The schema constrains amount_minor > 0 precisely so the sign lives in exactly one place.
  2. Hold one hash map keyed (account_id, currency) to an int64 running total. Twenty million keys at 16 bytes of payload plus map overhead is order 1 GB in most runtimes — quote the number, and offer the fallback: partition the stream by hash(account_id) % P and run P passes for 1/P of the memory.
  3. Ride the zero-sum check on the same pass. Because a transaction's entries are contiguous, keep a tiny currency -> int64 map for the current transaction_id only, test it against zero on the boundary and at EOF, then clear it. That is O(currencies in one transaction), typically one or two.
  4. Bound the arithmetic explicitly. Int64 holds about 9.22e18, so overflowing one account across 500 million entries needs an average of 1.8e10 minor units per entry — safe here, but use a checked add so an adversarial file fails loudly rather than wrapping.
  5. Complexity: O(n) time, O(distinct account-currency pairs) space, one sequential pass, no sort. The zero-sum check adds no asymptotic cost, which is the argument for doing it here rather than in a second job.

Follow-up

  • Entries of a transaction are no longer contiguous. What does the zero-sum check cost now, and which is cheaper: buffering open transactions or an external sort on transaction_id?
  • How would you produce the same balances as of an arbitrary business_date without a second full scan?
  • The job is restarted after a crash halfway through the file. What makes the second run produce identical output?

Compute peak held exposure from overlapping authorisation holds

Medium · sweep line · intervals · authorisation holds

An account has up to 2 million authorisations on one business date: (auth_id, amount_minor, created_at, expires_at) with the hold live over the half-open interval [created_at, expires_at), plus capture events (auth_id, captured_minor, captured_at) that reduce the hold at captured_at, and explicit reversals that drop the remainder to zero. Timestamps are microsecond-precision timestamptz. Return the maximum total held amount across the day and the earliest instant it is reached, with complexity. Then say what changes if the deliverable is the peak per minute instead.

Approach

  1. Expand each authorisation into signed delta events rather than reasoning about intervals: +amount at created_at, -remaining at expires_at, -captured_minor at each captured_at, -remaining at reversed_at. The problem collapses to a running sum over a sorted event list.
  2. Sort the 2n to 4n events by (timestamp, sign) with negative deltas ordered first on a tie. The half-open convention forces that: at exactly expires_at the hold is already gone, so a - must land before a + at the same instant or you report a one-microsecond peak that never existed. O(n log n) time, O(n) space.
  3. Sweep once, tracking running, best and best_at, taking the first instant that attains the maximum. Say out loud which tie rule you are using — 'the peak' is ambiguous when the same level is reached twice, and the caller needs to know which instant they are being handed.
  4. If timestamps are bucketed (1,440 minute buckets for the per-minute variant), drop the sort for a difference array: add the delta at the start bucket, subtract at the end bucket, prefix-sum once. O(n + B) time and O(B) space, strictly better, at the cost of answering only at bucket resolution.
  5. Assert the invariant during the sweep: running must never go negative. A negative total means a capture exceeded its authorisation, which is an invariant violation upstream rather than a sweep bug — fail loudly instead of clamping at zero and reporting a plausible number.
  6. Handle carry-in: a hold created before the window contributes its remaining amount as the sweep's initial value, not as a + event inside the window. Omitting that is the off-by-a-day that makes the first minute of every day look artificially quiet.

Follow-up

  • An incremental authorisation raises an existing hold after the fact. Where does that event go, and does it disturb the tie rule?
  • You now need the peak for 10 million accounts inside a nightly window. What changes, and what must the partitioning key be?
  • The peak sizes a funding transfer. Does the business date or the timestamp decide which day that transfer lands on?

Version a loan schedule instead of soft-deleting posted instalments

Medium · audit history · soft deletes · temporal queries

loan_instalment is keyed by (loan_id, schedule_version, instalment_no) and carries due_date, principal_minor, interest_minor, fee_minor, paid_principal_minor, paid_interest_minor, status, days_past_due, effective_from (date) and superseded_at (timestamptz). A borrower defers two payments on 2026-03-14, and instalments 1 to 6 already have allocations posted against them. Model exactly what the deferral writes, and write the query that returns the schedule as the borrower saw it on an arbitrary date. Say why stamping the old rows with deleted_at, or updating them in place, fails an audit.

Approach

  1. Treat the deferral as an insert, not an edit: write a complete new schedule_version with effective_from = the deferral instant on 2026-03-14, and in the same transaction stamp superseded_at on every row of the outgoing version with that identical instant, so the two version windows are half-open and adjacent rather than overlapping. Instalments 1 to 6 are reproduced unchanged, because they are what the borrower was told and what was posted to the ledger.
  2. Write the as-of read as a version selection, not a row filter: WHERE loan_id = $1 AND effective_from <= $2 AND (superseded_at IS NULL OR superseded_at > $2), then assert that exactly one schedule_version comes back - COUNT(DISTINCT schedule_version) = 1, not one row - so an overlapping window raises rather than silently returning two interleaved schedules under one instalment_no.
  3. Fix the type mismatch before writing that predicate: effective_from is declared date and superseded_at timestamptz, so comparing them casts the date at the session TimeZone and two readers in different zones select different versions near midnight. Put both columns in one domain, timestamptz, keep the window half-open with effective_from inclusive and superseded_at exclusive, and resolve a bare as-of date to an instant once, in the loan's booking timezone, at the edge of the system.
  4. Say what deleted_at loses. It records that a row stopped being current but not what replaced it or from when, it leaves every downstream query obliged to remember deleted_at IS NULL, and one query that forgets double-counts the schedule. Versioning puts the same information in the primary key where it cannot be forgotten.
  5. Close the arithmetic: under the loan's stated day-count convention the new version must still sum to outstanding principal plus scheduled interest to the minor unit, with the per-period rounding residual placed in one named instalment, conventionally the last, rather than smeared across the tail.
  6. Index (loan_id, effective_from DESC) for the as-of lookup, keep superseded versions online rather than archiving them, and enforce with a trigger that no UPDATE touches a row whose paid_principal_minor or paid_interest_minor is non-zero.

Worked solution (25 min)

  1. Insert a 12-instalment version 1 with effective_from at origination, then allocate payments against instalments 1 to 6.
  2. Apply the deferral: insert version 2 effective at the 2026-03-14 deferral instant, reproducing instalments 1 to 6 byte for byte and re-amortising 7 to 12, and set superseded_at on all twelve rows of version 1 to that same instant in the same transaction.
  3. Write the as-of query with the version-selection predicate and run it for instants resolved from 2026-03-01 and 2026-03-20 in the loan's booking zone, and for the deferral instant itself.
  4. Compare SUM(principal_minor) per version against the original principal and locate the rounding residual.

Expected result: Two schedule_versions for the loan. The as-of query returns version 1 for 2026-03-01 and version 2 for 2026-03-20, instalments 1 to 6 are identical across both versions, and SUM(principal_minor) equals the original principal exactly in each version.

Verify it yourself

  • The live-version invariant is per version, not per row: SELECT COUNT(DISTINCT schedule_version) FROM loan_instalment WHERE loan_id = $1 AND superseded_at IS NULL returns 1, while COUNT(*) over the same predicate returns that version's instalment count - 12 here - so an assertion of one NULL row per loan would fail on the correct design.
  • Run the as-of query for 200 random dates spanning the loan and assert it returns exactly one schedule_version every time.
  • Run the as-of query at exactly the deferral instant and assert version 2 alone: the windows are half-open and version 1's superseded_at equals version 2's effective_from, so the boundary belongs to exactly one version.
  • Attempt an UPDATE on an instalment with paid_principal_minor > 0 and confirm the trigger rejects it.
  • Change the session TimeZone and re-run the boundary instant; with both columns timestamptz the selected version must not change, which is the test that the date column was actually retyped.

Follow-up

  • What does days_past_due mean for an instalment that exists in two versions with different due_dates?
  • A payment arrives allocated to an instalment_no that exists only in the superseded version. What do you do with it?
  • How do you prove the ledger postings made under version 1 still reconcile once version 2 exists?

Stop the capture-and-refund join from double-counting merchant money

Medium · join fan-out · conditional aggregation · grain

Report captured_minor, refunded_minor and net_minor per merchant per business_date. payment_intent holds intent_id, merchant_id, amount_minor, captured_minor, refunded_minor. ledger_entry holds transaction_id, account_id, direction, amount_minor, source_type in (capture, refund, fee, chargeback), source_id = the intent_id as text, and business_date; every capture posts one debit and one credit. A draft joins payment_intent to ledger_entry twice, once filtered to captures and once to refunds, and sums both. On intents with two partial captures and one refund the totals are wrong. Name both multiplications precisely and write the correct query.

Approach

  1. Name the first multiplication by cardinality: intent to capture entries is one-to-N and intent to refund entries is one-to-M, so joining both yields N times M rows per intent. Every capture row is then counted M times and every refund row N times, and the two columns are inflated by different factors, which is why the totals look almost right rather than obviously broken.
  2. Name the second, which is easier to miss: double-entry means each capture posts two legs, so summing every entry of a capture counts the amount twice regardless of joins. Select the single leg by ACCOUNT - the merchant's settlement payable, which carries exactly one leg of every capture and of every refund - and not by direction. Direction is not a substitute for that filter: a capture credits the payable and a refund debits it, so adding direction = 'credit' keeps the captures and silently drops every refund leg, leaving refunded_minor at exactly zero on a query that otherwise reads as correct.
  3. Reject the reflexive fixes: SELECT DISTINCT and SUM(DISTINCT ...) de-duplicate values, not rows, so two genuine captures of equal amount collapse into one and the total is wrong in the other direction while looking tidier.
  4. Prefer a single pass with conditional aggregation over ledger_entry, restricted to that one account: SUM(amount_minor) FILTER (WHERE source_type = 'capture') and the same for refunds, grouped by merchant and business_date, joining payment_intent only to reach merchant_id. One scan, no intent-level fan-out, and fees or chargebacks are added as another FILTER rather than another join. amount_minor is a positive magnitude, so net_minor is captured minus refunded; the signed form over the same rows, SUM(CASE WHEN direction = 'credit' THEN amount_minor ELSE -amount_minor END), returns the same net in one column and is the check on the sign convention.
  5. Keep aggregate-then-join in reserve for when per-intent detail is required: aggregate each side to intent grain in its own CTE, then join the two aggregates one-to-one. LATERAL works too and reads better when the right side needs the left's key.
  6. Get the account predicate, the join column and the date right. Each merchant has its own settlement payable, so the predicate is a join to the account dimension (a.account_type = 'merchant_payable' AND a.merchant_id = pi.merchant_id); a single $settlement_account literal is only correct for a one-merchant report. source_id is text and intent_id is uuid, and PostgreSQL has no implicit cast between them, so one side must be cast and the side decides which index stays usable - pi.intent_id = e.source_id::uuid probes payment_intent's primary key per entry row, e.source_id = pi.intent_id::text probes an index on ledger_entry(source_id), and if neither fits the driving table the planner hashes both sides, which is fine for a day's report and not for a single-intent lookup. Group on ledger_entry.business_date rather than posted_at::date, because they differ across the cutoff, and decide explicitly whether a later-dated refund reduces its own day or the original capture's.

Follow-up

  • A refund lands on a later business_date than its capture. Which day does net_minor move, and what does that do to a daily merchant payout?
  • Add scheme fees, which arrive netted into a batch total rather than per transaction. How does the query change, and what can it no longer claim?
  • How would you reconcile this output against payment_intent.captured_minor, and which one is authoritative when they disagree?

What do you know about GIS software?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Clarify what is being asked and what a complete answer contains.
  3. Work from the requirement backwards to the design.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Do you have experience with Java?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Say what you would check first and why it is the highest-information step.
  3. Work from the requirement backwards to the design.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Capture endpoint that survives concurrent duplicate retries

Medium · idempotency · concurrency · unique constraints · retries

POST /payments/{intent_id}/capture carries an Idempotency-Key header and an amount. Callers retry on timeout, and two retries can land concurrently on different instances. You have idempotency_key(id, scope, key, UNIQUE(scope,key), request_fingerprint bytea, status in_progress|completed|failed, response_status, response_body jsonb, locked_at, completed_at, expires_at). The processor capture takes 200 to 2,000 ms. Design the path so exactly one capture reaches the processor, every duplicate receives the identical response, and a crash between the processor call and your commit converges. Name the single statement that is the concurrency control.

Approach

  1. The concurrency control is INSERT INTO idempotency_key (...) VALUES (...) ON CONFLICT (scope, key) DO NOTHING RETURNING id, and it must commit before the processor call. A returned row means you own the effect; zero rows means you lost and must read the winner's result. A SELECT-then-INSERT check cannot substitute: two requests both read 'absent' and both proceed, and the window is exactly the concurrency you are defending against.
  2. Commit the in_progress row in its own short transaction. A concurrent duplicate insert blocks on an in-flight conflicting insert until that transaction ends, so wrapping the 2 s processor call in the same transaction turns every duplicate into a 2 s lock wait and a retry storm into pool exhaustion.
  3. Define loser behaviour per status rather than uniformly: completed replays response_status and response_body unchanged; in_progress returns 409 with Retry-After and performs nothing; failed splits by cause, since a terminal processor decline should replay but a transport failure should let the key be retried. Getting this wrong in the safe direction (replay a decline) is better than returning an error for a capture that succeeded.
  4. Fingerprint the canonicalised body with SHA-256 and compare on every hit. Same key with a different amount is a client bug and must return 409 or 422, never the cached response, because returning the cached body silently captures the old amount and looks successful.
  5. Pass the same key to the processor so deduplication holds end to end, and generate it once at the originating caller. A key regenerated per attempt leaves every line of idempotency code in place while disabling the mechanism entirely.
  6. Converge after a crash by querying the authoritative side rather than guessing: a reaper picks up rows in_progress past locked_at plus a bound, asks the processor for that key or the intent's processor_reference, and completes the row from the answer. Set expires_at longer than the caller's full retry schedule and document that a replay after expiry is a new request.

Worked solution (30 min)

  1. Implement the endpoint with the ON CONFLICT DO NOTHING insert committed before the processor call, and a processor stub that counts calls and sleeps 1,500 ms.
  2. Drive 50 concurrent identical requests through two application instances.
  3. Repeat with the same key and the amount changed by one minor unit.
  4. Kill the instance between the stub's response and the local commit, restart, and run the reaper.

Expected result: Exactly one processor call across the 50 requests. One request returns the capture; the other 49 either replay the stored body unchanged or return 409 in_progress, and none returns a 500 or a second capture. The changed-amount request returns 409 or 422 and never the cached body. After the crash, the reaper completes the row from the processor's own answer, leaving exactly one capture.

Verify it yourself

  • Time the 49 losers: if they each take about 1,500 ms, the winner is holding the key row inside the transaction wrapping the processor call.
  • The stub's call counter reads 1, not 1 plus a retry.
  • Swapping the constraint for a check-then-insert reproduces two captures under the same load; keep that as the regression test rather than trusting the fix.

Follow-up

  • The processor does not honour idempotency keys. What is the end-to-end design now, and what can you no longer promise?
  • Two merchants send the same key value. What makes that safe?
  • You keep keys for 24 hours at 3,000 requests/s. Size the table and the index, and say what expires them.

Event gateway heap grows until the daily restart

Hard · memory leak · unbounded queue · backpressure · heap analysis

The outbound event gateway holds RSS at 1.2 GB for about nine hours, then climbs to its 4 GB limit and is OOM-killed. It runs per-destination queues with bounded concurrency and a retry schedule extending to 24 hours. Event rate, destination count and throughput are flat across the window. Heap dumps show the largest retained set is a map keyed by destination id. Give the ordered diagnostic checklist and the cause, and distinguish a leak from growth that is merely unbounded.

Approach

  1. Separate the two possibilities first, because they need different fixes and look identical on an RSS graph. A leak retains objects unreachable from any live work; unbounded growth retains objects that are genuinely still needed because nothing caps how many there are. The discriminator is whether the retained set corresponds to real pending work, so count the map's entries against the destination count and its values against in-flight events.
  2. Read the dominator tree, not the instance histogram. A histogram says byte[] is large, which is always true. The dominator tree says what holds it. A destination-keyed map with more entries than destinations means per-destination state is created and never removed, which is a leak; a map with exactly the destination count whose values are large means per-destination state grows without bound, which is a design limit.
  3. Follow the retry schedule to the worst case. A 24-hour schedule with at-least-once delivery means an event for a dead destination is retained for a day, so an unbounded in-memory queue accumulates a day of that destination's traffic. Compute it explicitly: events per second for the busiest destination times 86,400 times average payload size, then compare against the 2.8 GB of headroom. If it lands in the same order of magnitude, the design is unbounded rather than leaking.
  4. Use the shape of the curve to date the trigger. RSS flat for nine hours and then climbing does not mean slow accumulation all along; it means accumulation started at a point in time, so find what changed at hour nine rather than what the process does continuously. Measure the slope of the climbing segment alone, convert it to retained bytes per second, and compare it against one destination's event rate times payload size: a match to a single destination points at a specific endpoint that went unreachable then, while a match to the total points at a global retention path such as a delivered-event cache or an unbounded metrics label set.
  5. Fix by bounding, not by enlarging. Give each per-destination queue a maximum depth and a maximum age, and spill beyond it to outbox_event, which is durable, already exists and already carries aggregate_version for ordering. Memory then scales with in-flight concurrency rather than with backlog, and a destination down for a day costs disk, not heap.
  6. Prove it rather than watching it. Re-run with one destination black-holed and assert RSS stays inside a stated band for an hour. A fix that only raises the limit passes a short soak and still fails, just later, which is why the test has to inject the failure rather than wait for it.

Follow-up

  • Spilling to the table means some events for a destination are in memory and some are in PostgreSQL. How do you keep per-destination order across both?
  • The bounded queue is full and the event is already committed in the outbox. What does the enqueue path do, and what does it return?
  • Which single metric would have caught this at hour two rather than hour nine?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

  • Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
  • Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
  • Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

  • Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
  • Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
  • Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

  • Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
  • Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
  • Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

  • For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
  • Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
  • Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
  • Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

  • Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
  • Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
  • Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
  • Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

  • Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
  • Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
  • Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

  • Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
  • Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
  • Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

'We rewrote it in a better framework' is not a story

A migration is a cost you chose to pay, not an achievement. The story is what the old system made expensive, what you measured before committing, what kept serving traffic during the cutover, and what you would have done if the numbers had come back flat. Without those, a rewrite reads as taste.

What is your experience with SDET (Software Development Engineer in Test)?

Medium · behavioural and engineering judgement

Approach

  1. Give the blast radius: what could have broken, and what you measured.
  2. Name the disagreement and how you resolved it with evidence.
  3. Close with what you would do differently, concretely.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?

Can you walk us through your relevant experience as it relates to this project?

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement and how you resolved it with evidence.
  3. Give the blast radius: what could have broken, and what you measured.

Follow-up

  • How did you know your change caused the improvement?
  • What would you do differently if you ran that again?

Tell us about a time you had to adapt your workflow to meet a project's changing needs.

Medium · behavioural and engineering judgement

Approach

  1. State the situation in two sentences and spend the rest on the reasoning.
  2. Give the blast radius: what could have broken, and what you measured.
  3. Close with what you would do differently, concretely.

Follow-up

  • How did you know your change caused the improvement?
  • What did you decide not to do, and why?

Can you describe a time you applied your technical skills to solve a complex system challenge?

Medium · behavioural and engineering judgement

Approach

  1. Give the blast radius: what could have broken, and what you measured.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Pick a story where you made the decision, not one where you watched it.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?