Interview guides · Australian Payments Plus
Australian Payments Plus Software Engineer Interview Questions & Guide 2026
Australian Payments Plus software engineer prep: algorithm drills where every complexity claim has to name the assumption under it.
The 60-second brief
As a Software Engineer at Australian Payments Plus (AP+ ), you are at the heart of Australia’s critical payment infrastructure. AP+ brings together iconic brands like eftpos, BPAY, and NPP, creating a unified platform that facilitates the movement of money for millions of Australians every day. Your work directly impacts the reliability, speed, and security of the nation's payment ecosystem.
Prepare in one language you know well enough to debug in rather than the one you think reads best. Under a clock an unfamiliar language costs you standard-library lookups and iteration mechanics, and that time comes out of your thinking budget, not your typing budget.
Focus areas
- Store money as integer minor units
- Consume webhooks duplicated, delayed and out of order
- Model money movement as balanced double-entry postings
Overview
This role is both highly technical and strategically significant. You will not only be writing code but also navigating the complexities of high-scale transaction systems and legacy integration. Whether you are working on core payment gateways or internal developer tooling, you will be expected to balance rapid innovation with the stability required by the financial services sector. It is a challenging, high-impact environment where your contributions have tangible, real-world consequences.
While the hiring process is rigorous, the team values candidates who can demonstrate a balance of technical proficiency and a collaborative, pragmatic mindset when solving complex engineering problems.
The Australian Payments Plus interview process
Australian Payments Plus candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Initial Screening
The person on this call usually cannot evaluate your code and does not need to. They write a short paragraph, and that paragraph is what a hiring manager skims when deciding who to put on your loop. So the test is not whether your work was hard, it is whether a non-engineer can repeat it correctly. Name systems by what they did rather than by their internal codename, give each project a shape (what was breaking, what you changed, what happened after), and keep the whole walkthrough near ninety seconds. Depth that cannot survive a paraphrase reads as vagueness.
What to demonstrate
- Whether a non-engineer can restate your projects without distorting them, since their paraphrase is what travels to the hiring manager, not your sentences
- Whether each project has a shape rather than a stack list: the failure or constraint, the change you made, the result and how it was measured
- Whether you can say what was yours inside a team project without either inflating it or disappearing into the plural
How to prepare
- Rewrite each headline project as two sentences with no internal system names and no acronyms outside your company, then say them to someone outside engineering and have them repeat them back. Fix whatever came back wrong
- Attach one measured number to each project: the baseline, the change, and the window it was measured over. Where nothing was ever measured, say that plainly rather than reaching for a plausible percentage
- Time the background walkthrough against a clock. If it runs past two minutes, compress the earliest role to a single clause and spend the recovered time on the most recent one
Technical Assessment
The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.
What to demonstrate
- In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
- In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
- Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes
How to prepare
- Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
- Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
- For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first
Behavioral Interview
Your first answer is not really what is scored. It buys the follow-up questions, and those decide the round. An interviewer with fifteen minutes takes one thread and pushes on it four or five times, so a story you can only tell at a single level of detail collapses under the third why. That is an argument for fewer stories known deeply rather than one prepared per prompt. Four or five pieces of work you can still explain down to the code you changed and the argument you had about it will cover nearly anything asked in this round.
What to demonstrate
- Whether a story holds as the questioning moves from what you did to why that instead of the alternative, and then to what you would change knowing what you know now
- Whether you can re-cut a project to answer the question actually asked rather than delivering a rehearsed block that answers an adjacent one
- Whether your level of detail is chosen rather than habitual: going down to the schema when the question is about the data model, staying out of it when the question is about the person who disagreed with you
How to prepare
- Pick four projects and write the chain out four levels deep for each: what you did, why that, why not the alternative, and what would have to be true for the alternative to have won. Where you cannot reach the fourth level, you have a placeholder rather than a story
- Have someone ask why three times in a row on a single thread with nothing else added, and mark the point where you start repeating a sentence you already said. That point is where the interviewer stops learning anything
- Build a one-page index instead of an answer bank: the common prompts in this round (disagreement, a failure that was yours, thin requirements, a deadline you missed, work you inherited) mapped to which of your four projects you would use for each, so the choosing is done now rather than while an interviewer waits
Where candidates lose points
Calling a compensating action a rollback
A saga's compensation is a new, externally visible business event, not an undo. A refund after a capture leaves both movements on the customer's statement, may not return the scheme fee, and lands days later rather than immediately. Designing a multi-service flow as though the compensation restores the prior state produces flows that turn out to be unimplementable at the final step, when the thing that needs undoing has already left the building. The sequence has to be ordered so the irreversible step is last and the reversible ones precede it, with an explicit pending state shown to the customer while a compensation is in flight.
Retrying a charge after a timeout
A timeout is not a failure; it is an unknown outcome, and the request may have been processed in full with only the response lost. Re-sending it without an idempotency key that the processor itself honours produces a duplicate charge, which is a customer-visible incident and usually a dispute. The correct handling is to treat the state as unknown, query the processor for that key or client reference, and only then decide. The mechanism also depends on the key being generated once by the caller and reused across every attempt — generating a fresh key per retry turns the whole scheme into a no-op while leaving all the code that appears to implement it in place.
Answering a debugging question with a guess instead of a bisection
Give a procedure that halves the search space at each step: confirm the symptom reproduces, establish the last known-good version, input or timestamp, then bisect over commits, over the data, or over the layers of the request path. A plausible cause with no way to confirm it is the same move whether it happens to be right or wrong, which is why it scores nothing.
Writing code before the input contract is pinned down
Before the first line, state the types, the size bounds, whether duplicates, negatives or an empty input are possible, whether the input is sorted, whether you may mutate it, and what the function returns when nothing matches. Every one of those answers changes the code, and discovering one at minute twenty costs a rewrite you no longer have time for.
Questions & practice
How do you debug a production issue when the root cause is unclear?
Medium · data structures and algorithms
Approach
- Restate the input: its shape, its size, and what is guaranteed about it.
- Walk one small example through your approach before writing the whole thing.
- State the target complexity and say which constraint rules the naive version out.
Follow-up
- What is the worst case, and how likely is it on real data?
- How does this change if the input no longer fits in memory?
Answer as-of balance queries over an append-only entry log
Hard · prefix sums · offline queries · append-only
Given 400 million ledger_entry rows (entry_id, account_id, direction, amount_minor, currency, business_date) and 2 million queries of (account_id, currency, as_of_date) asking for the balance at the end of that business date, produce every answer. The obvious solution — per query, sum that account's entries with business_date <= as_of_date — is correct. Say precisely why it will not finish, then give one that will, with time and space complexity. Corrections are posted as new entries carrying their own business_date.
Approach
- Cost the naive version in numbers before rejecting it. Spread uniformly over 20 million accounts, each query touches about 20 rows behind a per-account index and 2 million queries is 4e7 row touches — perfectly fine. The problem is skew: one pooled clearing or merchant settlement account holding 3e7 entries, taking 10% of the queries, is 6e12 row touches. Name the skew; 'n is large' is not the reason.
- The structural fact that buys a cheap answer: entries are append-only and never updated, so a prefix sum over an account's entries ordered by
(business_date, entry_id)is stable — nothing behind position i can change. No mutable-balance design offers that, and it is why the storage is worth paying for. - Offline sweep, when all queries are known up front: externally sort entries by
(account_id, currency, business_date, entry_id)and queries by(account_id, currency, as_of_date), then merge-walk both with a running sum, emitting each query's answer as the sweep passes its date. O((n + q) log(n + q)) dominated by the sort, O(1) beyond sort buffers, one sequential pass over each input instead of 2 million random seeks. - Online alternative: materialise end-of-day snapshots — one row per
(account_id, currency, business_date)that had activity, holding the cumulative total. A query becomes one index seek for the latest snapshot at or beforeas_of_date, O(log n) per query, over far fewer rows than n. Use snapshots when queries arrive singly and the sweep when they arrive as a batch. - Corrections are the subtlety: an entry posted today but dated back changes historical answers, so every snapshot for that account from that date forward is stale. Either keep a Fenwick tree over dates per account (O(log D) update and prefix query) or recompute that account's snapshots from the corrected date onward. Then be precise about what reproducibility means — yesterday's statement is reproducible as of a stated snapshot time, not identical forever.
- Bound the resources: int64 sums throughout, no float; 400 million rows at roughly 48 bytes of the columns you actually need is about 19 GB, so the sort is external and its fan-out is chosen from the sort buffer, not from the row count.
Follow-up
- One account holds 30% of all entries. What does the external sort do with it, and what would you do for that one key instead?
- Queries now arrive online at 500 per second. Which design survives, and what does keeping the other one warm cost?
- A correction lands with a
business_date90 days back. Which snapshots are now wrong, and how does a reader find out?
Detect duplicate-charge bursts in an out-of-order authorisation stream
Hard · sliding window · event time · watermarks · deduplication
Authorisations arrive as (instrument_token_id, amount_minor, currency, event_time, arrival_time) at roughly 3,000 per second, up to 60 seconds late and out of order. Flag any token with three or more authorisations of identical (amount_minor, currency) inside any 10-minute window of event time. Report each flag once, as early as correctness allows. State memory per key and in total, how late an event you will accept, and what you do with one that arrives after you have already reported — or already declined to report — that window.
Approach
- Key state by
(instrument_token_id, amount_minor, currency), not by token: the predicate is about identical amounts, so the window belongs to the triple. Each key holds its own event-time-ordered deque. - In-order, this is two pointers: on insert, pop from the front while
front <= new - 10 min, then flag if the deque reaches length 3. Amortised O(1) per event, memory proportional to that key's window occupancy. - Out-of-order arrival breaks append-only monotonicity, so insert in position instead. With lateness bounded at 60 seconds the insertion point is always near the tail, so a short sorted vector or a 600-bucket per-second ring keeps it O(w) with tiny w; a balanced tree per key is correct but over-built for a one-minute reorder.
- Drive decisions off a watermark of
max(event_time seen) - 60 s, never off wall clock. Anything older than the watermark is too late to change an answer and is counted in alate_droppedmetric. Without an explicit watermark you have still chosen a lateness policy — you just cannot state it or test it. - Report-once needs its own state: per key, a set of already-flagged 10-minute window ids. A later event inside an already-flagged window must not re-flag, and a late event that completes a window you never flagged must flag — which is why state is retired at window close plus the lateness bound, not at window close.
- Size it: state lives 660 seconds, so at 3,000 events per second there are about 1.98 million in flight, plus one flagged-window set per active key. Bound total memory explicitly and shed by key age, and say what shedding costs — a shed key can miss a flag, making the threshold a product decision rather than a tuning knob.
Worked solution (40 min)
- Write the key, the window, the watermark and the report-once rule in four lines before any code; every later bug is one of these left implicit.
- Implement per-key state as a sorted deque of event times plus a set of flagged window ids, both retired at
watermark - 660 s. - Fixture 1, in order: identical amounts at t, t+1 min and t+9 min produce one flag. A fourth at t+11 min evicts t+1 exactly under the half-open rule
pop while front <= new - 10 min, leaving two events and no second flag; change that comparison to strict<and the same input flags twice. Pick one and write it in the spec. - Fixture 2, out of order: deliver the same three events as t+9, t, t+1 and assert the flag fires on the third arrival with the same window id as fixture 1.
- Fixture 3, too late: the t event arrives 90 seconds after the watermark passed it, so no flag fires and
late_droppedincrements by one. - Replay all three fixtures under twenty random arrival orders and assert the flag set is identical whenever every event is inside the lateness bound.
Expected result: Fixture 1 flags exactly once, at the t+9 event; the t+11 event does not flag under the half-open rule and does under the closed one, which is the boundary the fixture exists to pin. Fixture 2 yields the same single flag with the same window id. Fixture 3 yields no flag and late_dropped == 1. All twenty shuffles produce an identical flag set.
Verify it yourself
- Replaying the same events in any arrival order within the lateness bound produces identical flags — the only property that makes this job re-runnable.
- Per-key state never exceeds that key's events within 660 seconds; assert it rather than hope.
- Every flag's window id derives from event time alone;
arrival_timeappears only in the watermark computation and the lateness metric.
Follow-up
- Make the threshold and window configurable without rebuilding all in-flight state on every change. What does that constrain in the data structure?
- Two stream partitions hold events for the same token. What does that force the partitioning key to be?
- An event arrives three hours late. Does any answer change, and who is told?
Version a loan schedule instead of soft-deleting posted instalments
Medium · audit history · soft deletes · temporal queries
loan_instalment is keyed by (loan_id, schedule_version, instalment_no) and carries due_date, principal_minor, interest_minor, fee_minor, paid_principal_minor, paid_interest_minor, status, days_past_due, effective_from (date) and superseded_at (timestamptz). A borrower defers two payments on 2026-03-14, and instalments 1 to 6 already have allocations posted against them. Model exactly what the deferral writes, and write the query that returns the schedule as the borrower saw it on an arbitrary date. Say why stamping the old rows with deleted_at, or updating them in place, fails an audit.
Approach
- Treat the deferral as an insert, not an edit: write a complete new schedule_version with effective_from = the deferral instant on 2026-03-14, and in the same transaction stamp superseded_at on every row of the outgoing version with that identical instant, so the two version windows are half-open and adjacent rather than overlapping. Instalments 1 to 6 are reproduced unchanged, because they are what the borrower was told and what was posted to the ledger.
- Write the as-of read as a version selection, not a row filter: WHERE loan_id = $1 AND effective_from <= $2 AND (superseded_at IS NULL OR superseded_at > $2), then assert that exactly one schedule_version comes back - COUNT(DISTINCT schedule_version) = 1, not one row - so an overlapping window raises rather than silently returning two interleaved schedules under one instalment_no.
- Fix the type mismatch before writing that predicate: effective_from is declared date and superseded_at timestamptz, so comparing them casts the date at the session TimeZone and two readers in different zones select different versions near midnight. Put both columns in one domain, timestamptz, keep the window half-open with effective_from inclusive and superseded_at exclusive, and resolve a bare as-of date to an instant once, in the loan's booking timezone, at the edge of the system.
- Say what deleted_at loses. It records that a row stopped being current but not what replaced it or from when, it leaves every downstream query obliged to remember deleted_at IS NULL, and one query that forgets double-counts the schedule. Versioning puts the same information in the primary key where it cannot be forgotten.
- Close the arithmetic: under the loan's stated day-count convention the new version must still sum to outstanding principal plus scheduled interest to the minor unit, with the per-period rounding residual placed in one named instalment, conventionally the last, rather than smeared across the tail.
- Index (loan_id, effective_from DESC) for the as-of lookup, keep superseded versions online rather than archiving them, and enforce with a trigger that no UPDATE touches a row whose paid_principal_minor or paid_interest_minor is non-zero.
Follow-up
- What does days_past_due mean for an instalment that exists in two versions with different due_dates?
- A payment arrives allocated to an instalment_no that exists only in the superseded version. What do you do with it?
- How do you prove the ledger postings made under version 1 still reconcile once version 2 exists?
Store multi-currency amounts and prove each transaction balances
Easy · minor units · normalisation · constraints
ledger_entry holds entry_id, transaction_id, account_id, direction (debit or credit), amount_minor (bigint, CHECK > 0), currency char(3), source_type, source_id, business_date and posted_at. The service posts in JPY, USD and KWD. Specify the column types for money and the reference table that carries each currency's ISO 4217 minor-unit exponent, and say why that exponent must not be a constant in application code. Then write the query that returns every transaction_id whose entries fail to sum to zero within each currency, ordered by the largest absolute imbalance.
Approach
- State the representation: amount_minor is an integer count of minor units and the scale lives with the currency, not in the column. int4 tops out at 2,147,483,647 minor units, which is about 21.5 million units of a 2-exponent currency, so bigint, and the currency code travels on every row that carries an amount.
- Normalise the scale into currency(code char(3) primary key, exponent smallint, name) and join it only at display and parse boundaries. The exponent is 0 for JPY and KRW, 2 for USD and EUR, 3 for KWD, BHD, JOD, OMR and TND, so a hard-coded divide by 100 is wrong by 100x for JPY and by 10x for KWD, in opposite directions.
- Rule out binary floating point outright: IEEE 754 binary64 cannot represent 0.1, so repeated accrual drifts by a few minor units that later appear as reconciliation breaks. If a fixed-scale decimal is used instead, pin the scale per currency and name the single place rounding happens.
- Write the balance check as a grouped aggregate: sign the amount with a CASE on direction, GROUP BY transaction_id, currency, and HAVING the signed sum <> 0. The sign belongs in direction, and the CASE is the one place it becomes arithmetic.
- Then order in a second layer, because the ordering the question asks for cannot be written on the grouped query itself. PostgreSQL lets an output alias stand alone in ORDER BY but resolves anything inside an expression against the input columns, so ORDER BY abs(imbalance_minor) DESC raises SQLSTATE 42703, column "imbalance_minor" does not exist, while the bare ORDER BY imbalance_minor DESC is accepted and sorts signed rather than absolute. Wrap the grouped query and order by abs() outside it, or repeat the aggregate as ORDER BY abs(SUM(CASE ...)) DESC, which is legal because aggregates are allowed there.
- Group per (transaction_id, currency) rather than per transaction, because a cross-currency movement balances only within each currency, bridged by an explicit FX position account whose rate, source and timestamp are stored on the transaction.
- Note that no CHECK constraint can express this, since it spans rows: enforce it at write time with a DEFERRABLE INITIALLY DEFERRED constraint trigger that fires at commit, or by routing every posting through one function, and keep this query as the independent audit.
Worked solution (15 min)
- Define currency(code, exponent, name) and seed JPY 0, USD 2, KWD 3; declare amount_minor bigint NOT NULL CHECK (amount_minor > 0) and currency char(3) REFERENCES currency(code).
- Write it in two layers so it runs: SELECT transaction_id, currency, imbalance_minor FROM (SELECT transaction_id, currency, SUM(CASE WHEN direction = 'credit' THEN amount_minor ELSE -amount_minor END) AS imbalance_minor FROM ledger_entry GROUP BY transaction_id, currency HAVING SUM(CASE WHEN direction = 'credit' THEN amount_minor ELSE -amount_minor END) <> 0) b ORDER BY abs(imbalance_minor) DESC.
- Post one clean capture, one deliberately unbalanced transaction, and one cross-currency transfer with an FX position account, then run the query.
- Convert 1000 JPY and 1000 KWD through the exponent table in both directions and confirm the round trip is exact.
Expected result: Exactly one row: the deliberately unbalanced transaction, with a signed imbalance_minor equal to the amount omitted. The clean capture and both single-currency sides of the FX transfer return nothing.
Verify it yourself
- The cross-currency transaction produces two groups, one per currency, and neither appears in the output.
- Flipping one entry's direction makes imbalance_minor exactly twice that entry's amount, which confirms the sign convention.
- Inserting an entry with amount_minor = 0 or a negative value is rejected by the CHECK rather than by application code.
- Execute the statement rather than reading it: ordering the grouped query directly by abs(imbalance_minor) fails with 42703, which is why the abs() sits outside the subquery.
Follow-up
- A transaction has a USD leg and a JPY leg. Which accounts appear, and where does the rounding residual land?
- How would you enforce the zero-sum rule at write time without serialising all postings behind one lock?
- Someone proposes storing a display amount alongside amount_minor. What goes wrong at the first rate change or rounding rule change?
What is your decision-making framework when choosing between different frameworks or libraries for a new project?
Medium
Approach
- State your assumptions explicitly before working the problem.
- Clarify what is being asked and what a complete answer contains.
- Work from the requirement backwards to the design.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
How do you approach dependency management in large-scale applications?
Medium
Approach
- Clarify what is being asked and what a complete answer contains.
- Work from the requirement backwards to the design.
- Say what you would check first and why it is the highest-information step.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Raise the write ceiling on one hot clearing account
Hard · lock contention · isolation levels · sharding · write-heavy
The ledger posts balanced entry sets at 10,000 entries/s into append-only ledger_entry(entry_id, transaction_id, account_id, direction, amount_minor>0, currency, source_type, source_id, business_date, posted_at), roughly three entries per transaction. A materialised row per account carries the floor check available_minor >= floor_minor and is updated in the posting transaction. One pooled clearing account appears in 40 percent of transactions. Compute the throughput ceiling on that one row, then raise it. Name the isolation level each version requires and state precisely what sharding the balance costs the floor check.
Approach
- Compute the ceiling before designing anything. Throughput against a single row is one divided by the lock hold time, where the hold runs from first acquiring the row to commit, including the WAL flush. At a 2 ms hold that is about 500 commits/s on that row, and adding cores does not move it. Demand is 10,000 entries/s at three entries per transaction, so roughly 3,300 transactions/s, of which 40 percent touch this account: about 1,300/s against a 500/s ceiling. You are 2.6x over before any code exists.
- Shorten the hold before sharding, because it is free. Acquire the hot row last in the transaction and hold it for exactly one statement: UPDATE balance SET available_minor = available_minor - $1 WHERE account_id = $2 AND available_minor - $1 >= floor_minor. This is correct at READ COMMITTED, because an UPDATE that blocks on a concurrent writer re-evaluates its WHERE clause against the newly committed row version rather than its original snapshot.
- Name what the broken version needs. SELECT the balance, compute in application code, then UPDATE loses updates at READ COMMITTED, since the second statement takes a fresh snapshot. Making that shape safe requires REPEATABLE READ or SERIALIZABLE plus a bounded retry loop on SQLSTATE 40001, which is the part most implementations omit; SELECT ... FOR UPDATE also works but is simply the same lock held for longer. Note that the identical application code changes behaviour on a different engine: InnoDB's REPEATABLE READ does not abort the loser.
- Order locks deterministically on any two-account movement, always ascending by account_id. Concurrent opposing transfers without that ordering deadlock with SQLSTATE 40P01, and the deadlock rate rises with exactly the concurrency you added to gain throughput.
- Shard the hot balance into N sub-rows keyed (account_id, shard_no), writers choosing a shard by hash of transaction_id. Per-shard hold time is unchanged, so capacity rises roughly N-fold, and at N=16 you clear 1,300/s with headroom. The cost is precise: the floor is now a predicate over the sum of N rows and is no longer expressible as a single-row CHECK, so it must be paid for deliberately. Either give each shard its own floor, which rejects postings while other shards hold headroom, or add a slow path that locks all shards in ascending shard_no under SERIALIZABLE, rebalances and retries on 40001. For a clearing account whose floor is effectively unreachable, dropping the per-shard floor and enforcing it as a monitored invariant is a defensible stated choice; for a customer deposit account it is not.
- Keep the entries immutable throughout: no UPDATE or DELETE grant on ledger_entry plus a trigger, corrections as reversing transactions, and partitioning by business_date so index writes stay in the current partition and retention is a DETACH.
Worked solution (45 min)
- Run two concurrent sessions doing SELECT then UPDATE to a computed value against the same account under READ COMMITTED and observe one update lost.
- Replace it with the single atomic conditional UPDATE and re-run; then run the original read-modify-write under REPEATABLE READ with a bounded retry on 40001.
- Measure committed transactions per second against the single pooled row at 1, 8, 32 and 128 concurrent writers, recording mean lock hold time.
- Shard into 16 sub-rows hashed on transaction_id, re-measure, and write the query that answers the true available balance.
- Run concurrent opposing two-account transfers with and without ascending account_id lock ordering.
Expected result: The read-modify-write loses an update at READ COMMITTED and aborts with 40001 at REPEATABLE READ; the single atomic statement is correct at READ COMMITTED with no retry loop. Single-row throughput flattens near one divided by the lock hold time, about 500 commits/s at a 2 ms hold, against the roughly 1,300 transactions/s this account needs. Sixteen shards clear that, and the floor check becomes a predicate over sixteen rows. Unordered transfers produce 40P01 deadlocks; ascending order produces none.
Verify it yourself
- Past the plateau, adding writers raises latency without raising throughput; that is the ceiling, not a tuning problem.
- After sharding, a floor breach is reproducible when each shard enforces the floor independently; construct it deliberately rather than assuming it cannot happen.
- ledger_entry has no UPDATE or DELETE path throughout, and every correction in the test appears as a new reversing transaction.
Follow-up
- While the balance is sharded, what query answers the account's true available balance, and what does it cost?
- A transaction has legs in two currencies. What does the balanced-to-zero check mean now, and where does the rounding residual go?
- What does the nightly reconciliation compare the sharded balance against?
Approval rate collapses when one feature-store shard fails
Medium · cache stampede · partial outage · bulkhead · retry amplification
The risk decision service answers about 3,000 authorisations per second within an 80 ms p99, reading features from an eight-shard store behind a local cache with a fixed 60-second TTL, a 2-second client timeout and two retries on timeout. One shard becomes unreachable at 14:02. Approval rate falls from 96% to 71% within 90 seconds, and the seven healthy shards show p99 rising to 900 ms with normal error rates. Diagnose in order, name the cause of the approval drop, and state the degraded behaviour this service should have had.
Approach
- Establish what declined actually means here before touching the cache. Split the rate into approve, decline, refer and timeout. If decline counts are flat and timeouts account for the whole 25-point drop, the model is not declining anything; it is failing to answer, and something downstream is reading silence as a decline. That reading is the first finding and the most expensive one.
- Break the store's metrics out per shard. One shard at 100% errors and seven at elevated latency with baseline error rates is the signature of queueing, not of seven shards failing. Healthy dependencies with healthy error rates and bad latency are almost always victims of a shared resource.
- Follow the resource. A single connection or thread pool shared across shards means requests to the dead shard occupy slots for the full 2-second client timeout, which is 25 times the whole p99 budget, and with two retries each such request can hold a slot for 6 seconds. Confirm by comparing pool acquisition wait against per-shard service time: if wait dominates, the pool is the bottleneck.
- Quantify the amplification. Two retries on timeout offer 3x load, and an unjittered 60-second TTL expires a warm set together, so misses arrive as a synchronised wave rather than spread across the window. Retrying a timeout is only correct when the failure is likely transient; against a hard-down shard it is pure amplification and needs a circuit breaker to gate it.
- Fix the three separate defects separately: per-shard pools so one shard cannot starve seven, a breaker that stops retrying a shard that is failing, and per-key TTL jitter with single-flight refresh so N concurrent misses for one key produce one upstream read.
- Write down the degraded decision as a product decision, not a timeout handler's accident. With features unavailable the service must return a stated answer, such as deterministic rules alone below an amount threshold and refer above it, and must record that the decision was degraded so it is auditable and reproducible from stored inputs.
Follow-up
- Approving on deterministic rules alone during a feature outage carries a fraud cost. How do you size the amount threshold, and who signs it off?
- What should the orchestrator do with a risk timeout, given it cannot distinguish one from a decline?
- Does the circuit breaker fix the healthy shards' latency, or is the bulkhead doing that work? Which would you ship first if you could only ship one?
Suggested practice checklist
Roughly ninety minutes on weeknights with one longer weekend block. The plan cuts scope rather than compressing everything, on the assumption that one thing finished per night beats four half-started.
Day 1 — Fix the scope and take a cold baseline
- Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list and keep it visible all week.
- Take one twenty-five-minute coding problem and one fifteen-minute design prompt cold, and write the single sentence naming what blocked each, because those two sentences decide where the remaining evenings go.
- Set the week's rule: one thing finished every night, including the night you only have forty minutes.
Deliverable: A one-page scope with a not-doing list and two cold attempts, each carrying one sentence on what blocked it.
Day 2 — One pattern, written three times from blank
- Choose the single pattern most likely to appear in your loop and write it three times from an empty file rather than editing the previous attempt.
- On the third pass, write the invariant as a comment before the loop body and the complexity before the first line of code.
- Stop at ninety minutes even if the third version is imperfect, and write the one thing you would fix given another hour.
Deliverable: Three independent implementations of the same pattern plus a note on what changed between them.
Day 3 — One design, only to the depth you can defend
- Take one system shape and go only as far as requirements, interface and data model, refusing to draw a box you could not survive a follow-up about.
- Attach one number to each non-functional requirement, deriving it rather than asserting it, and write the assumption the number rests on.
- Write the one tradeoff you are choosing against and the observation that would make you reverse it.
Deliverable: One design at interface-and-schema depth with derived numbers and one written reversible tradeoff.
Day 4 — Only the fundamentals you will have to defend
- Write, in under two hundred words each, the answers to the two questions that follow almost any implementation: why this structure and not the obvious alternative, and what happens to this code at a hundred times the input.
- Write what an index actually costs: faster lookups on the indexed columns against a write that now maintains a second structure, plus the cases where the planner declines to use it anyway, low selectivity, or a predicate wrapping the column in a function.
- Delete any answer you cannot deliver aloud in under a minute, since an answer that needs reading is not an answer you have.
Deliverable: Three written answers, each under two hundred words and each timed aloud.
Day 5 — Your own work, timed
- Write a ninety-second and a four-minute version of your main project and time both aloud rather than reading them.
- Prepare the two follow-ups that always come: what you would do differently, and how you knew it worked.
- Put one number in the first sentence and be ready to say exactly where it came from and what it excludes.
Deliverable: Two timed narratives with one defensible number in the opening line.
Day 6 — The one full rehearsal, in the weekend block
- Run a sixty-minute mock covering a coding round and a design round in one sitting with no break, because sustained attention is the thing evenings have not trained.
- Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
- Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.
Deliverable: Mock notes naming three failure moments with a specific fix written under each.
Day 7 — Taper
- Write the twenty-minute warm-up you will actually do on the morning: one problem you can already solve from a blank file, one design you can narrate, and nothing you have never seen.
- Re-read only your own notes from this week and open no new material.
- Write the logistics down: the editor or shared document you will be working in, whether execution and lookups are permitted, and the sentence you will use when you do not know something.
Deliverable: A one-page card holding the design structure, the project numbers, and the logistics.
The estimate and the day you knew
A slipped date is only a bad story if you sat on it. What matters is what you believed when you gave the number, the signal that told you it was wrong, how many days passed before you said so, and what you cut rather than asking for more time. Scope you defended counts as much as scope you dropped.
Can you explain your experience with CI/CD pipelines and how you ensure deployment reliability?
Medium · behavioural and engineering judgement
Approach
- Close with what you would do differently, concretely.
- Pick a story where you made the decision, not one where you watched it.
- State the situation in two sentences and spend the rest on the reasoning.
Follow-up
- What did you decide not to do, and why?
- How did you know your change caused the improvement?
Tell me about a significant professional accomplishment you are proud of.
Medium · behavioural and engineering judgement
Approach
- State the situation in two sentences and spend the rest on the reasoning.
- Name the disagreement and how you resolved it with evidence.
- Close with what you would do differently, concretely.
Follow-up
- What would you do differently if you ran that again?
- How did you know your change caused the improvement?
How do you handle technical debt while maintaining feature delivery velocity?
Medium · behavioural and engineering judgement
Approach
- State the situation in two sentences and spend the rest on the reasoning.
- Give the blast radius: what could have broken, and what you measured.
- Name the disagreement and how you resolved it with evidence.
Follow-up
- What would you do differently if you ran that again?
- What did you decide not to do, and why?