cswork / libraryOpen practice workspace

Interview guides · AEG Power Solutions

AEG Power Solutions Software Engineer Interview Questions & Guide 2026

AEG Power Solutions software engineer prep: algorithm drills where every complexity claim has to name the assumption under it.

The 60-second brief

As a Software Engineer at AEG Power Solutions, you sit at the intersection of high-stakes industrial engineering and modern software architecture. You are not just writing code; you are building the digital intelligence that powers critical infrastructure, from nuclear power plants to complex energy management systems. Your work ensures that electrical equipment performs reliably, safely, and efficiently in environments where downtime is not an option.

Find out what you will be typing into. A shared plain-text editor with no autocomplete, compiler or test runner changes what you have to hold in your head, and practising inside your own configured environment hides exactly that gap.

Focus areas

  • Diagnose failures inside environments you cannot instrument
  • Make integration retries idempotent without target-side idempotency keys
  • Evolve schemas expand-contract across un-upgradable client deployments

Overview

This role requires a unique blend of technical precision and domain-specific problem-solving. You will contribute to projects that directly impact the stability of global power grids and industrial operations. Whether you are working on SAP integrations, field application engineering, or specialized technical leadership, your contributions will be central to how AEG Power Solutions delivers value to its clients. It is a position for those who thrive on technical complexity and want to see their software drive physical, high-impact results.

The AEG Power Solutions interview process

AEG Power Solutions candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.

What to demonstrate

  • Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
  • Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
  • Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not

How to prepare

  • Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
  • Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
  • If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window

Focused Interview

An unlabelled round is first an information problem, and the cheapest information is free. Whoever schedules it can usually tell you how long it runs, who will be in the room and what they work on, whether you will be writing code and in what environment, and whether anything is being sent beforehand. Ask in writing so the answer is on record, then prepare for the two or three formats those answers still leave open instead of betting on one. What separates a strong candidate is not guessing right; it is having an opening that works whichever one it turns out to be.

What to demonstrate

  • Whether you can start work from an ambiguous brief, since tolerating a vague scope without stalling is the same thing the job asks for
  • Whether the questions you asked beforehand were ones that change your preparation, such as duration, medium and who is joining, rather than ones whose answers you could not have acted on
  • Whether you adapt when the round turns out to be something other than what you were told, instead of spending the first ten minutes visibly recalibrating

How to prepare

  • Send one short scheduling message asking four things: how long, who is joining and what they work on, whether you will be writing code and where, and whether to prepare anything in advance. Treat a vague reply as real information, since it means the round is loosely structured and you will be shaping it yourself.
  • Write one opening that works in any of the formats still open: restate in your own words what you have been asked to do, then ask which of two directions is more useful to them. Say it aloud until it stops sounding recited.
  • Set up for the two most likely formats before the call starts, with a blank editor in the language you would choose and a shared document you can type into, so a format surprise costs you nothing in the first minutes

Technical Evaluation

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

  • In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
  • In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
  • Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

  • Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
  • Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
  • For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Where candidates lose points

Treating an ambiguous failure as a definite one

A timeout, a 502 from an intermediary, or a connection reset after the request bytes were sent all leave the target's state unknown. Classifying those as failures and retrying duplicates the effect; classifying them as successes and advancing the watermark loses data silently. Both wrong answers are common because the ambiguous case is rare in a sandbox and routine in production. The run needs a distinct ambiguous state, a dedupe key that makes the retry safe, and a reconciliation read against the target when the key alone cannot settle it.

Validating against a client sandbox and assuming production parity

Sandboxes typically carry smaller data, looser or absent rate limits, a schema version behind production, and sometimes synchronous behaviour where production is asynchronous. Code that passes there fails first in the client's production, during a change window you do not control and often cannot get a second one of. The mitigations are specific: assert the observed schema version at the boundary of every run, measure the production rate limit empirically rather than reading it from a document, and design the first production run to be a bounded, reversible slice rather than a full backfill.

Assuming fixed-width integer arithmetic cannot overflow

In languages with fixed-width integers, including C, C++, Java, Go and Rust, computing a midpoint as (lo + hi) / 2 overflows once the sum passes the type's maximum, so write lo + (hi - lo) / 2 instead. Say which language you are in: arbitrary-precision integers, as in Python or Ruby, remove this specific hazard and none of the others.

Hardcoding to the sample inputs

Solve the stated problem rather than the two examples; special-casing a literal to make a sample pass is obvious immediately and reads as either a misunderstanding or an attempt to fake progress. If you genuinely cannot generalise yet, say which part is a stub and what would replace it.

Questions & practice

Assign connector bindings to rollout waves and name the cycle

Medium · topological sort · cycle detection · dependency graph

Connector bindings declare ordering dependencies: an edge u -> v means v must not run in a wave earlier than one after u has succeeded. You have up to 50,000 bindings and 200,000 edges, and a misconfigured dependency may have created a cycle. Assign each binding the earliest wave number such that every predecessor sits in a strictly earlier wave, return the total wave count, and if a cycle exists return the bindings on one concrete cycle rather than a boolean. The implementation must be iterative: a dependency chain can be 50,000 deep.

Approach

  1. Build an adjacency list and an indegree array, then run Kahn's algorithm from every indegree-zero node. Bindings with no predecessors get wave 0. O(V+E) time, O(V+E) space.
  2. Relax waves as you decrement: when popping u, set wave[v] = max(wave[v], wave[u]+1) for each successor before enqueueing v at indegree zero. Because v is only enqueued after its last predecessor is popped, wave[v] is 1 + max over predecessors, which is the earliest legal wave. Total wave count is max(wave)+1, the longest path in nodes.
  3. If Kahn emits fewer than V nodes, the residual subgraph is exactly the nodes lying on or downstream of a cycle. Every residual node still has an in-edge inside the residual, otherwise it would have been popped, so walking backwards along in-edges cannot dead-end and must revisit a node within |residual| steps; the segment between the two visits is a concrete cycle. O(V) with a visited-position map.
  4. Use an explicit stack or a queue rather than recursion. Recursive DFS on a 50,000-node chain exceeds the default stack in most runtimes, and the failure looks like a crash rather than a cycle.
  5. At this size, recompute the whole plan whenever an edge changes. Incremental topological order maintenance costs far more code than the O(V+E) rebuild it saves.

Follow-up

  • Two bindings land in the same wave but hit the same client endpoint, whose rate budget is shared. How does the wave plan have to change, and what problem class does that turn into?
  • One dependency holds only for overlapping watermark ranges rather than globally. Is the dependency graph still a DAG over bindings, and what is the right vertex if not?
  • Report the critical path so an operator can see which chain sets the wave count, not just that it is five.

Find peak window utilisation and the first rejected request

Medium · sliding window · two pointers · rate limiting

One connector binding's requests against a client endpoint arrive as timestamps t[0..n-1] in milliseconds, sorted non-decreasing, n up to 1,000,000. The endpoint's budget is R requests per rolling window of W milliseconds, and a request at time t is admitted only if strictly fewer than R already-admitted requests fall in the half-open window (t-W, t]. Return (a) the maximum number of requests falling in any W-millisecond window, counting every request regardless of admission, and (b) the index of the first request the limiter would reject. Both parts in O(n) time.

Approach

  1. Part (a) is a two-pointer scan: for each right index, advance left while t[right] - t[left] >= W, which leaves the window exactly (t[right]-W, t[right]], and track max(right-left+1). Left is monotone non-decreasing, so total work is O(n) amortised with O(1) extra space. The maximum over all windows of width W is attained at a window whose right edge is a request timestamp, so scanning right edges is sufficient.
  2. Part (b) is not the same scan. A rejected request consumes no budget, so the window must count admitted requests only. Keep a FIFO deque of admitted timestamps: pop from the front while front <= t - W, admit if the deque holds fewer than R, and return the first index where it does not. O(n) total pushes and pops, O(R) memory.
  3. Pin the boundary convention before writing either loop. With a half-open (t-W, t] window a request exactly W milliseconds after an earlier one does not see it; switching to a closed window changes the answer at every boundary and is where most disagreements with the client's own limiter come from.
  4. Note what fixed buckets of width W buy and cost: O(1) memory, but they admit up to 2R inside a span shorter than W, R at the end of one bucket and R at the start of the next. If buckets are unavoidable, halve the bucket width or use a weighted estimate across two adjacent buckets.
  5. If part (a) already exceeds R, the binding cannot run at this concurrency under any limiter shape. That is a scheduling problem, not a retry-policy problem, and no backoff tuning fixes it.

Follow-up

  • Three bindings and four workers share this one endpoint's quota. Where does the counter live, and what is the correct behaviour when that store is unreachable?
  • Add full-jitter backoff to the retry path and state precisely what it prevents that plain exponential backoff does not.
  • The client documents the limit as 'about 100 per minute'. How would you measure the real one without tripping it in their production?

Resolve effective-dated engagement versions for two million access checks

Hard · binary search · effective dating · complexity budget

The engagement table holds 500,000 effective-dated rows: (engagement_id, version, effective_from, effective_to which is NULL for the open version). A database exclusion constraint guarantees that per engagement_id the tstzrange(effective_from, effective_to) values never overlap, and the ranges are half-open. You must answer 2,000,000 access checks of the form (engagement_id, at_time) in one process with no database, returning the version open at that instant or none. Also report, per engagement, any coverage gap between adjacent versions. Give the complexity of the scan-per-check approach and say why it is not an option.

Approach

  1. State the naive cost first: scanning the table per check is 2e6 x 5e5 = 1e12 range comparisons. Even at 1e9 simple comparisons per second that is roughly fifteen minutes of pure compare work, against a budget measured in seconds. It is correct, just unaffordable, and the fix is an index built once rather than a faster comparison.
  2. Build a hash map engagement_id -> array of versions sorted by effective_from, once. O(V log V) to sort, O(V) memory. The build is amortised over 2e6 lookups, which is what makes it worth doing at all.
  3. Answer each check by binary-searching for the last effective_from <= at_time, then confirming effective_to IS NULL OR at_time < effective_to. If the confirmation fails, the instant lies in a gap and the answer is none, not the neighbouring version. O(log v) per check, O(V log V + Q log v) total.
  4. Name the precondition: predecessor search is only valid because the ranges per engagement do not overlap, which the exclusion constraint enforces with btree_gist for the equality operator. Without that guarantee you need an interval tree returning all stabbed intervals, at O(log n + k). Use the same half-open convention as the database, where at_time equal to effective_to belongs to the next version.
  5. Find gaps with one O(V) pass over each sorted array comparing prev.effective_to against next.effective_from. The exclusion constraint forbids overlaps, not holes, so a hole is entirely possible and is where an access check fails closed for a reason nobody can see in the contract.
  6. If the checks can be buffered, sort them by (engagement_id, at_time) and merge against the version arrays: O(Q log Q + V log V), usually a large constant-factor win over 2e6 random binary searches because the merge walks memory in order.

Worked solution (35 min)

  1. Index engagement E1 with v1 [2026-01-01, 2026-04-01), v2 [2026-04-01, 2026-07-01), v3 [2026-07-01, NULL), and engagement E2 with v1 [2026-01-01, 2026-03-01) and v2 [2026-04-01, NULL).
  2. Query (E1, 2025-12-31): predecessor search finds no effective_from <= at_time, so the answer is none.
  3. Query (E1, 2026-04-01T00:00:00Z): the predecessor is v2, not v1, because the boundary instant belongs to the later range under half-open semantics. Query (E1, 2026-06-30T23:59:59Z) returns v2 and (E1, 2026-09-01) returns v3.
  4. Query (E2, 2026-03-15): the predecessor is v1, but at_time is not less than v1.effective_to (2026-03-01), so the confirmation fails and the answer is none.
  5. Adjacent-pair pass on E2: v1.effective_to = 2026-03-01 is earlier than v2.effective_from = 2026-04-01, so report the gap [2026-03-01, 2026-04-01).

Expected result: E1: none, v2, v2, v3 for the four queries in order. E2: none at 2026-03-15, with a reported coverage gap of [2026-03-01, 2026-04-01). The scan-per-check approach is O(Q*V) = 1e12 comparisons; the indexed approach is O(V log V + Q log v).

Verify it yourself

  • An at_time exactly equal to a version's effective_to resolves to the later version, matching PostgreSQL's default [) range bound.
  • Every engagement has at most one version with effective_to IS NULL, which the unique partial index guarantees and the index build can assert.
  • The number of answers equals the number of checks; a 'none' is an answer, not a dropped row.
  • Brute-force scanning this five-row fixture per query reproduces every answer, including both 'none' cases.

Follow-up

  • The open version has effective_to NULL. How do you keep both the sort and the comparison total without a sentinel that overflows your timestamp type?
  • A gap exists for one engagement and a check lands in it. What should the access path return, and what should it emit so someone fixes the amendment?
  • Amendments are being written while you answer checks. What read model keeps an answer self-consistent, and what does a check mean if the snapshot is a second stale?

Generate invoice lines once under concurrent month-end runs

Hard · isolation levels · unique constraints · idempotency

Month-end generation reads approved work_record rows for one engagement and period, sums minutes, inserts one invoice_line whose generation_key is UNIQUE over (engagement_id, period_start, period_end, line_type, generator_version), then sets those work records to invoiced with locked_at and invoice_line_id. Two runs execute concurrently for the same engagement while late approvals are still committing. Working in PostgreSQL, name the anomaly at READ COMMITTED and at REPEATABLE READ, and specify the controls that make a retry a no-op rather than a second charge.

Approach

  1. At READ COMMITTED each statement takes a fresh snapshot, so the SELECT that computed the sum and the later UPDATE see different data: an approval committed between them is a phantom the sum missed but the UPDATE can still mark invoiced. Worse, an UPDATE that blocks on a row another transaction is changing re-evaluates its WHERE against the new row version once the lock releases, so WHERE status = 'approved' silently skips rows the other run already moved — fewer rows than you counted, and no error anywhere.
  2. At REPEATABLE READ, which in PostgreSQL is snapshot isolation, the sum and the update agree because the whole transaction shares one snapshot, but a concurrent update to the same row aborts you with serialization_failure, SQLSTATE 40001, which the application must catch and retry. Snapshot isolation still permits write skew across different rows; only SERIALIZABLE with SSI excludes it, at the cost of more 40001s and predicate-lock memory bounded by max_pred_locks_per_transaction.
  3. Make the second run collide instead of race: take a transaction-scoped advisory lock on a hash of (engagement_id, period_start, line_type) at the top of the run, so one generates while the other waits and then finds finished work. Keep the UNIQUE generation_key as the backstop, because the advisory lock is per-database-connection state that a failover or a second database node does not carry.
  4. Treat the unique violation as success. Catch SQLSTATE 23505, re-select the existing invoice_line by generation_key, and return it. INSERT ... ON CONFLICT DO NOTHING ... RETURNING returns zero rows on conflict, so a handler that trusts RETURNING writes a NULL invoice_line_id and reports a failure for work that in fact completed; ON CONFLICT DO UPDATE is worse, because it overwrites a line that has already been issued.
  5. Bound the read deterministically rather than by timing: select the work records FOR UPDATE ordered by work_record_id so both runs acquire row locks in the same order and cannot deadlock, and filter on approved_at < the run's start timestamp so a late approval is out of scope by definition and lands in the next period.
  6. Enforce immutability in the database, not in the service. A BEFORE UPDATE trigger raising when locked_at IS NOT NULL and a billed column changes, plus CHECK (status <> 'invoiced' OR invoice_line_id IS NOT NULL), means the constant and legitimate business pressure to edit an invoiced entry resolves into an appended credit_note row instead of a quiet update.

Follow-up

  • A line has been issued and then an approval behind it is reversed. What rows do you write, and what does the original line look like afterwards?
  • How many times does the run retry before giving up, and what state does it leave behind for the operator who picks it up?
  • After the fact, how do you prove no engagement-period was billed twice, without trusting the application code that wrote it?

Explain why the failed-run backlog query ignores three indexes

Medium · indexing · query planning · partial indexes

connector_run holds roughly six million rows over three years. Its indexes are (connector_id, started_at DESC); (started_at) WHERE status = 'failed'; and ((finished_at AT TIME ZONE 'UTC')::date). The operator backlog query is SELECT run_id, connector_id, failure_class, started_at FROM connector_run WHERE environment_id = $1 AND status = ANY($2) AND started_at >= now() - interval '7 days' ORDER BY started_at DESC LIMIT 50, with $2 bound to {failed, ambiguous}. It sequentially scans. Explain index by index why each is unusable, then give the one index you would add.

Approach

  1. Index one leads on connector_id, which this query never constrains. A B-tree can only be probed from a prefix of its key columns, so with no equality on connector_id it degenerates into a full index scan plus heap fetches, which the planner will rightly cost above a sequential scan. PostgreSQL 18 added B-tree skip scan for an unconstrained prefix, so state your server version before assuming this is fatal; it helps only when that prefix has few distinct values, and hundreds of connectors is already too many to rely on it.
  2. Index two is partial on status = 'failed', and the rows the query wants are partly not in it at all. Predicate implication runs one way: the query's predicate must imply the index's. status IN ('failed','ambiguous') is weaker than status = 'failed' and implies nothing, so no amount of inlining the parameter helps. It would serve only the failed branch of a UNION ALL.
  3. Index three is an expression index on a function of finished_at, while the query filters started_at as a plain range. Even on the right column, a range predicate on the bare column cannot use an index on a function of it. Note also why the AT TIME ZONE 'UTC' wrapper is present: finished_at::date on a timestamptz is STABLE rather than IMMUTABLE because it depends on the session TimeZone, so PostgreSQL refuses to index it without a fixed zone.
  4. Add (environment_id, started_at DESC) WHERE status IN ('failed','ambiguous'): equality on the scoping column first so the scan starts in the right place, the ordering column second so ORDER BY started_at DESC LIMIT 50 stops after fifty index entries instead of sorting the week, and partial on the status set so the index covers the small failing minority rather than six million rows.
  5. Confirm the partial predicate is provable for the query's parameterised form. Under a custom plan the array parameter is substituted as a constant and the prover handles the ScalarArrayOpExpr; under a generic plan it stays a Param and the proof fails, so check plan_cache_mode or the sixth execution of a prepared statement before declaring victory.
  6. Say what you deliberately did not do. Adding INCLUDE (run_id, connector_id, failure_class) to chase an index-only scan pays only while the visibility map is current, which ties the query's latency to whether autovacuum is keeping up with the write rate — a dependency worth accepting consciously, not by accident.

Worked solution (30 min)

  1. Generate a few million rows with a skewed environment_id and status distribution, then ANALYZE.
  2. Run EXPLAIN (ANALYZE, BUFFERS) on the backlog query and record the plan, the rows read, and the time.
  3. Create the proposed partial index, ANALYZE again, and re-run the same EXPLAIN.
  4. Rewrite $2 as inline literals and compare the plan against the parameterised prepared statement executed at least six times.

Expected result: Three specific reasons — missing leading equality, a partial predicate the query does not imply, and an expression index on the wrong column — plus CREATE INDEX CONCURRENTLY ON connector_run (environment_id, started_at DESC) WHERE status IN ('failed','ambiguous'), and a plan that reads on the order of 50 index entries instead of the table.

Verify it yourself

  • After the new index, EXPLAIN ANALYZE reports rows close to 50 at the index scan and no Sort node.
  • Remove LIMIT 50 and watch the plan change: the ordered index wins partly because of the limit, which is worth being able to say out loud.
  • Attempt CREATE INDEX ON connector_run ((finished_at)::date) and read the 'functions in index expression must be marked IMMUTABLE' error for yourself.

Follow-up

  • The same query is also run without the environment filter, for an estate-wide backlog. Does your index still help, and what would you add instead?
  • How does row-level security on this table interact with your index, given that policy quals are applied before non-leakproof user quals?
  • Six months from now the partial index covers 30 percent of the table. What signal tells you to change it?

How would you handle a time-sensitive customer case involving critical electrical equipment?

Medium

Approach

  1. Clarify what is being asked and what a complete answer contains.
  2. State your assumptions explicitly before working the problem.
  3. Work from the requirement backwards to the design.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

What is your approach to troubleshooting complex bugs in a production industrial environment?

Medium

Approach

  1. Clarify what is being asked and what a complete answer contains.
  2. Work from the requirement backwards to the design.
  3. Say what you would check first and why it is the highest-information step.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Apply records to a client system exactly once under retries

Medium · idempotency · exactly-once · ambiguous failures · transactions

A connector run claims a watermark range, reads records from a client endpoint and writes them to a client-owned target that offers no idempotency key of its own. connector_run stores (connector_id, idempotency_key, attempt_no, status, failure_class, watermark_from, watermark_to, records_read, records_applied), and status includes 'ambiguous'. A write can time out after the request bytes were sent, leaving the target's state unknown. Design the apply loop so that any number of retries leaves the target exactly as one clean run would, and say precisely what advances the watermark and when.

Approach

  1. Derive the dedupe key deterministically from stable source fields - the source's own identifier plus a hash over exactly the fields you write - and store the input field list and a derivation version beside the hash. The client will eventually rename or retype one of those fields, and every key computed before that day must stay reproducible; without the version, the first schema change re-applies the entire history.
  2. Order the local writes so a crash is recoverable: insert the ledger row (connector_id, dedupe_key, unique) as a claim, write to the target, then mark it applied and advance the watermark in one local transaction. The target write cannot join that transaction - no client grants you a distributed one - so the claim row exists precisely to tell the retry that a write may have happened.
  3. Give ambiguity its own state. A timeout, a 502 from an intermediary and a reset connection after the bytes went out all leave the outcome unknown, and collapsing them into 'failed' duplicates on retry while collapsing them into 'succeeded' loses data silently. On retrying an ambiguous run, resolve per record: read back by natural key if the target has one, and if it does not, negotiate one writable reference field at integration time so the read-back is possible at all. That negotiation is cheaper than every reconciliation you would otherwise run forever.
  4. Advance the watermark only across a range whose ledger rows are all applied or all deduped, and advance it with deliberate overlap. Many source systems stamp updated_at at transaction start, so a record committed after your read can carry a timestamp inside a range you already consumed; re-claim a bounded lag window each time and let the dedupe key absorb the duplicates, which is what it is for.
  5. Bound the ledger's growth against that same window. Retention must exceed the maximum overlap plus the longest plausible late arrival, because pruning earlier converts a late record into a silent re-apply. Partition the ledger by watermark date so pruning is a partition drop rather than a mass delete.

Worked solution (30 min)

  1. Build the stand-in target the way the prompt describes the real one: an insert-only table with no unique index, no primary key over the dedupe key and no upsert path, plus one writable reference column holding the dedupe key, indexed but not unique. A duplicate write must succeed and be visible, or the harness dedupes for you and the run proves nothing.
  2. Implement claim, fetch, validate, write, ledger, advance, with the ambiguous-retry path resolving each record by selecting the reference column for the range's keys before writing, and writing only the misses. One run owns a watermark range at a time - read-then-write across two concurrent runs over the same keys is a race no constraint-free target can arbitrate.
  3. Kill the process between each adjacent pair of steps and, for all five crash points, run the retry and count rows in the target.
  4. Make the target accept a write but drop the response, then retry and count again.
  5. Remove the read-back from the ambiguous path and repeat the dropped-response case, to measure what the reconciliation is actually buying.
  6. Rename one source field, bump the derivation version, and re-run a range already processed under the old version.

Expected result: Every crash point ends with the same target row count as one clean run: the retry finds its ledger row either unapplied, in which case it reconciles the range against the reference column and writes only what is missing, or applied, in which case it skips. The dropped-response case yields one row because the read-back finds the record the first attempt wrote; with the read-back removed the identical run yields two, and that pair of numbers is the whole argument, since the target rejects nothing either way. The version bump re-applies nothing, because keys for already-processed rows are still computed under the old derivation.

Verify it yourself

  • The stand-in target carries no unique index, no primary key over the reference column and no upsert path, asserted against its schema, so no row can be deduped by the target rather than by the apply loop.
  • records_applied plus the dedupe hit count equals records_read on every run.
  • No watermark has advanced past a range containing a ledger row that is claimed but not applied.
  • Replaying an entire previous day's range end to end changes zero rows in the target, and the run log attributes each skip to a read-back hit rather than to a rejected write.

Follow-up

  • The target exposes no natural key and no field you are allowed to write a reference into. What is your next move, and what does it cost per run?
  • A succeeded run reports records_read 10,000 and records_applied 9,997. Is that a bug? What single query settles it?
  • The client restores their source database to a point two hours earlier. What does your next run do, and what should it do?

Entitlement snapshot expires in lockstep across the fleet

Medium · cache stampede · ttl jitter · single-flight

Delivery requests across the estate return 403 for roughly 200 ms every five seconds. The entitlement service publishes one versioned snapshot; every caller caches it for five seconds and fails closed on a miss. Its request rate is near zero for most of each window and about 4,000 in a 300 ms burst, spread across the 800 application processes a rolling deploy restarted this morning. Its own handler service time is unchanged. Diagnose the shape, then give a fix that preserves fail-closed behaviour.

Approach

  1. Establish the period before the cause. Plot the origin's request rate at one-second resolution: a sawtooth whose period equals the cache TTL, with the mass inside a few hundred milliseconds, says the misses are synchronised, not that traffic grew.
  2. Measure two things next, in this order: key cardinality and in-flight concurrency per process. One key across the whole fleet means every process misses at the same instant. Four thousand requests from 800 processes means roughly five concurrent misses inside each process, so there is no single-flight and each in-flight request for the same key issues its own fetch.
  3. Explain why the phases are locked: the rolling deploy populated every cache within the same few seconds, so 800 independent five-second TTLs expire together and nothing in the system ever scatters them again.
  4. Fix three independent defects rather than one. Single-flight per key per process, so a process has at most one outstanding fetch for a key. A per-process randomised TTL, for example uniform over four to six seconds, so expiry decorrelates after any synchronising event. Refresh-ahead at about 80 percent of the TTL, so the hot key is replaced before it expires and a request never waits on a miss at all.
  5. Keep fail-closed honest about what staleness costs. Serving a stale snapshot on a fetch error is safe for the engagement end-date case, because ends_on travels inside the snapshot and the caller evaluates it locally. It is not safe for a mid-engagement suspension, which only arrives as an invalidation event. So bound the stale window explicitly at tens of seconds, fail closed beyond it, and state the resulting suspension propagation delay as the accepted price.
  6. Name the new steady state so nobody is surprised by it: compute it from the refresh interval rather than the TTL, because refresh-ahead is what now sets the period. Refreshing at 80 percent of a 5-second TTL means each process fetches every 4 seconds, assuming the refreshed entry restarts the TTL clock, so the floor is fleet size divided by the refresh interval: 800 / 4 = 200 requests per second, not the 800 / 5 = 160 that the TTL alone would suggest. Jitter nudges it up rather than down, because the rate averages 1/T and not 1/mean(T): with TTL uniform over 4 to 6 seconds the floor is about 203 requests per second. A process that stops asking for the key stops refreshing it, so 200 is the floor for a uniformly hot fleet and an upper bound for any other traffic mix. Verify by replaying 800 synchronised clients and confirming the origin sees a flat rate at that level with no burst above a stated ceiling.

Follow-up

  • At 8,000 processes the steady refresh floor becomes 8,000 / 4 = 2,000 requests per second, set by process count and refresh interval alone and independent of user traffic. What changes in the design?
  • A client is suspended for non-payment at 09:00. Walk the worst-case time until every process stops serving them, with your stale window in place.
  • How would you detect the next stampede automatically, given that the origin's average rate barely moves when one occurs?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

  • Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
  • Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
  • Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

  • Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
  • Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
  • Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

  • Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
  • Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
  • Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

  • For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
  • Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
  • Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
  • Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

  • Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
  • Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
  • Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
  • Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

  • Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
  • Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
  • Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

  • Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
  • Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
  • Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

Outage answers are graded on instruments

Nobody is scoring your stamina at three in the morning. What carries weight is which signal told you something was wrong, what you measured before touching anything, what you rolled back versus what you fixed forward, and why you picked one. 'We restarted it and it went away' is a story about not knowing.

Describe your experience managing software deployments for hardware-dependent systems.

Medium · behavioural and engineering judgement

Approach

  1. Pick a story where you made the decision, not one where you watched it.
  2. Close with what you would do differently, concretely.
  3. Name the disagreement and how you resolved it with evidence.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?

Describe a time you had to explain a complex technical issue to a non-technical stakeholder.

Medium · behavioural and engineering judgement

Approach

  1. State the situation in two sentences and spend the rest on the reasoning.
  2. Give the blast radius: what could have broken, and what you measured.
  3. Close with what you would do differently, concretely.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?

Can you discuss a situation where you had to collaborate with field engineers to resolve a software-related hardware failure?

Medium · behavioural and engineering judgement

Approach

  1. State the situation in two sentences and spend the rest on the reasoning.
  2. Give the blast radius: what could have broken, and what you measured.
  3. Name the disagreement and how you resolved it with evidence.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?

Tell us about a time you identified a potential system failure before it occurred.

Medium · behavioural and engineering judgement

Approach

  1. Pick a story where you made the decision, not one where you watched it.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Name the disagreement and how you resolved it with evidence.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?