cswork / libraryOpen practice workspace

Interview guides · APR Consulting

APR Consulting Software Engineer Interview Questions & Guide 2026

API design practice for APR Consulting software engineer candidates: compatibility, versioning, and changes you cannot take back.

The 60-second brief

At APR Consulting, a Software Engineer serves as a vital bridge between complex technical requirements and high-stakes project delivery. You will often be embedded within specialized teams working on critical infrastructure, aerospace, and defense-related projects. Your work is not just about writing code; it is about ensuring that systems—whether they are missile tracking models, satellite systems, or regional strike programs—operate with precision and reliability.

Find out what you will be typing into. A shared plain-text editor with no autocomplete, compiler or test runner changes what you have to hold in your head, and practising inside your own configured environment hides exactly that gap.

Focus areas

  • Make integration retries idempotent without target-side idempotency keys
  • Diagnose failures inside environments you cannot instrument
  • Evolve schemas expand-contract across un-upgradable client deployments

Overview

This role is inherently collaborative and demanding. You will contribute to projects that require a high degree of technical rigor and, in many cases, strict adherence to security protocols. Because APR Consulting places you within diverse client environments, you must be adaptable, capable of navigating legacy systems, and ready to solve problems that have significant real-world implications for national security and advanced engineering sectors.

Many roles at APR Consulting require active security clearances, such as Secret or Top Secret. Ensure you are prepared to discuss your clearance status early in the process, as it is often a prerequisite for the specific mission-critical projects you will support.

The APR Consulting interview process

APR Consulting candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

The person on this call usually cannot evaluate your code and does not need to. They write a short paragraph, and that paragraph is what a hiring manager skims when deciding who to put on your loop. So the test is not whether your work was hard, it is whether a non-engineer can repeat it correctly. Name systems by what they did rather than by their internal codename, give each project a shape (what was breaking, what you changed, what happened after), and keep the whole walkthrough near ninety seconds. Depth that cannot survive a paraphrase reads as vagueness.

What to demonstrate

  • Whether a non-engineer can restate your projects without distorting them, since their paraphrase is what travels to the hiring manager, not your sentences
  • Whether each project has a shape rather than a stack list: the failure or constraint, the change you made, the result and how it was measured
  • Whether you can say what was yours inside a team project without either inflating it or disappearing into the plural

How to prepare

  • Rewrite each headline project as two sentences with no internal system names and no acronyms outside your company, then say them to someone outside engineering and have them repeat them back. Fix whatever came back wrong
  • Attach one measured number to each project: the baseline, the change, and the window it was measured over. Where nothing was ever measured, say that plainly rather than reaching for a plausible percentage
  • Time the background walkthrough against a clock. If it runs past two minutes, compress the earliest role to a single clause and spend the recovered time on the most recent one

Technical Deep-Dive

What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.

What to demonstrate

  • Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
  • Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
  • Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly

How to prepare

  • Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
  • Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
  • Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live

Behavioral Discussion

Many of these questions are about something that went wrong, and the grading sits mostly in the hours after you knew. Who found out first, whether that was you or an alert or a user, how long it took you to say it out loud, and whether the people who needed the news got it while they could still act on it. Engineers under-tell this part because it feels like confessing. The pattern it is looking for is the opposite: the quiet fix, an incident absorbed without telling anyone, after which nothing changed and the same failure is still available.

What to demonstrate

  • How the problem was found, and whether that route was one you had built or one that happened to you, since a user reporting it first means your instrumentation did not cover that failure
  • Whether time-to-detect and time-to-tell are separate numbers in your account and whether you know both, because a fast fix that nobody heard about until the retro is a different answer from a slow one that was announced immediately
  • Whether the resolution left something durable behind, a check that fires or a default that changed, rather than depending on people remembering to be careful
  • Whether you can say what the failure cost without either inflating it or waving it away

How to prepare

  • Reconstruct one incident you were part of as a timeline with clock times: first bad request, first signal, first person who knew, first message outside the team, mitigation, permanent fix. The gaps between those entries are what gets asked about
  • Look up the configuration of the signal that caught it, including its evaluation window and threshold. An alert defined on a five-minute aggregate cannot fire until the condition holds across that window, which puts a floor under time-to-detect that has nothing to do with how severe the failure was. Be able to say what that floor was and whether anyone had chosen it deliberately
  • Prepare one story where you escalated early and the severity turned out to be smaller than you thought, including what it cost the people you pulled in. Without it, every answer you give about raising alarms is unfalsifiable

Final Selection

Where the day includes a partner from product, design or data, that conversation is weighted like the technical ones and prepared for least. They are deciding one thing: whether having you in the room makes their decisions cheaper. That means options with costs attached, not implementation detail and not "it depends". An estimate someone can plan against — a range, the assumption that would push it to the high end, and what you would drop to hit the low one — is worth more than a confident single number, which everyone present already knows is wrong.

What to demonstrate

  • Whether an estimate comes as a range with the assumption most likely to break it, and states what a specific scope cut would actually buy
  • Whether a technical constraint is handed over as a choice with consequences on their side, rather than as a verdict they have no standing to argue with
  • Whether you establish what decision is on the table before proposing anything
  • Whether risk is raised while it can still change the plan, with the trigger that would confirm it, instead of reported afterwards as a slip

How to prepare

  • Take a project that shipped late and write the two-sentence warning you could have given three weeks earlier, naming what you would have needed decided at that point
  • Rehearse one estimate out loud until it arrives in three parts: the range, the single assumption that would blow it, and the smallest thing you would cut to protect the date
  • Rewrite an objection you have actually made — the "we can't do that" version — as two options with their costs, so the choice ends up with the person who owns it

Where candidates lose points

Depending on a client field because it is always populated

It is populated by their operational process, not by a constraint in their database, and their process changes without telling you. When it stops being populated, a permissive pipeline writes nulls or defaults into derived data and the damage spreads for weeks before anyone notices. Validate at the boundary and fail the run with failure_class='schema_mismatch' plus a redacted sample, so the failure is loud, attributable to the client's change, and cheap to diagnose. A run that fails clearly is far cheaper than a backfill that has to reconstruct which rows were silently wrong.

Treating an ambiguous failure as a definite one

A timeout, a 502 from an intermediary, or a connection reset after the request bytes were sent all leave the target's state unknown. Classifying those as failures and retrying duplicates the effect; classifying them as successes and advancing the watermark loses data silently. Both wrong answers are common because the ambiguous case is rare in a sandbox and routine in production. The run needs a distinct ambiguous state, a dedupe key that makes the retry safe, and a reconciliation read against the target when the key alone cannot settle it.

Trusting input because it came from your own front end

Anything crossing a trust boundary is hostile: parameterise queries instead of building SQL by concatenation, validate against an allow-list rather than a deny-list, and bound the size of anything you allocate from a request. Raising this unprompted in an API or design question is a cheap and unusually strong signal.

Sorting when the problem never required a total order

Match the algorithm to the guarantee actually needed: the top k comes from a size-k heap in O(n log k) time and O(k) space, distinctness needs a set rather than an ordering, and a small bounded integer key range admits a linear counting pass. A full O(n log n) sort is the right default only when you genuinely need everything in order.

Questions & practice

Compare release versions and report the live estate spread

Easy · parsing · version skew · ordering

Each environment row carries reported_release as TEXT: either NULL, or a version of the form MAJOR.MINOR.PATCH with an optional -rc.N suffix. Up to 400 environments. Ordering rule: compare major, then minor, then patch numerically; a -rc.N build precedes the same MAJOR.MINOR.PATCH with no suffix, and rc numbers compare numerically. Return the lowest and highest reported versions, the minor-version spread between them, and the count of environments whose reported_release is NULL. NULL means the environment's version is unknown, not that it converged.

Approach

  1. Parse each string once into a comparable tuple: (major, minor, patch, is_final, rc), where is_final is 1 for a plain release and 0 for a pre-release, and rc is 0 when absent. Tuple comparison then yields the stated order for free, including 3.10.0-rc.1 < 3.10.0. O(L) per string, O(n*L) total.
  2. Track min and max in one linear scan rather than sorting: O(n) time, O(1) extra space, and n is only a few hundred so the parse dominates either way.
  3. Fail loudly on a string that does not match the grammar instead of defaulting it to 0.0.0 or to the newest release. An unparseable reported_release is a reconciliation-ingest bug, and either default hides it in the direction that looks safe.
  4. Compute the minor spread only when min and max share a major version. Across majors the difference of minor numbers is meaningless, so return the pair of versions and flag the condition rather than a number.
  5. Report the NULL count as its own output line. Folding unknowns into 'converged' is how an install ends up running a release nobody knew was still live.

Follow-up

  • Two majors are live at once. What does 'spread' mean now, and what single number, if any, would you still publish?
  • reported_release is a claim from the environment, and last_heartbeat_at may be days old for a client-managed install. At what staleness does this figure stop being usable?
  • How would you enforce the supported window when the rollout plan is built, instead of reporting a violation after it already exists?

Resolve effective-dated engagement versions for two million access checks

Hard · binary search · effective dating · complexity budget

The engagement table holds 500,000 effective-dated rows: (engagement_id, version, effective_from, effective_to which is NULL for the open version). A database exclusion constraint guarantees that per engagement_id the tstzrange(effective_from, effective_to) values never overlap, and the ranges are half-open. You must answer 2,000,000 access checks of the form (engagement_id, at_time) in one process with no database, returning the version open at that instant or none. Also report, per engagement, any coverage gap between adjacent versions. Give the complexity of the scan-per-check approach and say why it is not an option.

Approach

  1. State the naive cost first: scanning the table per check is 2e6 x 5e5 = 1e12 range comparisons. Even at 1e9 simple comparisons per second that is roughly fifteen minutes of pure compare work, against a budget measured in seconds. It is correct, just unaffordable, and the fix is an index built once rather than a faster comparison.
  2. Build a hash map engagement_id -> array of versions sorted by effective_from, once. O(V log V) to sort, O(V) memory. The build is amortised over 2e6 lookups, which is what makes it worth doing at all.
  3. Answer each check by binary-searching for the last effective_from <= at_time, then confirming effective_to IS NULL OR at_time < effective_to. If the confirmation fails, the instant lies in a gap and the answer is none, not the neighbouring version. O(log v) per check, O(V log V + Q log v) total.
  4. Name the precondition: predecessor search is only valid because the ranges per engagement do not overlap, which the exclusion constraint enforces with btree_gist for the equality operator. Without that guarantee you need an interval tree returning all stabbed intervals, at O(log n + k). Use the same half-open convention as the database, where at_time equal to effective_to belongs to the next version.
  5. Find gaps with one O(V) pass over each sorted array comparing prev.effective_to against next.effective_from. The exclusion constraint forbids overlaps, not holes, so a hole is entirely possible and is where an access check fails closed for a reason nobody can see in the contract.
  6. If the checks can be buffered, sort them by (engagement_id, at_time) and merge against the version arrays: O(Q log Q + V log V), usually a large constant-factor win over 2e6 random binary searches because the merge walks memory in order.

Follow-up

  • The open version has effective_to NULL. How do you keep both the sort and the comparison total without a sentinel that overflows your timestamp type?
  • A gap exists for one engagement and a check lands in it. What should the access path return, and what should it emit so someone fixes the amendment?
  • Amendments are being written while you answer checks. What read model keeps an answer self-consistent, and what does a check mean if the snapshot is a second stale?

Top-k longest connector runs from a stream too large to sort

Medium · heap · top-k · streaming

A day of connector_run rows sits in object storage: 200,000,000 rows, unsorted, far larger than memory, and streamable once. Each row has run_id, connector_id, started_at and finished_at, where finished_at is NULL for runs that never reached a terminal state. Return the 200 longest terminated runs by (finished_at - started_at), ties broken in favour of the lower run_id, plus the count of rows with a NULL finished_at. You may not sort the input or hold it in memory. State the time and space complexity you achieve.

Approach

  1. Keep a bounded min-heap of capacity k=200 ordered on (duration, -run_id). The root is then the weakest retained entry: smallest duration, and on a duration tie the largest run_id, which is exactly the one the stated tie-break should evict.
  2. Per row: if finished_at is NULL, increment a counter and skip; otherwise compute the duration and push when the heap holds fewer than k, else compare against the root and replace only if strictly better. O(n log k) worst case, O(k) space, and after the heap warms nearly every row costs a single comparison against the root.
  3. Reject the alternatives explicitly. A full sort is O(n log n) and at 2e8 rows means an external merge sort over hundreds of gigabytes to produce 200 rows. Quickselect is O(n) average but needs the whole array resident, which the constraint forbids.
  4. Handle the NULL as data, not as an edge case: depending on the language, subtracting from NULL either throws or yields a sentinel that lands at the top of the list. Count those rows and report them, because a spike of unterminated runs is its own signal.
  5. To parallelise, give each of p shard readers its own size-k heap and merge the p results. The union of per-shard top-k sets contains the global top-k, because an element beaten by fewer than k rows globally is beaten by fewer than k rows in its own shard, so the merge is exact at O(pk log k).

Worked solution (20 min)

  1. Stream eleven rows as (run_id, duration_seconds): (1,12) (2,405) (3,7) (4,88) (5,405) (6,3) (7,1200) (8,61) (9,NULL) (10,99) (11,405). Use k=3.
  2. Fill the heap with the first three terminated rows, then maintain it: after (2,405) and (5,405) and (7,1200) have been seen, the heap holds those three and the root is (405, -5), which is run 5.
  3. Row 9 has no finished_at, so it increments the unterminated counter and never reaches the heap.
  4. Row 11 has duration 405, equal to the root's duration, but run_id 11 > 5, so under (duration, -run_id) it is not strictly better than the root and is discarded on the tie-break rather than on duration.
  5. Drain the heap in descending order to produce the final list.

Expected result: Top 3 = run 7 (1200s), run 2 (405s), run 5 (405s). Unterminated count = 1 (run 9). Run 11, also at 405s, is excluded by the lower-run_id tie-break.

Verify it yourself

  • The heap never holds more than k=3 entries at any point in the stream.
  • The heap root is always the weakest retained entry, so every discarded row compared unfavourably against it.
  • Brute-force sorting the 10 terminated rows and taking the first 3 gives the same set, including the same tie resolution.
  • Splitting the same stream into two shards with independent size-3 heaps and merging yields the identical top 3.

Follow-up

  • Now return the top 20 bindings by total run time instead of the top runs. What changes, and how many distinct keys must you hold to make it exact?
  • The key space is too large to hold an exact aggregate. What does a Misra-Gries summary with m counters actually guarantee about the counts it returns?
  • Rows now arrive continuously and you want a rolling one-hour top-k. Does the bounded heap still work, and if not, what breaks?

Denormalise the entitlement snapshot without freezing the clock

Medium · denormalisation · read patterns · caching

Authorisation currently joins engagement (effective-dated, with status, ends_on, access_grace_days, effective_from, effective_to), entitlement and principal_assignment on every request, at the platform's full read rate, against tables taking a few hundred writes a day. You are asked to publish a denormalised snapshot that callers cache with a few seconds of staleness and invalidate on change events. Specify the snapshot's key and shape, say which columns you denormalise, name the one thing that must not be precomputed, and say what a cache miss does.

Approach

  1. Justify the shape from the ratio rather than from taste: a few hundred writes a day against the whole estate's read rate is the case for a published read model. The normalised tables stay the system of record; the snapshot is derived, versioned and disposable, and can be rebuilt from source at any time.
  2. Key it on (client_id, principal_id) with a monotonic snapshot_version and a generated_at. Consumers cache by version, refresh on a change event, and reject anything older than the staleness bound, which turns the bound into an enforced property instead of an assumption about how fast events usually arrive.
  3. Denormalise only what changes on a write: the open engagement version's engagement_id, status, isolation_tier, data_region, ends_on, access_grace_days, and the resolved scope set. Resolving the effective-dated row once at publish time — selecting the version WHERE effective_to IS NULL — is the entire saving, because that resolution is what the per-request join was paying for.
  4. Do not precompute is_active or any boolean that folds now() into the stored value. A write event invalidates the snapshot; the passage of time does not, so an engagement that ended an hour ago keeps authorising until something unrelated triggers a republish. Ship ends_on and access_grace_days as data and compare them to the current time at read, so time-based expiry needs no invalidation at all.
  5. Fail closed on a miss or on over-staleness: deny, and do not fall through to the source tables. A fallback read path means the first mass cache miss converts an availability incident into either an authorisation bypass or a stampede onto the one service whose availability target already exceeds everyone else's.
  6. State the cost you accepted: a revoked entitlement is still honoured for up to the staleness bound. That is defensible for widening access and not for narrowing it, so pair it with an out-of-band deny signal for revocation and closure, and write the residual window into the runbook instead of leaving it for an auditor to discover.

Follow-up

  • How do you detect the snapshot quietly diverging from the source tables, given that a wrong snapshot is silently permissive?
  • An amendment creates a new engagement version mid-request. Which version does the in-flight request use, and can you defend that?
  • The snapshot carries data_region. What stops the snapshot itself from being replicated somewhere its own residency rule forbids?

Soft-delete environments without breaking uniqueness or audit history

Easy · soft deletes · partial indexes · audit history

environment holds one row per provisioned deployment: environment_id, client_id, engagement_id, env_name, region, status, destroyed_at TIMESTAMPTZ NULL. Decommissioned environments stay readable for seven years, and env_name must be unique per client among live environments only, because a client may reuse a retired name. Design the constraint set and the history table, write the query that lists a client's live environments, and say what your design does about connector_run rows whose foreign key points at a destroyed environment.

Approach

  1. Keep the tombstone as a nullable timestamp rather than a boolean: destroyed_at carries when as well as whether, and enforce live-only uniqueness with a partial index, CREATE UNIQUE INDEX env_live_name ON environment (client_id, env_name) WHERE destroyed_at IS NULL, which indexes only the live subset and so stays small.
  2. Get the NULL semantics right before you pick a spelling. In a B-tree unique index two NULLs compare distinct by default, so plain UNIQUE (client_id, env_name, destroyed_at) enforces nothing at all among live rows — every live row is unique on the third column purely by virtue of being NULL. PostgreSQL 15's UNIQUE NULLS NOT DISTINCT (client_id, env_name, destroyed_at) does satisfy the requirement: the NULL live rows now compare equal and collide with each other, while retired rows carry distinct destroyed_at timestamps and stay distinct, so a name can be retired and reused any number of times.
  3. Prefer the partial index anyway, and be able to say why rather than asserting it. It indexes only the live subset — tens of rows per client — instead of seven years of tombstones; it needs no version floor; and it keeps destroyed_at out of the key, so it cannot be defeated by two rows retired at the same instant. That last case is reachable: now() is transaction-start time, so retiring a name, recreating it and retiring it again inside one transaction writes two identical destroyed_at values and raises a duplicate-key error under the constraint form. clock_timestamp() avoids it if you take that route.
  4. Put history in a separate append-only table written by an AFTER INSERT OR UPDATE OR DELETE trigger holding environment_id, changed_at, changed_by, op and the before and after row images. PostgreSQL has no system-versioned tables, so this is a trigger or a logical-decoding consumer, not a DDL flag, and the soft delete arrives at that trigger as an UPDATE rather than a DELETE.
  5. State the foreign-key consequence rather than leaving it implicit: a soft delete leaves every referencing connector_run row resolvable, which is exactly what the audit requirement wants, but it means no read is protected by the database. Give the predicate one home, a view or an RLS-backed view that every caller uses, instead of repeating destroyed_at IS NULL across hundreds of call sites.
  6. Size the retention: seven years of history plus tombstoned rows in the hot table. Partition the history table by month so expiry is a DETACH PARTITION rather than a DELETE that has to be vacuumed, and decide separately whether a residency-driven erasure request can be honoured against an append-only store at all.

Worked solution (20 min)

  1. Write the DDL with destroyed_at TIMESTAMPTZ NULL plus the partial unique index on (client_id, env_name) WHERE destroyed_at IS NULL.
  2. Insert two live rows sharing (client_id, env_name) and confirm the second is rejected; set destroyed_at on the first and confirm the insert now succeeds.
  3. Build the same table a second time under UNIQUE NULLS NOT DISTINCT (client_id, env_name, destroyed_at) and confirm it passes the same two tests, so you are choosing between two working designs rather than rejecting a broken one.
  4. Add environment_history (history_id, environment_id, changed_at, changed_by, op, row_before JSONB, row_after JSONB) and the AFTER trigger that populates it.
  5. Write the live-listing query: SELECT environment_id, env_name, region, status FROM environment WHERE client_id = $1 AND destroyed_at IS NULL ORDER BY env_name.

Expected result: A partial unique index covering only live rows, a separate append-only history table with its own partitioned retention, and a listing query whose predicate matches the index predicate exactly so the planner can use it.

Verify it yourself

  • Run retire-and-recreate three times for the same name: three tombstoned rows and one live row coexist without violating the index.
  • On the NULLS NOT DISTINCT variant, retire and recreate the same name twice inside one transaction and watch it fail on duplicate destroyed_at, since now() does not advance within a transaction; the partial index has no such failure mode.
  • EXPLAIN the listing query and see the partial index; widen the predicate to destroyed_at IS NULL OR destroyed_at > now() and watch the index drop out, because that predicate no longer implies the index's.
  • Soft-delete a row and confirm the history trigger fired; a DELETE-only trigger records nothing, which is the common way this audit trail ends up empty.

Follow-up

  • What stops the next endpoint from forgetting the destroyed_at IS NULL predicate, and does that mechanism survive a developer writing raw SQL?
  • The history table stores row images as JSONB and the base table gains a column. What do old rows now mean, and how do you read them six years later?
  • A residency erasure request requires a genuine hard delete for one client. What breaks, and what do you keep?

How do you ensure your code or system design meets strict security or compliance standards?

Medium

Approach

  1. Say what you would check first and why it is the highest-information step.
  2. Work from the requirement backwards to the design.
  3. State your assumptions explicitly before working the problem.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

What is your approach to troubleshooting critical failures in a production environment?

Medium

Approach

  1. Say what you would check first and why it is the highest-information step.
  2. State your assumptions explicitly before working the problem.
  3. Clarify what is being asked and what a complete answer contains.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Fan out desired state and collect convergence evidence per tier

Medium · fan-out · reconciliation · partial observability · rollout

The control plane is the only writer of desired state for a few hundred environments (environment_id, isolation_tier, region, desired_release_id, reported_release_id, reported_config_fingerprint, status, last_heartbeat_at). Shared-pool and dedicated environments sit in networks you can reach; client-managed installs do not and can only report outward on their own schedule, sometimes not for weeks. You have to roll out a release and then, later, decide it is safe to remove a deprecated API field. Design the distribution path and the reporting path, and define exactly what converged means.

Approach

  1. Split delivery by reachability instead of pretending one mechanism covers both. Reachable tiers get a push-and-reconcile loop you drive; client-managed installs pull desired state and post a report, because nothing you build reaches into a network you have no route to. Both write the same reported_ columns, so the question of what is live stays one query rather than two subsystems with different truths.
  2. Define converged as a positive statement from the environment: reported_release_id equals desired_release_id, reported_config_fingerprint equals the fingerprint of the desired config, and last_heartbeat_at falls inside that tier's freshness window. A NULL reported_release_id means unknown, and unknown must never be folded into converged - that fold is precisely how a client's install ends up on a version nobody knew was running.
  3. Recognise that a few hundred targets is not a throughput problem, so spend the design on evidence rather than on delivery concurrency. Carry reported_at per environment, expose its age, and make any 'N of M converged' number carry the age distribution of the missing rows; without that, the dashboard's reassuring figure is the thing that causes the incident.
  4. Gate the contract step on observed convergence, never on elapsed time. The removal condition is that no fresh report names a release below the target and no environment is stale or unknown, which means one silent install blocks the removal - the correct and unpopular answer. Then make the block actionable by reporting it per client with an age, so it becomes a specific conversation rather than an indefinite wait.
  5. Bound the version spread rather than discovering it later: refuse to provision or upgrade onto a release outside the supported window, and track max(desired) minus min(reported) as a first-class number. An unmeasured spread widens until some schema change becomes impossible, and by then the cost lands on whoever proposes it.

Worked solution (25 min)

  1. Load 300 environment rows across the three tiers with a dozen distinct release ids and heartbeat ages from seconds to four months.
  2. Write convergence as a three-part condition and removal-safety as its negation, then run both and read the outputs.
  3. Set reported_release_id to NULL on 20 client-managed rows and re-run removal-safety.
  4. Introduce a config fingerprint mismatch on an environment whose release id matches, and see which condition catches it.

Expected result: Removal-safety returns blocked whenever any environment is stale or unknown, and names them with their client and report age rather than returning a bare boolean. The NULL rows block removal instead of counting as converged. The fingerprint mismatch is caught by the config term while the release term passes, which is why convergence is three conditions rather than one.

Verify it yourself

  • No query on the rollout path reads reported_release_id without also testing heartbeat freshness.
  • Converged, diverged, stale and unknown are exhaustive and mutually exclusive, and their counts sum to the environment total.
  • Provisioning an environment onto a release outside the supported window fails rather than widening the spread.

Follow-up

  • A client-managed install has not reported for four months. It may be decommissioned or it may be serving. What do you do before removing the field?
  • One environment reports a release id you never published. What are the candidate explanations, and what do you do with the row meanwhile?
  • How would a rollout stop itself automatically, and what signal could you possibly have for a tier you cannot instrument?

One connector burns a client's entire rate budget

Easy · poison message · retry loop · rate limiting

A client reports that your integration is exhausting their API quota, and every connector binding for that client now fails with client_429. For one binding, connector_run holds 300 rows sharing one idempotency_key with attempt_no incrementing, all with status 'running', finished_at NULL and failure_class NULL, created exactly 120 seconds apart over the last ten hours. The run lease is 120 seconds. Workers in that pool show restart counts in the hundreds. Other clients are unaffected. Diagnose in order, say who is at fault, and state what stops one range doing this again.

Approach

  1. Split client fault from your fault before anyone contacts the client. Group the last hour's failures by client and by binding: every binding for one client failing while other clients are clean means a resource shared with that client is exhausted, and the only thing you share with them is their quota.
  2. Read the ledger's shape, which is the entire diagnosis. Three hundred attempts under one idempotency_key, none with a terminal status and none with a failure_class, spaced at exactly the lease interval. A handled failure writes failure_class and finished_at; writing nothing at all means the worker was killed before it could classify. The exact spacing names the killer as lease expiry, not the client endpoint.
  3. Quantify the damage from the same rows: records_applied is zero across all 300, while each attempt got far enough to pull pages. The binding consumes quota continuously and applies nothing, which is why the other bindings for that client see 429 and the faulty one does not.
  4. Reproduce offline. watermark_from and watermark_to are recorded, so replay that exact range against a captured payload with the lease disabled and find what exceeds 120 seconds: a page far larger than the others, a validator input that degrades badly, or an unbounded in-memory accumulation over the range.
  5. Fix the loop before the payload. Cap attempt_no, move the range to status 'abandoned' with an alert, and hold the watermark rather than advancing past it, because a silent gap is data loss and skipping must require a human decision. Then make the work fit the lease: stream and chunk the page, renew the lease only while progress is provable, and bound the page size at the boundary.
  6. Add the detector this evaded. Alert on attempts that never reach a terminal state, not only on failure counts, because a run that never finishes never increments any failure metric.

Follow-up

  • The client asks how much of their quota you consumed. Which rows answer that, and what would you have to add to answer it exactly?
  • The abandoned range contains records you still need. What is the safe way to get past it without breaking the at-most-once guarantee?
  • Lease renewal introduces a new failure: a worker that renews steadily but makes no progress. How do you catch that one?

Suggested practice checklist

For someone fluent in a dynamic language who has shipped real work but has never had to say what the runtime is doing underneath. The week is built on measuring and deliberately breaking things, because the questions that expose this background are the ones where the interviewer asks why a second time.

Day 1 — Measure before reasoning

  • Take a slow piece of your own code, write down in advance where you believe the time goes, then profile it and record how wrong the guess was. The cost is usually an allocation you did not notice or an accidental quadratic membership test.
  • Replace one list membership test inside a loop with a set and measure at a thousand, ten thousand and a hundred thousand elements, confirming the shape of the curve rather than only that it got faster.
  • Write down the three quantities you can now measure instead of assert: wall time, peak memory, and call count for the function you suspected.

Deliverable: A before-and-after profile of real code plus a written note on the size of the gap between the guess and the measurement.

Day 2 — References, copies, and the bugs they produce

  • Write the function with a mutable default argument, call it three times, and explain the accumulating result: the default is evaluated once when the function is defined, so every call shares one object.
  • Build a nested structure, take a shallow copy, mutate an inner element, and show that both views changed, because a shallow copy duplicates the container and not the elements. Then fix it with a deep copy and state the cost you just accepted.
  • Write two functions, one mutating its argument in place and one rebinding the local name, and predict the caller's view of each before running it. That single distinction produces most of the bugs that pass their tests.

Deliverable: Three small programs whose output you predicted correctly before running, each with a one-line statement of the rule underneath.

Day 3 — Types, once, in a language that checks them

  • Port one module you have already written, roughly a hundred lines, into a statically typed language, and record every place the compiler demanded an answer your original had left implicit: a value that can be absent, a numeric width, a case never handled.
  • Write the same signature in both languages and state what the static one guarantees before the program runs and what it does not, since it will not save you from a wrong algorithm or an index out of range.
  • Write the difference between an interface satisfied by declaration and one satisfied structurally, with one case each where the other approach would miss the mistake.

Deliverable: One module in two languages plus a list of the questions the type checker forced you to answer.

Day 4 — Concurrency, starting with what actually runs at the same time

  • Run the same CPU-bound function across four threads and four processes and measure both. Under the default CPython build the threaded version will not speed up, because only one thread executes bytecode at a time; the process version will. Check which build you are on first, since free-threaded builds remove that lock and change the result.
  • Then run a blocking I/O workload across four threads and measure it speeding up, because the interpreter releases that lock around blocking calls, which is why treating threads as useless is wrong as a general claim.
  • Build the lost update: two threads each incrementing a shared counter a hundred thousand times, and show a final value below the expected sum, because an increment is a load, an add and a store and the thread can be suspended between them. Fix it with a lock and then measure what the lock costs.

Deliverable: Three measurements, threads against processes on CPU work, threads on I/O work, and a demonstrated lost update, each with the mechanism written underneath.

Day 5 — Debugging as a procedure rather than an instinct

  • Work one real failure as a bisection: find a revision or an input size where it is good and one where it is bad, halve repeatedly, and state the two assumptions bisection needs, that the property changes exactly once across the range and that the test is reliable.
  • Minimise one failing input to the smallest version that still fails, and record how many rounds it took.
  • Keep a hypothesis log for one bug in three columns, what I believe, what would disprove it, what I observed, and stop yourself the first time you are about to change two things at once.

Deliverable: One bug worked to root cause with a written hypothesis log and a minimised reproducing input.

Day 6 — Tests that catch the bug you are about to write

  • Implement an LRU cache with a capacity bound, then write the three test cases that would catch an off-by-one in eviction: insert exactly capacity items and assert nothing was evicted, insert one more and assert the least recently used key is the one gone, and read an old key just before that insert so the eviction victim changes.
  • Add a property test comparing your implementation against a deliberately slow reference, an ordered list scanned linearly, over a few thousand random operation sequences, because a slow reference finds the cases you would not have thought to write.
  • Write one numeric test that fails under exact equality and passes with a tolerance, and state why the tolerance has to be relative rather than absolute once the magnitudes grow.

Deliverable: An LRU implementation with three boundary tests, one property test against a slow reference, and one tolerance-based numeric test.

Day 7 — Debug something broken, out loud

  • Have someone plant three defects in a two-hundred-line program, an off-by-one, a shared mutable state bug, and a wrong error-handling path, then find them while narrating, under a fixed rule: state the hypothesis before touching anything.
  • Time each one and record which tool found it, reading, a printed value, a debugger, or a test, because the question asked in interviews is how you would find it rather than what it was.
  • Write the sentence you will use when you do not yet know the cause, one that names the next measurement instead of offering a guess.

Deliverable: A recorded debugging session with time-to-find per defect and the method that found each.

The failure that was yours

Keep one story where the bad call was yours rather than a dependency's or a manager's. Name the check that would have caught it, whether you added that check afterwards, and whether it has fired since. Answers that route blame outward end the conversation early; answers that end in a guardrail someone still relies on tend to open it up.

Tell us about a time you identified a flaw in a design before it reached the implementation phase.

Medium · behavioural and engineering judgement

Approach

  1. State the situation in two sentences and spend the rest on the reasoning.
  2. Pick a story where you made the decision, not one where you watched it.
  3. Name the disagreement and how you resolved it with evidence.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?

Can you describe your experience with systems engineering lifecycles in an aerospace or defense context?

Medium · behavioural and engineering judgement

Approach

  1. Give the blast radius: what could have broken, and what you measured.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?

Describe a situation where you had to pivot your technical strategy due to changing project requirements.

Medium · behavioural and engineering judgement

Approach

  1. Name the disagreement and how you resolved it with evidence.
  2. Give the blast radius: what could have broken, and what you measured.
  3. Close with what you would do differently, concretely.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?

How do you handle integration challenges when working with complex, multi-layered systems?

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Give the blast radius: what could have broken, and what you measured.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?