8th Light Software Engineer Interview Questions & Guide 2026

The 60-second brief

At 8th Light, the Software Engineer role is defined by a commitment to the principles of software craftsmanship. You are not just writing code; you are solving complex business problems while acting as a mentor, consultant, and technical leader. The work often involves high-stakes projects where your ability to build maintainable, scalable, and high-quality systems directly impacts client success.

Prepare in one language you know well enough to debug in rather than the one you think reads best. Under a clock an unfamiliar language costs you standard-library lookups and iteration mechanics, and that time comes out of your thinking budget, not your typing budget.

Focus areas

• Scope every query by tenant below application code
• Make integration retries idempotent without target-side idempotency keys
• Version an API you cannot force clients to upgrade

Overview

This position is critical because 8th Light operates as a consultancy, meaning you will frequently interface with clients to translate their needs into robust technical solutions. You will work within diverse problem spaces, ranging from large-scale data systems to agentic AI implementations. Success in this role requires a balance of deep technical expertise and the interpersonal skills necessary to guide non-technical stakeholders through the software development lifecycle.

The 8th Light interview process

8th Light candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Before anything technical happens, someone has to decide which rung of the ladder your loop is calibrated to, and that decision sets the bar for every round after it. It comes from how you describe scope, not from your title, because titles do not convert cleanly between companies. The weak version of the answer is team size and years. The strong version names the largest change you shipped where nobody reviewed the design, what would have broken if you had been wrong, and what you were paged for. Get the level said out loud on this call, because the range and the loop both follow from it.

What to demonstrate

• Whether the scope in your own account maps onto a level the team actually has an opening at, so a mismatch ends the process cheaply rather than after four interviewers have spent a day
• Whether your title needs re-mapping: the same word describes very different amounts of independent decision-making at a twenty-person company and a ten-thousand-person one
• Whether your compensation expectation can be filled at that level in the structure the role pays in, which is why the number gets asked for before any engineer is scheduled

How to prepare

• Write down two changes from the last two years: the largest one you designed with nobody reviewing the design, and the largest one where someone more senior did. Lead with the first when scope comes up, and be ready to say which parts of the second were yours
• Ask which level the loop is calibrated to and what changes at the level above it, then plan your weeks from that answer rather than from the posting
• Settle a total-compensation range beforehand with the split named, base against bonus against equity and its vesting period, so a question about numbers gets a number instead of the word market

Technical Assessments

Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.

What to demonstrate

• Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
• Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
• Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
• Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply

How to prepare

• For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
• For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
• Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last

Take-Home Assessment

A test suite is read as a statement about what you believe can break, which is why the selection matters far more than the count. Assertions that repeat the happy path with different numbers tell a reviewer nothing. Cases at the boundaries tell them where you looked: empty input, one element, duplicates, a malformed row, a value at the edge of the accepted range. The sharpest question to ask of a test you have just written is whether it would fail if the logic were wrong. If deliberately breaking the function leaves the suite green, that test is documentation rather than a check.

What to demonstrate

• Which cases you chose, and whether the suite reaches empty, single, duplicate and malformed inputs instead of the prompt's example with the numbers changed
• Whether the suite is deterministic and runs offline, so it produces the same result on a reviewer's machine as on yours
• Whether assertions are written against behaviour rather than internals, so a reasonable refactor does not force the tests to be rewritten
• Whether a failure is readable on its own, naming the input and the expected value rather than reporting a bare assertion

How to prepare

• Mutate your own code by hand before submitting: change a < to <=, drop the last iteration of a loop, return the input unchanged. Anything that stays green is a missing test, and this takes minutes
• Remove nondeterminism before writing assertions. Pass the clock in rather than calling now() inside the logic, seed any random generator explicitly, and do not assert on the iteration order of a Go map or a Python set, since Go randomises that order deliberately and Python salts string hashing per process unless PYTHONHASHSEED is pinned
• Time a full run of the suite and trace anything slow to its cause, usually a sleep or a real network call, then replace it with a fake so the reviewer is not waiting

System Design Discussion

You cannot drill a format you do not know, so put the preparation into material that travels. Three pieces of your own work, each rehearsed until you can take a follow-up you did not anticipate, will carry a conversation or a code walkthrough equally well. Specificity is what separates that from filler. A number needs its definition before it means anything: a p99 is over some window and measured at some hop, and a server-side figure excludes the queueing and network time a client would see. The number you cannot qualify is the one to leave out.

What to demonstrate

• Whether your examples carry detail only someone who did the work would hold, such as what the binding constraint actually was, which alternative you rejected and why it was worse, and what you measured on each side of the change
• Whether a number survives one follow-up, meaning you can say what it was measured over and whether it moved because of your change or merely alongside it
• Whether a failure is described with the specific change that followed it, rather than a lesson stated in general terms
• Whether your part in a team effort is stated accurately, including what other people did

How to prepare

• Write a page on each of three projects covering the constraint, the option you rejected, the measurement before and after, and what went wrong. Cut any line you cannot take a follow-up on, since you are writing the parts you will be pressed on rather than a summary.
• Recover the real figures while you still have access: request volume, data size, latency with its percentile and window, team size, timeline. Note where each came from, whether a dashboard, a design document or memory, and mark the estimates so you can say which they are out loud.
• Take your weakest project story to someone who works in a different area and have them ask why four times in succession. The point where you run out of answer is the part to go and re-read before the round.

Behavioral Interview

This round is deciding whether a change you make without supervision can be allowed to reach production. It is scored on what you knew at the moment you decided, not on how it turned out, so a story that opens with the result and works backwards reads as luck retold as judgement. Say what the options were, what you did not know, what you did to shrink the unknown before committing, and what you accepted as the worst plausible case. The detail that separates answers is a bound: how many users, how much data, and for how long, if you had been wrong.

What to demonstrate

• Whether the reasoning you give was available at the time you decided rather than after the result came in, since a story whose deciding evidence arrived later describes an outcome and not a judgement
• Whether you can put units on the exposure (users, rows, minutes of degraded service) and whether the containment you chose actually bounded it: a canary bounds the request path it fronts, while a background job writing to a shared table reaches every user regardless of which version served their requests
• Whether the reversal path existed before you shipped or was improvised during the incident, and whether it restores state or only stops further damage

How to prepare

• For your three largest changes, write down the one thing you would have had to be wrong about for it to fail, and what your best estimate of it was on the day you shipped. If you never held an estimate, that is the gap the follow-up questions will find
• Write the undo procedure for one of those changes as it existed at the time, then mark which steps restore data and which only stop new damage. Turning a flag off or reverting a deploy ends the new writes; rows already written come back only from a copy you kept, and a dropped column comes back empty unless something outside the schema holds the values
• Rehearse one story from the decision point forward and stop before the outcome, then have someone ask what you would do next. If the story only works with the ending attached, it is an anecdote rather than a decision you can defend

Where candidates lose points

Tenant context leaking across a connection pool

Setting the tenant on a connection and relying on it for the rest of the request looks correct in every test that runs one request at a time. Under transaction pooling the connection goes back to the pool with that session state still attached, and the next checkout, serving a different client, inherits it; row-level security then enforces the previous tenant's policy flawlessly and returns the wrong rows. It reproduces only under concurrency, which is the load profile your integration tests do not have. Use a transaction-scoped setting, and assert that the setting matches the request's tenant immediately before the first query rather than trusting that it was set correctly upstream.

Validating against a client sandbox and assuming production parity

Sandboxes typically carry smaller data, looser or absent rate limits, a schema version behind production, and sometimes synchronous behaviour where production is asynchronous. Code that passes there fails first in the client's production, during a change window you do not control and often cannot get a second one of. The mitigations are specific: assert the observed schema version at the boundary of every run, measure the production rate limit empirically rather than reading it from a document, and design the first production run to be a bounded, reversible slice rather than a full backfill.

Reading the constraints as preamble rather than as part of the problem

The bounds are usually there to eliminate the obvious approach: n up to 10^5 makes an O(n^2) scan roughly 10^10 operations, far outside any per-test time budget, and an input larger than memory rules out loading it at all. When a bound is not given, ask for it, then say out loud which approach it kills.

Abandoning working code to chase the optimal solution

Get the straightforward version correct, state its complexity, and only then optimise, keeping the working version until the faster one passes the same cases. A correct quadratic solution with a stated path to linear beats a half-written optimal one that never ran.

Questions & practice

Collapse connector retry attempts into logical jobs and failure counts

Easy · hash aggregation · idempotency · retries

You are given one UTC day of connector_run rows: (connector_id, idempotency_key, attempt_no, status, failure_class, records_read, records_applied), unique on (connector_id, idempotency_key, attempt_no), up to 5,000,000 rows in arbitrary order. Retries of one logical job share the idempotency_key; the job's outcome is the status of its highest attempt_no. Return, per connector_id: the number of logical jobs, a count of terminally failed jobs by failure_class, and the dedupe hit total, meaning sum(records_read - records_applied) over terminal-succeeded attempts only. One pass over the input; state your memory bound.

Approach

1. Key a hash map on (connector_id, idempotency_key) and keep only the highest attempt_no seen so far plus that row's status, failure_class and record counts. One pass, O(n) time, O(d) space where d is the number of distinct logical jobs, not O(n).
2. Treat a repeated (key, attempt_no) as corrupt input and raise, since the table's uniqueness constraint says it cannot happen; silently overwriting hides a double-insert in the runtime.
3. Fold the d surviving entries into per-connector counters in a second pass over the map, not over the rows. The failure_class histogram is a small fixed-width map per connector because failure_class is an enum.
4. Compute dedupe hits only from terminal-succeeded attempts. A failed attempt's records_read is work attempted, not work deduplicated, and adding it counts the same source records once per retry.
5. If d does not fit in memory, partition the input by hash(connector_id, idempotency_key) into p files and aggregate each partition independently. Same O(n) total work, p sequential passes, memory traded for I/O; the partition function must use the full key or a job's attempts split across files.

Worked solution (20 min)

1. Take six rows: (c=7,K=A,1,failed,client_429,0,0), (c=7,K=A,2,succeeded,null,500,480), (c=7,K=B,1,ambiguous,client_timeout,300,300), (c=7,K=B,2,succeeded,null,300,0), (c=9,K=C,1,failed,schema_mismatch,0,0), (c=9,K=C,2,failed,schema_mismatch,0,0).
2. Build the map: A -> attempt 2 succeeded, B -> attempt 2 succeeded, C -> attempt 2 failed/schema_mismatch. Three entries from six rows.
3. Fold per connector: connector 7 has 2 logical jobs, 0 terminal failures; connector 9 has 1 logical job, 1 terminal failure classed schema_mismatch.
4. Dedupe hits from terminal-succeeded attempts only: (500-480) + (300-0) = 320 for connector 7, 0 for connector 9.
5. Compute the wrong answer deliberately by counting attempt rows: 6 jobs and 3 failures, which is what a naive GROUP BY on run rows returns.

Expected result: connector 7: 2 logical jobs, no terminal failures, 320 dedupe hits. connector 9: 1 logical job, 1 terminal failure with failure_class='schema_mismatch', 0 dedupe hits. Attempt-level counting would have reported 6 jobs and 3 failures.

Verify it yourself

• The sum of terminal statuses across all connectors equals the number of distinct (connector_id, idempotency_key) pairs, here 3.
• records_applied <= records_read holds on every row, so the dedupe total is non-negative.
• Rerunning the aggregation over a shuffled copy of the input gives identical output, since the highest attempt_no wins regardless of arrival order.
• Map size equals 3, strictly less than the 6 input rows, confirming collapse actually happened.

Follow-up

• A job's highest attempt is 'cancelled' but an earlier attempt succeeded. What is the job's outcome, and what does that say about who writes the cancel?
• Produce this incrementally as runs land instead of as a daily batch: what state do you keep per key, and what happens when an attempt arrives out of order?
• records_applied is written by your runtime, not by the client. What would you reconcile it against before anyone trusts the dedupe-hit number?

Resolve effective-dated engagement versions for two million access checks

Hard · binary search · effective dating · complexity budget

The engagement table holds 500,000 effective-dated rows: (engagement_id, version, effective_from, effective_to which is NULL for the open version). A database exclusion constraint guarantees that per engagement_id the tstzrange(effective_from, effective_to) values never overlap, and the ranges are half-open. You must answer 2,000,000 access checks of the form (engagement_id, at_time) in one process with no database, returning the version open at that instant or none. Also report, per engagement, any coverage gap between adjacent versions. Give the complexity of the scan-per-check approach and say why it is not an option.

Approach

1. State the naive cost first: scanning the table per check is 2e6 x 5e5 = 1e12 range comparisons. Even at 1e9 simple comparisons per second that is roughly fifteen minutes of pure compare work, against a budget measured in seconds. It is correct, just unaffordable, and the fix is an index built once rather than a faster comparison.
2. Build a hash map engagement_id -> array of versions sorted by effective_from, once. O(V log V) to sort, O(V) memory. The build is amortised over 2e6 lookups, which is what makes it worth doing at all.
3. Answer each check by binary-searching for the last effective_from <= at_time, then confirming effective_to IS NULL OR at_time < effective_to. If the confirmation fails, the instant lies in a gap and the answer is none, not the neighbouring version. O(log v) per check, O(V log V + Q log v) total.
4. Name the precondition: predecessor search is only valid because the ranges per engagement do not overlap, which the exclusion constraint enforces with btree_gist for the equality operator. Without that guarantee you need an interval tree returning all stabbed intervals, at O(log n + k). Use the same half-open convention as the database, where at_time equal to effective_to belongs to the next version.
5. Find gaps with one O(V) pass over each sorted array comparing prev.effective_to against next.effective_from. The exclusion constraint forbids overlaps, not holes, so a hole is entirely possible and is where an access check fails closed for a reason nobody can see in the contract.
6. If the checks can be buffered, sort them by (engagement_id, at_time) and merge against the version arrays: O(Q log Q + V log V), usually a large constant-factor win over 2e6 random binary searches because the merge walks memory in order.

Follow-up

• The open version has effective_to NULL. How do you keep both the sort and the comparison total without a sentinel that overflows your timestamp type?
• A gap exists for one engagement and a check lands in it. What should the access path return, and what should it emit so someone fixes the amendment?
• Amendments are being written while you answer checks. What read model keeps an answer self-consistent, and what does a check mean if the snapshot is a second stale?

Assign connector bindings to rollout waves and name the cycle

Medium · topological sort · cycle detection · dependency graph

Connector bindings declare ordering dependencies: an edge u -> v means v must not run in a wave earlier than one after u has succeeded. You have up to 50,000 bindings and 200,000 edges, and a misconfigured dependency may have created a cycle. Assign each binding the earliest wave number such that every predecessor sits in a strictly earlier wave, return the total wave count, and if a cycle exists return the bindings on one concrete cycle rather than a boolean. The implementation must be iterative: a dependency chain can be 50,000 deep.

Approach

1. Build an adjacency list and an indegree array, then run Kahn's algorithm from every indegree-zero node. Bindings with no predecessors get wave 0. O(V+E) time, O(V+E) space.
2. Relax waves as you decrement: when popping u, set wave[v] = max(wave[v], wave[u]+1) for each successor before enqueueing v at indegree zero. Because v is only enqueued after its last predecessor is popped, wave[v] is 1 + max over predecessors, which is the earliest legal wave. Total wave count is max(wave)+1, the longest path in nodes.
3. If Kahn emits fewer than V nodes, the residual subgraph is exactly the nodes lying on or downstream of a cycle. Every residual node still has an in-edge inside the residual, otherwise it would have been popped, so walking backwards along in-edges cannot dead-end and must revisit a node within |residual| steps; the segment between the two visits is a concrete cycle. O(V) with a visited-position map.
4. Use an explicit stack or a queue rather than recursion. Recursive DFS on a 50,000-node chain exceeds the default stack in most runtimes, and the failure looks like a crash rather than a cycle.
5. At this size, recompute the whole plan whenever an edge changes. Incremental topological order maintenance costs far more code than the O(V+E) rebuild it saves.

Follow-up

• Two bindings land in the same wave but hit the same client endpoint, whose rate budget is shared. How does the wave plan have to change, and what problem class does that turn into?
• One dependency holds only for overlapping watermark ranges rather than globally. Is the dependency graph still a DAG over bindings, and what is the right vertex if not?
• Report the critical path so an operator can see which chain sets the wave count, not just that it is five.

Fix a billing report that double-counts through a join

Medium · join fan-out · aggregation · reporting

A monthly report joins engagement to work_record (engagement_id, principal_id, work_date, minutes, is_billable, status) and to invoice_line (engagement_id, period_start, period_end, amount_minor, line_type, status) in a single FROM clause, then aggregates sum(minutes), count(DISTINCT principal_id) and sum(amount_minor) grouped by engagement. Finance reports the minutes are roughly triple the timesheets while the headcount column looks correct. Explain the arithmetic, then write the corrected query for one client and one period, returning zero rather than NULL for engagements with no work.

Approach

1. Name the arithmetic before touching SQL: joining two independent one-to-many children of the same parent produces the Cartesian product per engagement, |W| x |L| rows. sum(minutes) is therefore multiplied by the invoice-line count and sum(amount_minor) by the work-record count. Three lines per engagement — fees, expense, credit note — is exactly the factor of three finance reported.
2. Explain why the headcount column looked fine: count(DISTINCT principal_id) collapses the duplication, so it is correct by accident. That is precisely why the bug survived review, and it is the reason a column agreeing with expectations is not evidence that a join is at the right grain.
3. Pre-aggregate each branch to the join grain first: one subquery per fact table, grouped by engagement_id, each producing at most one row per engagement. Join those results, which are now one-to-one, so no multiplication is possible by construction rather than by care.
4. Drive the outer query from engagement with LEFT JOINs and wrap each aggregate in COALESCE, because sum over zero rows is NULL rather than 0. For counts, count the key column and not count(*), which returns 1 for a non-matching LEFT JOIN row.
5. Keep credit notes inside the amount sum deliberately: amount_minor is negative for line_type = 'credit_note', so filtering them out overstates what was billed. Exclude status = 'void' instead, and state that choice in the query as a comment because the next reader will assume the opposite.
6. Compare cost: the fan-out plan materialises |W| x |L| intermediate rows before aggregating, while pre-aggregation reads each table once under its own index and hash-joins two small results. The base-table I/O is identical; the intermediate work differs by orders of magnitude on a large engagement.

Follow-up

• The report now needs a per-principal breakdown alongside the per-engagement totals. Does your shape survive, or do you need a different grain entirely?
• How do you stop the next analyst reintroducing this — a view, a naming convention, or a test that fails on a fixture with two children?
• Why does sum(DISTINCT minutes) not fix it, and what does it actually compute?

Soft-delete environments without breaking uniqueness or audit history

Easy · soft deletes · partial indexes · audit history

environment holds one row per provisioned deployment: environment_id, client_id, engagement_id, env_name, region, status, destroyed_at TIMESTAMPTZ NULL. Decommissioned environments stay readable for seven years, and env_name must be unique per client among live environments only, because a client may reuse a retired name. Design the constraint set and the history table, write the query that lists a client's live environments, and say what your design does about connector_run rows whose foreign key points at a destroyed environment.

Approach

1. Keep the tombstone as a nullable timestamp rather than a boolean: destroyed_at carries when as well as whether, and enforce live-only uniqueness with a partial index, CREATE UNIQUE INDEX env_live_name ON environment (client_id, env_name) WHERE destroyed_at IS NULL, which indexes only the live subset and so stays small.
2. Get the NULL semantics right before you pick a spelling. In a B-tree unique index two NULLs compare distinct by default, so plain UNIQUE (client_id, env_name, destroyed_at) enforces nothing at all among live rows — every live row is unique on the third column purely by virtue of being NULL. PostgreSQL 15's UNIQUE NULLS NOT DISTINCT (client_id, env_name, destroyed_at) does satisfy the requirement: the NULL live rows now compare equal and collide with each other, while retired rows carry distinct destroyed_at timestamps and stay distinct, so a name can be retired and reused any number of times.
3. Prefer the partial index anyway, and be able to say why rather than asserting it. It indexes only the live subset — tens of rows per client — instead of seven years of tombstones; it needs no version floor; and it keeps destroyed_at out of the key, so it cannot be defeated by two rows retired at the same instant. That last case is reachable: now() is transaction-start time, so retiring a name, recreating it and retiring it again inside one transaction writes two identical destroyed_at values and raises a duplicate-key error under the constraint form. clock_timestamp() avoids it if you take that route.
4. Put history in a separate append-only table written by an AFTER INSERT OR UPDATE OR DELETE trigger holding environment_id, changed_at, changed_by, op and the before and after row images. PostgreSQL has no system-versioned tables, so this is a trigger or a logical-decoding consumer, not a DDL flag, and the soft delete arrives at that trigger as an UPDATE rather than a DELETE.
5. State the foreign-key consequence rather than leaving it implicit: a soft delete leaves every referencing connector_run row resolvable, which is exactly what the audit requirement wants, but it means no read is protected by the database. Give the predicate one home, a view or an RLS-backed view that every caller uses, instead of repeating destroyed_at IS NULL across hundreds of call sites.
6. Size the retention: seven years of history plus tombstoned rows in the hot table. Partition the history table by month so expiry is a DETACH PARTITION rather than a DELETE that has to be vacuumed, and decide separately whether a residency-driven erasure request can be honoured against an append-only store at all.

Worked solution (20 min)

1. Write the DDL with destroyed_at TIMESTAMPTZ NULL plus the partial unique index on (client_id, env_name) WHERE destroyed_at IS NULL.
2. Insert two live rows sharing (client_id, env_name) and confirm the second is rejected; set destroyed_at on the first and confirm the insert now succeeds.
3. Build the same table a second time under UNIQUE NULLS NOT DISTINCT (client_id, env_name, destroyed_at) and confirm it passes the same two tests, so you are choosing between two working designs rather than rejecting a broken one.
4. Add environment_history (history_id, environment_id, changed_at, changed_by, op, row_before JSONB, row_after JSONB) and the AFTER trigger that populates it.
5. Write the live-listing query: SELECT environment_id, env_name, region, status FROM environment WHERE client_id = $1 AND destroyed_at IS NULL ORDER BY env_name.

Expected result: A partial unique index covering only live rows, a separate append-only history table with its own partitioned retention, and a listing query whose predicate matches the index predicate exactly so the planner can use it.

Verify it yourself

• Run retire-and-recreate three times for the same name: three tombstoned rows and one live row coexist without violating the index.
• On the NULLS NOT DISTINCT variant, retire and recreate the same name twice inside one transaction and watch it fail on duplicate destroyed_at, since now() does not advance within a transaction; the partial index has no such failure mode.
• EXPLAIN the listing query and see the partial index; widen the predicate to destroyed_at IS NULL OR destroyed_at > now() and watch the index drop out, because that predicate no longer implies the index's.
• Soft-delete a row and confirm the history trigger fired; a DELETE-only trigger records nothing, which is the common way this audit trail ends up empty.

Follow-up

• What stops the next endpoint from forgetting the destroyed_at IS NULL predicate, and does that mechanism survive a developer writing raw SQL?
• The history table stores row images as JSONB and the base table gains a column. What do old rows now mean, and how do you read them six years later?
• A residency erasure request requires a genuine hard delete for one client. What breaks, and what do you keep?

What do you look for in a code review?

Medium

Approach

1. Work from the requirement backwards to the design.
2. Clarify what is being asked and what a complete answer contains.
3. State your assumptions explicitly before working the problem.

Follow-up

• What assumption would you test first?
• How would you know your answer was wrong?

Lots of agentic programming questions.

Medium

Approach

1. Say what you would check first and why it is the highest-information step.
2. Work from the requirement backwards to the design.
3. Clarify what is being asked and what a complete answer contains.

Follow-up

• How would you know your answer was wrong?
• What assumption would you test first?

Paginate an append-only credential audit log for client auditors

Easy · pagination · keyset-cursors · audit · indexing

credential_grant is append-only apart from revoked_at and revocation_reason, and holds grant_id, engagement_id, principal_id, target_system_id, scopes, issued_at, not_after, revoked_at and issued_by_request_id. A client's compliance tool exports every grant for one engagement over the contracted retention period. It runs nightly, is single-threaded, and restarts its export from scratch whenever a page fails. Rows are inserted continuously while it reads, and the largest engagement has roughly four million rows. Design the list endpoint: ordering, cursor, page-size contract, and what the caller does when a page fails or its cursor is rejected.

Approach

1. Order on a total order and page by keyset, not OFFSET. issued_at alone is not unique, so the sort key is (issued_at, grant_id): WHERE engagement_id = $1 AND (issued_at, grant_id) > ($2, $3) ORDER BY issued_at, grant_id LIMIT $n, served by a composite index on (engagement_id, issued_at, grant_id). The row-value comparison matches the index prefix, so each page costs an index descent plus the page. OFFSET does O(offset) server-side work per page because those rows are generated and discarded, which makes the whole four-million-row export quadratic in the number of pages.
2. Make the cursor opaque and self-describing: the sort key values, a fingerprint of the filter and ordering, and a cursor format version. If the caller changes engagement_id or the sort direction mid-scan, the fingerprint mismatch is a 400 with its own code rather than a quietly wrong page. Opaque also means the caller cannot hand-assemble one, so you can change the encoding later without a version bump.
3. Decide what 'every grant' means while rows are arriving. Pin an upper bound at page one: return an as_of timestamp and filter every subsequent page on issued_at < as_of, so the export is a consistent prefix and the next night's run resumes from the previous as_of. Ascending order with an upper bound makes concurrent inserts invisible to a scan in progress; a descending scan without one has every insert shift the window under it.
4. Say out loud that mutable columns break the snapshot. revoked_at can be set after a page was served, so an export can carry a row whose revocation the client never sees. Put it in the contract: the export is grants issued within a window with revocation state as of as_of, and a client that needs final state re-reads by grant_id. Do not let a keyset scan imply a consistent snapshot of fields that still change.
5. Cap page size server-side with a documented default, and return the next cursor as a link rather than a page number. Do not return a total count for a four-million-row filter; an exact count is a full index scan on every call. If the caller wants progress, it already has the as_of bound and the timestamp in its last cursor.
6. Define failure handling. A GET with a cursor is idempotent, so a failed page is safe to retry verbatim. A rejected cursor is terminal and the caller resumes from the last cursor it durably consumed, which is why the contract must tell it to persist the cursor rather than a page number.

Worked solution (25 min)

1. Write the page query with the row-value comparison and the index it requires, then state which index columns the predicate and the ORDER BY each consume.
2. Compute the work for page 4,000 at 1,000 rows per page under OFFSET and under keyset, and write both numbers down.
3. List the cursor payload's fields and name the specific failure each one prevents.
4. Walk one interleaving on paper: page 3 served, two hundred rows inserted with issued_at below as_of, page 4 requested. Say exactly which rows the caller sees.
5. Write the contract sentence that covers revoked_at changing after its row was exported.

Expected result: An ascending keyset endpoint over (issued_at, grant_id) backed by a composite index on (engagement_id, issued_at, grant_id), an opaque versioned cursor carrying the sort key and a filter fingerprint, an as_of upper bound pinned at page one, a server-capped page size, no total count, and a documented rule that revocation state is as-of rather than final.

Verify it yourself

• Page latency is flat from page 1 to page 4,000 instead of climbing.
• Two grants sharing an issued_at to the microsecond each appear exactly once across the whole export.
• Re-requesting the same cursor returns the same page, and a cursor built under a different engagement_id is rejected with its own error code rather than served.

Follow-up

• The compliance tool is down for three days and does not want to re-read what it already has. What does it store, and what does the endpoint need for that to work?
• The caller supplies engagement_id itself. What stops this endpoint returning another client's grants?

Environment list page slows as the estate grows

Easy · n+1 query · latency · batching

An estate console lists environments for an operator: environment_id, client name, isolation_tier, the desired release version string, the reported release version string, and the engagement's current status. A 10-row filter returns in 25 ms; the unfiltered 200-row page takes 640 ms. No single statement exceeds 0.5 ms, and the database records 553 statements for that request. Rows where reported_release_id is NULL are measurably cheaper than the rest. Diagnose the cause and give the fix, stating the statement count you expect afterwards.

Approach

1. Get the shape before theorising about plans. Divide the per-request statement delta in pg_stat_statements by the rows returned: 553 statements for 200 rows, none over 0.5 ms, is round trips rather than work, and rules out a bad plan immediately.
2. Account for 553 exactly, because the arithmetic is the diagnosis and not a detail. One driver query, one desired-release lookup per row, one current-engagement lookup per row, and one reported-release lookup only for the rows where reported_release_id is non-null: 1 + 200 + 200 + 152. The 152 is why client-managed rows are cheaper, and if the sum did not land on the observed number something else is firing too.
3. Sanity-check the cost model: 553 round trips at roughly 1 ms each is about 0.55 s on top of a 25 ms baseline, which accounts for the 640 ms. A hypothesis that does not reproduce the measured number is not finished.
4. Batch it. Collect the ids from the driver query, then fetch releases with WHERE release_id = ANY($1) and the open engagement versions with WHERE engagement_id = ANY($1) AND effective_to IS NULL, and assemble in memory. Three statements per page, independent of page size.
5. Check index usage before declaring victory. The engagement fetch must use the partial unique index on (engagement_id) WHERE effective_to IS NULL; without it, ANY($1) scans every amendment version of every engagement and the fix regresses harder than the bug did.
6. Pin the behaviour with a test asserting a constant statement count per request, since no individual query here is slow enough to ever trip a slow-query alarm.

Follow-up

• The operator wants to sort the page by reported release version. Where does that sort have to happen now, and what does it cost?
• Page size goes to 1,000. Which breaks first, the driver query or the ANY($1) parameter list?
• reported_release_id NULL means unknown, not converged. How should the list render that, and what does the count at the top of the page then mean?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

• Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
• Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
• Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

• Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
• Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
• Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

• Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
• Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
• Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

• For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
• Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
• Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
• Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

• Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
• Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
• Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
• Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

• Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
• Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
• Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

• Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
• Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
• Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

Review and mentoring: what you let through

Counting review comments or mentees proves nothing. The useful version is a specific change you approved with a reservation you stated, or one you blocked and the delay that cost. Say which standard you were holding and why it was worth the friction. A mentoring story needs the thing the other person can now do without you.

Describe a time where you had to give pushback for a project deadline.

Medium · behavioural and engineering judgement

Approach

1. Close with what you would do differently, concretely.
2. Give the blast radius: what could have broken, and what you measured.
3. Name the disagreement and how you resolved it with evidence.

Follow-up

• What did you decide not to do, and why?
• What would you do differently if you ran that again?

Why are you interested in front-end or back-end development?

Medium · behavioural and engineering judgement

Approach

1. Name the disagreement and how you resolved it with evidence.
2. Close with what you would do differently, concretely.
3. Pick a story where you made the decision, not one where you watched it.

Follow-up

• How did you know your change caused the improvement?
• What would you do differently if you ran that again?

What does being a "crafter" mean to you?

Medium · behavioural and engineering judgement

Approach

1. Close with what you would do differently, concretely.
2. Give the blast radius: what could have broken, and what you measured.
3. Name the disagreement and how you resolved it with evidence.

Follow-up

• What would you do differently if you ran that again?
• How did you know your change caused the improvement?

Give a 5-minute presentation where you teach something.

Medium · behavioural and engineering judgement

Approach

1. Close with what you would do differently, concretely.
2. Pick a story where you made the decision, not one where you watched it.
3. State the situation in two sentences and spend the rest on the reasoning.

Follow-up

• What would you do differently if you ran that again?
• How did you know your change caused the improvement?