cswork / libraryOpen practice workspace

Interview guides · Basis Technologies

Basis Technologies Software Engineer Interview Guide

Basis Technologies software engineer prep: algorithm drills where every complexity claim has to name the assumption under it.

The 60-second brief

As a Software Engineer at Basis Technologies, you play a crucial role in shaping the future of technology solutions that drive efficiency and innovation in business operations. This position is integral to developing software that enhances the scalability and performance of the company’s products, directly impacting users and stakeholders. You will work on complex projects that require creative problem-solving skills and technical expertise, contributing to solutions that address real-world challenges in various industries.

If the loop includes an asynchronous take-home, treat it as a code review of you rather than as a puzzle: structure, what you chose to test, and what you wrote down about the constraint you were working under. Hold the stated time box and say what you would have done with more of it.

Focus areas

  • Make every write idempotent under client retries
  • Evolve APIs without breaking pinned SDK clients
  • Scope every query and cache key by tenant

Overview

Your contributions will not only focus on coding but also on understanding user needs, collaborating with cross-functional teams, and ensuring that the software developed aligns with the strategic goals of the company. You will engage in building and maintaining systems that support the company’s vision, making your role both impactful and rewarding.

At Basis Technologies, you can expect to work on cutting-edge technologies, with opportunities to innovate and refine existing processes. The dynamic and collaborative environment encourages continuous learning and growth, making it an exciting opportunity for aspiring and experienced software engineers alike.

The Basis Technologies interview process

Basis Technologies candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Phone Screen

The person on this call usually cannot evaluate your code and does not need to. They write a short paragraph, and that paragraph is what a hiring manager skims when deciding who to put on your loop. So the test is not whether your work was hard, it is whether a non-engineer can repeat it correctly. Name systems by what they did rather than by their internal codename, give each project a shape (what was breaking, what you changed, what happened after), and keep the whole walkthrough near ninety seconds. Depth that cannot survive a paraphrase reads as vagueness.

What to demonstrate

  • Whether a non-engineer can restate your projects without distorting them, since their paraphrase is what travels to the hiring manager, not your sentences
  • Whether each project has a shape rather than a stack list: the failure or constraint, the change you made, the result and how it was measured
  • Whether you can say what was yours inside a team project without either inflating it or disappearing into the plural

How to prepare

  • Rewrite each headline project as two sentences with no internal system names and no acronyms outside your company, then say them to someone outside engineering and have them repeat them back. Fix whatever came back wrong
  • Attach one measured number to each project: the baseline, the change, and the window it was measured over. Where nothing was ever measured, say that plainly rather than reaching for a plausible percentage
  • Time the background walkthrough against a clock. If it runs past two minutes, compress the earliest role to a single clause and spend the recovered time on the most recent one

Technical Interviews

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

  • In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
  • In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
  • Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

  • Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
  • Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
  • For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Where candidates lose points

Serialising a tenant's writes through select ... for update on a single counter row

It is the first change that makes a counter correct, and it caps that tenant's write throughput at roughly one divided by the lock hold time. A transaction that takes the lock, makes a network call and then commits holds it for the entire round trip: at 2 ms that is about 500 writes per second for the whole tenant, and the largest tenants are exactly the ones that exceed it. The damage then spreads, because every waiter holds a database connection while it queues, so one hot tenant drains the shared pool and the symptom presents as a site-wide latency incident rather than as a lock problem. The repairs are to shrink the critical section to a single statement, to shard the counter into per-(tenant, hour) or per-(tenant, bucket) rows and sum on read, or to batch in memory and flush periodically while accepting the bounded loss that batching implies.

Holding money in a floating-point type, or rounding it more than once

Binary floating point cannot represent 0.01 or 0.1 exactly, so sums drift and two code paths that should agree disagree by cents nobody can trace back. The fix is integer minor units or an exact decimal type end to end, with sub-cent rates expressed as scaled integers such as micro-units, because a per-request price genuinely is smaller than a cent. The second half of the trap is rounding position: rounding each line and then summing gives a different total from summing and rounding once, and half-up and half-even diverge systematically across many lines, so rounding must happen at one named place and every downstream reader must carry the rounded value rather than recompute it from quantity and rate.

Treating a network call as though it were a local function call

A remote call can be slow, fail, or return after you stopped waiting, so name the timeout, the retry policy, and what the caller sees while the dependency is down. A call with no timeout turns one slow dependency into an exhausted thread or connection pool in every service upstream of it.

Choosing a schema before the access patterns are known

Write the queries first, with their filters, sort orders, cardinalities and which ones sit on the latency-critical path, then design tables and indexes to serve them. An index nothing queries still costs write throughput and storage, and a hot query with no supporting index becomes a full scan that only hurts once the table is big.

Questions & practice

Write a function to reverse a string.

Medium · data structures and algorithms

Approach

  1. State the target complexity and say which constraint rules the naive version out.
  2. Restate the input: its shape, its size, and what is guaranteed about it.
  3. Walk one small example through your approach before writing the whole thing.

Follow-up

  • How does this change if the input no longer fits in memory?
  • What is the worst case, and how likely is it on real data?

Explain the difference between a stack and a queue.

Medium · data structures and algorithms

Approach

  1. Name the brute-force solution and its complexity before improving on it.
  2. Walk one small example through your approach before writing the whole thing.
  3. State the target complexity and say which constraint rules the naive version out.

Follow-up

  • How does this change if the input no longer fits in memory?
  • What is the worst case, and how likely is it on real data?

How would you approach optimizing a slow-performing function?

Medium · data structures and algorithms

Approach

  1. Choose the data structure from the access pattern, not from familiarity.
  2. Walk one small example through your approach before writing the whole thing.
  3. Restate the input: its shape, its size, and what is guaranteed about it.

Follow-up

  • How does this change if the input no longer fits in memory?
  • What is the worst case, and how likely is it on real data?

Can you solve a problem using dynamic programming?

Medium · data structures and algorithms

Approach

  1. State the target complexity and say which constraint rules the naive version out.
  2. Choose the data structure from the access pattern, not from familiarity.
  3. Restate the input: its shape, its size, and what is guaranteed about it.

Follow-up

  • Which test case would catch an off-by-one here?
  • How does this change if the input no longer fits in memory?

Find peak concurrent sandbox usage from run intervals

Medium · sweep-line · intervals · concurrency-caps · null-semantics

Given up to 5 million job_run rows for one tenant over one day, with run_id, started_at, finished_at, status and wall_clock_limit_seconds, report the maximum number of sandboxes running at once, the earliest instant that maximum is reached, and the first run_id that would breach a per-tenant cap of C. started_at is null while a run is queued; finished_at is null both for runs still executing and for runs in status lost. Treat a run as occupying [started_at, finished_at). Give the complexity and state how you handle each null.

Approach

  1. Turn each run into two sweep events, (started_at, +1) and (end, -1), then sort the 2n events by timestamp with -1 ordered before +1 at equal timestamps. That tie-break is what makes the interval half-open, so a run finishing at 10:00:00 and one starting at 10:00:00 never overlap.
  2. Decide each null out loud before sweeping, because each choice moves the answer. A null started_at means queued and contributes nothing. A null finished_at with status running or leased is clipped to the window end. Status lost has no observed end at all, so clip it at started_at + wall_clock_limit_seconds on the grounds that the supervisor owns the timeout, and record that you did. The table's check (finished_at is null or started_at is not null) guarantees you never see an end without a start.
  3. Sweep once, maintaining a running counter, the maximum, and the timestamp at which the maximum was first attained (update peak_at only on a strict increase, or you will report the last such instant instead of the earliest). Capture the first run_id whose +1 takes the counter to C+1 during the same sweep rather than in a second pass.
  4. Complexity: O(n log n) dominated by the sort, O(n) space. If rows already arrive ordered by started_at, a min-heap of end times gives O(n log k) time and O(k) space with k the peak concurrency, which is the better shape when the rows come from an index scan on (tenant_id, started_at).
  5. If second resolution is acceptable, counting-sort the endpoints into an 86,400-slot delta array and prefix-sum it: O(n + T) time and O(T) space, which beats the comparison sort at 5 million rows. It answers only at second granularity, so state which resolution the cap is defined in.

Worked solution (20 min)

  1. Write the null policy as three lines of prose first, one per case, and keep them beside the output.
  2. Emit 2n endpoint tuples (timestamp, delta, run_id) and sort on the key (timestamp, delta) so -1 precedes +1.
  3. Sweep, tracking cur, peak, peak_at updated only on a strict increase, and the first run_id whose +1 takes cur to C+1.
  4. Build a fixture with two runs where one ends exactly when the next starts, three genuinely overlapping runs, one run with a null finished_at and status running, and one with status lost and a 300-second wall_clock_limit_seconds.
  5. Re-run with every timestamp shifted by a constant and confirm the peak is unchanged while peak_at shifts by the same constant.

Expected result: Peak is 3, from the overlapping trio. The exact-handoff pair yields a peak of 1, not 2. peak_at is the start instant of the third overlapping run. The lost run occupies exactly [started_at, started_at + 300s) under the stated policy.

Verify it yourself

  • Setting peak_at on >= rather than > moves the reported instant to the last time the peak was attained. Flip it and watch the fixture change.
  • Sorting by timestamp alone, without the delta tie-break, raises the reported peak on the handoff fixture.
  • The deltas over the whole sweep sum to zero once every open-ended run has been clipped.

Follow-up

  • Now report peak concurrency per tenant for 10,000 tenants from one globally sorted stream. What changes about memory and about the sort?
  • The cap has to be enforced at dispatch rather than reported afterwards. What does the admission check look like, and where does it race?
  • How would you answer 'peak concurrency within any 5-minute window' without re-sorting?

Migrate a live partitioned event table without blocking ingest

Hard · online migration · ddl locks · expand and contract · partitioning

usage_event is range-partitioned daily on ingested_at, holds roughly 250M rows per day across 400 live partitions, and is written at 10-40k rows/second. Two changes are required: quantity must move from double precision to numeric(20,6), and a new environment column must become NOT NULL with a default of 'production'. Ingest cannot stop. Give the ordered plan, naming for each step the lock it takes, what that lock blocks, and roughly how long it is held. Identify the one step that cannot be rolled back cleanly once traffic depends on it.

Approach

  1. Classify the two changes before planning anything. Adding a column with a non-volatile default has been metadata-only since PostgreSQL 11, so it is cheap. Changing double precision to numeric is not binary-coercible, so alter column ... type rewrites every partition under ACCESS EXCLUSIVE and rebuilds its indexes; on this volume that is hours of blocked ingest and is simply not an option, which is why the plan is expand-and-contract rather than one statement.
  2. Expand: add quantity_numeric numeric(20,6) and environment with its default on the parent. Both are catalogue-only but both take a brief ACCESS EXCLUSIVE that cascades to partitions, so run each with lock_timeout set to a second or two and retry on failure. A queued ACCESS EXCLUSIVE request blocks every reader behind it, which is how a metadata-only change turns into an outage.
  3. Dual-write: deploy producer code that populates both columns on every insert, and leave it running before anything reads the new column. This is the step that cannot be reverted cleanly. Once readers depend on quantity_numeric, reverting the writer leaves rows with a null there, and the gap is only discoverable by re-reading the old column, which the readers have stopped doing.
  4. Backfill older partitions in batches keyed on the primary key, oldest first, committing every few thousand rows with a pause between batches, and skipping the partition still receiving writes until it rotates. Each batch is an ordinary UPDATE taking row locks only. The cost is bloat and WAL rather than blocking, so watch dead tuples and let autovacuum keep pace instead of wrapping 400 partitions in one transaction.
  5. Make NOT NULL cheap with the three-step form: add constraint ... check (environment is not null) not valid (brief ACCESS EXCLUSIVE, no scan), then validate constraint (SHARE UPDATE EXCLUSIVE, scans while reads and writes continue), then set not null, which from PostgreSQL 12 uses the validated check and skips its own full scan. Do this per partition, then on the parent.
  6. Switch and contract: move reads to the new column behind a flag, verify over a full period that both columns agree on freshly written rows, drop the old column (metadata-only), and only then remove the dual-write. Any index on the new column goes on with CREATE INDEX CONCURRENTLY per partition, since CIC is not supported on a partitioned parent: create the parent index with ONLY, build each child concurrently, then ALTER INDEX ... ATTACH PARTITION until the parent index becomes valid.

Follow-up

  • A CREATE INDEX CONCURRENTLY fails halfway through the partition list. What state is the table in, how do you detect it, and what do you run?
  • The producer computes quantity itself. What happens to a request already in flight when the dual-write deploy lands, and does it matter?
  • Give two queries that prove the backfill is complete: one cheap enough to run every minute, one authoritative.

Explain why the metering dashboard scans every daily partition

Medium · indexing · partition pruning · sargability · tenant skew

usage_event is range-partitioned daily on ingested_at and holds tenant_id, workspace_id, environment, sku, quantity numeric(20,6), occurred_at and ingested_at. The only relevant index is on (occurred_at). A dashboard runs select sku, sum(quantity) from usage_event where tenant_id = $1 and date_trunc('hour', occurred_at) >= $2 and environment = 'production' group by sku, and EXPLAIN shows a sequential scan of every partition. Give each distinct reason, rewrite the predicate so an index can serve it, propose the index, and state the write cost its column order adds.

Approach

  1. Separate the three causes rather than blaming one. First, date_trunc('hour', occurred_at) wraps the column, so the predicate is not sargable against a btree on the bare column. Second, pruning keys off ingested_at while the query constrains occurred_at, so no partition can be excluded. Third, even made sargable, (occurred_at) is not tenant-leading, so for one tenant among thousands the scan reads the whole time range and discards nearly all of it.
  2. Rewrite the bound carefully, because the obvious rewrite is only conditionally equivalent. date_trunc('hour', x) >= $2 equals x >= $2 only when $2 is already hour-aligned; for an arbitrary $2 it means x >= date_trunc('hour', $2) + interval '1 hour'. Normalise the parameter in the caller and leave the column bare.
  3. Restore pruning with a second, redundant predicate on the partition key: ingested_at >= $2 - interval '<late-data horizon>'. State both sides of it. It prunes to a handful of partitions, and it silently omits any event whose ingest lagged past that horizon, which is precisely what a producer replay produces. Either document the horizon as a stated bound, or partition on occurred_at and move the problem into the dedup window instead.
  4. Propose (tenant_id, occurred_at) include (sku, quantity) per partition. A partial index where environment = 'production' mostly saves size rather than selectivity, since production dominates the three environments; take it if non-production is a meaningful share and skip it otherwise.
  5. Price the write path honestly. At roughly 250M rows/day each extra index is another insert plus WAL per row, and a tenant-leading key scatters inserts across one hot leaf per active tenant instead of appending to a single rightmost leaf, so page dirtying and random I/O both rise. An INCLUDE payload widens every leaf entry and enlarges the index accordingly.
  6. Add the index-only-scan caveat before someone reports it as a regression: on a freshly appended table the visibility map is not yet set for recent pages, so the INCLUDE columns still cost heap fetches until autovacuum has been through, and the newest hour is exactly the data the dashboard reads.

Worked solution (30 min)

  1. Build 30 daily partitions with skewed tenants, one holding about 40% of the rows, then ANALYZE.
  2. Run explain (analyze, buffers) on the original query and record how many partitions were scanned and the rows removed by filter.
  3. Apply the rewritten predicate and the index, re-run, and confirm the plan lists only the partitions inside the ingested_at bound.
  4. Re-run with $2 set to a non-hour-aligned timestamp and confirm the rewritten and original predicates return identical rows.
  5. Insert an event with ingested_at six hours past occurred_at and check whether the pruning predicate excludes it.

Expected result: Three causes named separately: a non-sargable date_trunc, pruning on ingested_at against a predicate on occurred_at, and a non-tenant-leading index. A rewrite to occurred_at >= <hour-normalised $2> plus a redundant ingested_at >= $2 - horizon, an index on (tenant_id, occurred_at) including sku and quantity, a per-row write cost stated, and the late-data omission the pruning predicate introduces stated as a known bound.

Verify it yourself

  • The plan lists only partitions inside the ingested_at bound.
  • Rows removed by filter on the index scan is small relative to rows returned.
  • The rewritten and original predicates agree for both hour-aligned and non-aligned bounds.
  • A deliberately late-ingested event is visible or omitted exactly as the stated horizon predicts.

Follow-up

  • CREATE INDEX CONCURRENTLY is not supported on a partitioned parent. Give the sequence that gets this index onto 400 existing partitions without blocking ingest.
  • One tenant holds 200 times the median row count and the dashboard still times out for them with the index in place. What changes?
  • Should this read hit usage_rollup_hourly instead? State what that costs in freshness and what the watermark lets you promise.

What considerations would you take into account for system security?

Medium · system design and architecture

Approach

  1. Name the failure you are designing for, then the recovery path.
  2. Choose a partition key and say what query it makes expensive.
  3. State the consistency you need, and where you are willing to be stale.

Follow-up

  • What would you drop to keep the system up under load?
  • What breaks first when traffic grows ten times?

How would you handle data storage and retrieval in your application?

Medium · system design and architecture

Approach

  1. Name the read and write paths separately; they rarely have the same bottleneck.
  2. Choose a partition key and say what query it makes expensive.
  3. Name the failure you are designing for, then the recovery path.

Follow-up

  • What would you drop to keep the system up under load?
  • How does this behave when that dependency is down for an hour?

How do you ensure code quality and maintainability in your projects?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Clarify what is being asked and what a complete answer contains.
  3. Say what you would check first and why it is the highest-information step.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Sealing a billing period against late-arriving usage

Medium · idempotency · immutable ledger · rounding · conditional writes

billing seals a tenant's period once the metering watermark passes the period end, prices the sealed rollups against the plan (included allowance, tier boundaries, negotiated discount) and writes invoice_line_item rows: quantity numeric(28,6), unit_price_micros bigint, amount_minor bigint, currency char(3), source_rollup_watermark timestamptz. The job is re-run after failures and two workers may attempt the same tenant. Events legitimately arrive with occurred_at inside the period and ingested_at after it. Specify the seal transition, the idempotency key for every write including the payment-processor call, the rounding position and mode, and the fate of a post-seal event.

Approach

  1. Make sealing a conditional write rather than a read followed by a write: UPDATE usage_rollup_hourly SET status = 'sealed', sealed_at = now() WHERE tenant_id = $1 AND hour_start >= $2 AND hour_start < $3 AND status = 'open' RETURNING rollup_id. Two concurrent sealers cannot both win because the loser's predicate no longer matches, and the loser learns it lost from an empty result rather than from a lock timeout.
  2. Gate the seal on the watermark, not on the clock - the period closes when the fold is trustworthy past period_end - but cap the wait, because an unsealed period blocks the whole billing run. Seal anyway after a stated maximum (six hours past period end is a reasonable default) and let whatever arrives afterwards become an adjustment. The trade-off is named in one line: adjustments are cheap, a missed billing cycle is not.
  3. Make every write converge under re-run. Line items are idempotent on (invoice_id, sku, rate_tier) with ON CONFLICT DO UPDATE permitted only while the invoice is draft; the payment-processor call carries (tenant_id, billing_period_start) as its idempotency key, because a timeout there is an unknown outcome, not a failed one, and a plain retry is how a customer gets charged twice.
  4. Keep the arithmetic exact and round exactly once. quantity stays numeric, the rate is an integer unit_price_micros in millionths of a minor unit because per-request prices are genuinely below a cent, and amount_minor = round_half_even(quantity x unit_price_micros / 1,000,000) is computed once per line and stored. Tiering walks the boundaries in order emitting one line per (sku, rate_tier) with tier 0 as the included allowance at price zero; the invoice total is the sum of stored amount_minor values and is never recomputed downstream from quantity and rate.
  5. Handle the post-seal event as a forward-only correction. The sealed rollup is frozen, so the difference between the sealed value and the restated value becomes a new invoice_line_item with kind = 'adjustment' and voided_by_line_id pointing at the line it reverses, carrying its own source_rollup_watermark. Nothing is edited in place, because the original line is the only evidence of what the customer was actually charged and is precisely what a dispute, a refund or an audit asks to see.

Worked solution (35 min)

  1. Write the seal as a single conditional UPDATE ... WHERE status = 'open' RETURNING and state exactly what the losing worker sees.
  2. Price one line by hand: quantity 4,318,904.250000 at unit_price_micros 1200 gives 5,182.6851 minor units and 5,183 after a single half-even rounding. Then do quantity 2,500,000.000000 at unit_price_micros 1, which lands exactly on 2.5 and rounds to 2 under half-even but 3 under half-up.
  3. Take four hundred synthetic lines with fractional minor units and compute the total two ways - sum of per-line rounded amounts, and a single rounding of the summed exact amounts - then record the gap.
  4. Trace one event with occurred_at inside the period and ingested_at two days after the seal all the way to the row that eventually reflects it.

Expected result: Seal as a conditional UPDATE from status = 'open' gated on the metering watermark with a bounded maximum wait; line writes idempotent on (invoice_id, sku, rate_tier) and the processor call idempotent on (tenant_id, billing_period_start); amount_minor rounded once per line with half-even; late data becoming an adjustment line that points at the line it reverses and records its own watermark, with no in-place edit anywhere.

Verify it yourself

  • Re-running the entire billing job on an already-invoiced period must produce identical line rows and must not call the payment processor a second time. If it charges again, the idempotency key was on the retry rather than on the business fact.
  • Over four hundred lines with uniformly distributed fractional parts the two totals should differ by a handful of minor units - standard deviation around sqrt(400/12), about 5.8, with a worst case of 200. If your two totals match exactly, the test data has no fractional parts and has not exercised the trap.
  • Every line carries a source_rollup_watermark, and two lines for the same SKU with different watermarks should explain their difference without opening any other table.

Follow-up

  • Two workers start the same tenant's seal a millisecond apart and the winner crashes after sealing but before writing any line. What does the second worker observe, and is the resulting invoice correct?
  • Show the divergence between rounding each line and rounding the total once over four hundred lines, and say which direction it goes.
  • A customer disputes a charge from two quarters ago. Which rows answer it, and which single design decision made that answer possible?

One tenant's counter writes stall the whole connection pool

Medium · lock contention · connection pool · multi-tenancy

A change that made a per-tenant usage counter correct now produces site-wide latency whenever one large tenant writes: unrelated endpoints time out waiting for a connection while database CPU stays low and no statement is slow. The change wraps the counter update in a transaction that takes SELECT ... FOR UPDATE on one row, calls an external pricing service, then updates and commits. Give an ordered checklist, the arithmetic that bounds that tenant's write rate, and three repairs with the cost each one accepts.

Approach

  1. Separate waiting from working. Low database CPU alongside high application latency points at a queue, so instrument connection-acquisition wait separately from query execution time; that queue forms in the application and is invisible in database metrics, which is why the database looks healthy throughout.
  2. Confirm the lock rather than assuming it: sample waiting sessions and group by wait event, relation and tuple. Contention concentrated on one tuple belonging to one tenant is the signature; a deadlock would instead show the database aborting transactions after its detection timeout, which is not happening here.
  3. Do the arithmetic out loud. Throughput on a serialised row is one divided by the lock hold time, and the hold spans the external call, so a 20 ms pricing call caps that tenant near 50 writes per second no matter how many pods run. Every waiter also holds a pooled connection while it queues, so the shared pool drains and unrelated tenants fail at acquisition.
  4. Repair one: shrink the critical section to a single statement with the price resolved before the transaction opens. Cost is a stale price for the duration of one request and a second round trip; benefit is a hold time measured in the database's own execution time.
  5. Repairs two and three change where the contention lives rather than how long it is held. Sharding the counter into per-(tenant, bucket) rows and summing on read multiplies write throughput by the shard count, at the cost of an aggregate on every read and a shard count you must size against the largest tenant rather than the median. Accumulating in memory and flushing periodically removes the per-write round trip entirely, paid for with a bounded loss window on crash, which is acceptable for a rate limiter and not for a billing counter.
  6. Contain independently of which repair wins: a separate pool or per-tenant concurrency cap for this write class, a statement timeout low enough that a pathological query dies before it accumulates waiters, and an idle-in-transaction timeout so a stuck client cannot pin a connection and its locks.

Follow-up

  • What would a genuine deadlock look like here, which two code paths would produce one, and how does the database's response differ from what you observed?
  • If a transaction-pooling proxy sits in front of the database, which of your three repairs changes behaviour, and what stops working that would have worked on a direct connection?
  • The counter also enforces a quota. Why is SELECT the count and then INSERT still wrong after you have fixed the contention?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

  • Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
  • Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
  • Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

  • Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
  • Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
  • Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

  • Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
  • Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
  • Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

  • For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
  • Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
  • Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
  • Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

  • Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
  • Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
  • Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
  • Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

  • Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
  • Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
  • Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

  • Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
  • Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
  • Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

Outage answers are graded on instruments

Nobody is scoring your stamina at three in the morning. What carries weight is which signal told you something was wrong, what you measured before touching anything, what you rolled back versus what you fixed forward, and why you picked one. 'We restarted it and it went away' is a story about not knowing.

What motivates you to succeed as a software engineer?

Medium · behavioural and engineering judgement

Approach

  1. Close with what you would do differently, concretely.
  2. Give the blast radius: what could have broken, and what you measured.
  3. State the situation in two sentences and spend the rest on the reasoning.

Follow-up

  • What did you decide not to do, and why?
  • What would you do differently if you ran that again?

How do you handle tight deadlines and pressure?

Medium · behavioural and engineering judgement

Approach

  1. Pick a story where you made the decision, not one where you watched it.
  2. Give the blast radius: what could have broken, and what you measured.
  3. Name the disagreement and how you resolved it with evidence.

Follow-up

  • What did you decide not to do, and why?
  • How did you know your change caused the improvement?

Argue against failing open when the control plane is unreachable

Hard · revocation · cache staleness · fail-open vs fail-closed · influence

The gateway caches credential-to-context decisions with a sixty-second TTL. A design proposal says that when control-plane reads fail, pods should keep serving from expired entries indefinitely so a control-plane outage never becomes a product outage. You believe that converts every revocation into an unbounded one. Describe a design you argued against while it was still a live proposal: what you measured or modelled to make the case, what you conceded, who decided, and what happened afterwards. Say what would have changed your mind before the decision, not after it.

Approach

  1. Reframe it from a values argument into a bounded-staleness argument. Both sides already accept the cache; the disagreement is only about the ceiling on how long a revoked credential keeps authorising. Put a number on the table — serve stale for up to fifteen minutes, then fail closed — and make the other side argue against a number rather than against a principle.
  2. Bring arithmetic rather than adjectives: the rate of revocations with revoked_reason in ('suspected_leak','auth_version_bump'), the observed distribution of control-plane unavailability, and the product of the two, which is expected requests served by revoked credentials per outage-hour. At 30k requests/second the unbounded version is not a subtle exposure and the number says so.
  3. Concede the strong half of the opposing case first, because that is what buys you the room: failing closed turns one service's outage into a total outage across three regions, and a control plane doing tens of writes per second is not engineered to the gateway's availability target. A proposal you have not steelmanned reads as reflex.
  4. Propose the asymmetry that usually resolves this: stale entitlements cost bounded money (a quota fifteen minutes out of date over-serves by a computable amount), while a stale revocation costs unbounded access. Split the cached decision by what it authorises, give the two halves different staleness ceilings, and let the entitlement half fail open while the revocation half fails closed.
  5. State the propagation dependency plainly, since it is the part that is missed: validity is also derived from the principal's auth_version, so password reset and sign-out-everywhere flow through this same cache. A design that bounds staleness for explicit revocation and not for auth_version bumps has only fixed half of it.
  6. Say who decided, and what you did afterwards in either outcome: write the decision down with its number and a review date, and instrument the exposure you were worried about so the next round of the argument is settled by data instead of by seniority.

Follow-up

  • Publish-subscribe invalidation is lossy under a partition, and a TTL is the only hard bound. What TTL do you pick, and what does it cost you at 30k requests/second?
  • The key was revoked because it was found in a public repository. Does your answer change, and where does that urgency live in the design?
  • You lost the argument and six weeks later the failure you predicted happens. What do you say in the review, and what do you not say?