Interview guides · Attentive.ai
Attentive.ai Software Engineer Interview Questions & Guide 2026
Software engineer prep for Attentive.ai: design exercises weighted toward the constraints of B2B software and infrastructure work.
The 60-second brief
At Attentive AI, the Software Engineer role is at the intersection of high-scale data processing and sophisticated automation. You are not just writing code; you are building the infrastructure behind Attentive AI's AI solutions, which translate complex visual data into actionable business intelligence. Whether you are working on Backend systems, Fullstack development, or specialized BIM (Building Information Modeling) engineering, your work directly affects the efficiency and precision of Attentive AI's automated platforms.
Browser-facing seats are not covered by algorithm practice. Scope in state ownership, what the page does on a slow or failed request, and how you would diagnose something that renders correctly but feels slow.
Focus areas
- Make every write idempotent under client retries
- Evolve APIs without breaking pinned SDK clients
- Build at-least-once pipelines with explicit deduplication horizons
Overview
This role requires a unique blend of technical rigor and product-minded thinking. You will be expected to tackle challenging problems involving large-scale data, infrastructure stability, and responsive user interfaces. As a Software Engineer here, you will contribute to critical projects that demand clean, scalable code and a deep understanding of the underlying tech stack. Candidates report that interviewers look for engineers who can move fast, maintain high standards, and contribute to a culture of constant learning.
Where to focus your preparation
Treat the sections below as preparation areas and confirm the format with your recruiter.
Preparation focus
No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.
What to demonstrate
- Breadth across SQL, experimentation and product reasoning
- Ability to state assumptions before choosing a method
How to prepare
- Drill the practice exercises below and time yourself
- Prepare three quantified stories about decisions you drove
Where candidates lose points
One shared connection pool for every tenant and every query class
A single tenant with a large table and a missing index can occupy every connection with slow queries, and every other tenant then waits in connection acquisition -- a queue invisible in database metrics, because the database itself looks healthy while the application starves. Containment is bulkheads: separate pools or per-tenant concurrency caps for interactive requests, background jobs and exports, a statement timeout low enough that a pathological query dies before it accumulates, and an idle-in-transaction timeout so a stuck client cannot pin a connection and its locks indefinitely. One caveat worth knowing in advance: if a transaction-pooling proxy sits in front of the database, session-scoped behaviour changes, so session-level advisory locks and settings applied outside a transaction do not survive the way they do on a direct connection.
Holding money in a floating-point type, or rounding it more than once
Binary floating point cannot represent 0.01 or 0.1 exactly, so sums drift and two code paths that should agree disagree by cents nobody can trace back. The fix is integer minor units or an exact decimal type end to end, with sub-cent rates expressed as scaled integers such as micro-units, because a per-request price genuinely is smaller than a cent. The second half of the trap is rounding position: rounding each line and then summing gives a different total from summing and rounding once, and half-up and half-even diverge systematically across many lines, so rounding must happen at one named place and every downstream reader must carry the rounded value rather than recompute it from quantity and rate.
Starting work without saying what you are about to spend time on
State the plan before executing it: the approach, roughly how long it will take, and what you intend to leave hand-waved. That gives the interviewer a chance to redirect you in ten seconds rather than watching you spend fifteen minutes on the wrong sub-problem.
Writing code before the input contract is pinned down
Before the first line, state the types, the size bounds, whether duplicates, negatives or an empty input are possible, whether the input is sorted, whether you may mutate it, and what the function returns when nothing matches. Every one of those answers changes the code, and discovering one at minute twenty costs a rewrite you no longer have time for.
Questions & practice
Given a specific constraint, how would you approach a problem requiring dynamic programming or recursion?
Medium · data structures and algorithms
Approach
- Name the brute-force solution and its complexity before improving on it.
- Walk one small example through your approach before writing the whole thing.
- State the target complexity and say which constraint rules the naive version out.
Follow-up
- Which test case would catch an off-by-one here?
- What is the worst case, and how likely is it on real data?
Can you explain and implement a solution using the Two Pointers technique to solve a string manipulation problem?
Medium · data structures and algorithms
Approach
- Choose the data structure from the access pattern, not from familiarity.
- State the target complexity and say which constraint rules the naive version out.
- Restate the input: its shape, its size, and what is guaranteed about it.
Follow-up
- What is the worst case, and how likely is it on real data?
- Which test case would catch an off-by-one here?
How would you implement an efficient search or traversal within a Linked List?
Medium · data structures and algorithms
Approach
- Choose the data structure from the access pattern, not from familiarity.
- State the target complexity and say which constraint rules the naive version out.
- Name the brute-force solution and its complexity before improving on it.
Follow-up
- Which test case would catch an off-by-one here?
- What is the worst case, and how likely is it on real data?
Parse and verify a timestamped multi-signature webhook header
Easy · parsing · hmac · constant-time-compare · replay-protection
An inbound webhook carries a signature header of at most 1 KiB shaped t=<unix seconds>,v1=<64 hex chars>, with up to five v1 values during secret rotation and possibly unknown scheme keys. You hold the raw request body bytes and the currently active signing secrets. Write the parser and the verifier: accept when any active secret reproduces a signature and the timestamp is within a five-minute tolerance in either direction, reject otherwise. Single left-to-right pass over the header, no regular expression. State what is inside the MAC and why.
Approach
- Parse in one scan: split on
,, then on the first=only, since a value may itself contain=under a future scheme. Accepttexactly once and treat a secondtas a reject rather than last-wins. Push everyv1onto a short list and ignore any other key, so av2can be introduced later without breaking this verifier. - Say what is signed: HMAC-SHA256 over the exact byte string
<t>.<raw body bytes>, yielding 32 bytes or 64 hex characters. The timestamp sits inside the MAC because otherwise an attacker replays yesterday's body with its still-valid signature and only has to edit the header timestamp. - Hash the bytes as received. Verifying against a re-serialised JSON body is the usual defect: key order, whitespace and number formatting all change the bytes while the parsed objects compare equal, so signatures fail for honest senders and the popular 'fix' is to stop checking.
- Compare in constant time over fixed-length digests. Decode the hex to 32 bytes, accumulate
acc |= a[i] ^ b[i]across the whole length, and testacc == 0at the end. Evaluate every candidate without an early exit; at five candidates that is five HMACs over the body, linear in body size and negligible beside the network. - Apply the tolerance as a two-sided bound, rejecting when
|now - t| > 300seconds. A sender whose clock runs ahead of yours is an ordinary case, and an unbounded future timestamp is a free replay window. - Complexity: O(L) over the header producing k candidates, plus k HMACs at O(|body|) each. Space is O(k) beyond the body itself. Do the cheap rejections, including the tolerance check, before any cryptography runs.
Worked solution (15 min)
- Write the grammar on one line before coding:
header := field (',' field)*,field := key '=' value, split on the first=only. - Implement the parser to return
{t: int, v1: [hex, ...]}, rejecting a missingt, a duplicatet, anyv1that is not 64 hex characters, and a header over 1 KiB, all before any cryptography runs. - Implement the verifier: for each active secret compute
HMAC-SHA256(secret, f'{t}.'.encode() + raw_body), compare it in constant time against each parsedv1, and OR the results with no early exit. - Test with a valid signature; the same body with
tmoved 400 seconds into the past; the same body witht400 seconds into the future; a header carrying an unknownv2=alongside a validv1; and a body re-serialised with different JSON key order.
Expected result: The valid case accepts. Both out-of-tolerance cases reject, including the future one. The unknown v2 field is ignored and the v1 still verifies. The re-serialised body fails, which is correct and is exactly why the raw bytes must be retained.
Verify it yourself
- Timing the comparison over 10,000 mismatching signatures that differ at the first character versus the last shows no separation you can measure.
- Removing
tfrom the MAC input makes an old captured body with its original signature pass. Write that test, watch it pass, then puttback. - Every rejection except the signature mismatch itself returns before an HMAC is computed.
Follow-up
- The body is 40 MB. What changes about where you verify, and what can you do before the whole body has arrived?
- A customer reports that signatures fail for exactly the requests whose body contains a non-ASCII character. What is your first hypothesis?
- How do you rotate the signing secret with no failed deliveries, and how long do both secrets stay live?
Explain why the metering dashboard scans every daily partition
Medium · indexing · partition pruning · sargability · tenant skew
usage_event is range-partitioned daily on ingested_at and holds tenant_id, workspace_id, environment, sku, quantity numeric(20,6), occurred_at and ingested_at. The only relevant index is on (occurred_at). A dashboard runs select sku, sum(quantity) from usage_event where tenant_id = $1 and date_trunc('hour', occurred_at) >= $2 and environment = 'production' group by sku, and EXPLAIN shows a sequential scan of every partition. Give each distinct reason, rewrite the predicate so an index can serve it, propose the index, and state the write cost its column order adds.
Approach
- Separate the three causes rather than blaming one. First,
date_trunc('hour', occurred_at)wraps the column, so the predicate is not sargable against a btree on the bare column. Second, pruning keys off ingested_at while the query constrains occurred_at, so no partition can be excluded. Third, even made sargable, (occurred_at) is not tenant-leading, so for one tenant among thousands the scan reads the whole time range and discards nearly all of it. - Rewrite the bound carefully, because the obvious rewrite is only conditionally equivalent.
date_trunc('hour', x) >= $2equalsx >= $2only when $2 is already hour-aligned; for an arbitrary $2 it meansx >= date_trunc('hour', $2) + interval '1 hour'. Normalise the parameter in the caller and leave the column bare. - Restore pruning with a second, redundant predicate on the partition key:
ingested_at >= $2 - interval '<late-data horizon>'. State both sides of it. It prunes to a handful of partitions, and it silently omits any event whose ingest lagged past that horizon, which is precisely what a producer replay produces. Either document the horizon as a stated bound, or partition on occurred_at and move the problem into the dedup window instead. - Propose
(tenant_id, occurred_at) include (sku, quantity)per partition. A partial indexwhere environment = 'production'mostly saves size rather than selectivity, since production dominates the three environments; take it if non-production is a meaningful share and skip it otherwise. - Price the write path honestly. At roughly 250M rows/day each extra index is another insert plus WAL per row, and a tenant-leading key scatters inserts across one hot leaf per active tenant instead of appending to a single rightmost leaf, so page dirtying and random I/O both rise. An INCLUDE payload widens every leaf entry and enlarges the index accordingly.
- Add the index-only-scan caveat before someone reports it as a regression: on a freshly appended table the visibility map is not yet set for recent pages, so the INCLUDE columns still cost heap fetches until autovacuum has been through, and the newest hour is exactly the data the dashboard reads.
Worked solution (30 min)
- Build 30 daily partitions with skewed tenants, one holding about 40% of the rows, then ANALYZE.
- Run
explain (analyze, buffers)on the original query and record how many partitions were scanned and the rows removed by filter. - Apply the rewritten predicate and the index, re-run, and confirm the plan lists only the partitions inside the ingested_at bound.
- Re-run with $2 set to a non-hour-aligned timestamp and confirm the rewritten and original predicates return identical rows.
- Insert an event with ingested_at six hours past occurred_at and check whether the pruning predicate excludes it.
Expected result: Three causes named separately: a non-sargable date_trunc, pruning on ingested_at against a predicate on occurred_at, and a non-tenant-leading index. A rewrite to occurred_at >= <hour-normalised $2> plus a redundant ingested_at >= $2 - horizon, an index on (tenant_id, occurred_at) including sku and quantity, a per-row write cost stated, and the late-data omission the pruning predicate introduces stated as a known bound.
Verify it yourself
- The plan lists only partitions inside the ingested_at bound.
- Rows removed by filter on the index scan is small relative to rows returned.
- The rewritten and original predicates agree for both hour-aligned and non-aligned bounds.
- A deliberately late-ingested event is visible or omitted exactly as the stated horizon predicts.
Follow-up
- CREATE INDEX CONCURRENTLY is not supported on a partitioned parent. Give the sequence that gets this index onto 400 existing partitions without blocking ingest.
- One tenant holds 200 times the median row count and the dashboard still times out for them with the index in place. What changes?
- Should this read hit
usage_rollup_hourlyinstead? State what that costs in freshness and what the watermark lets you promise.
Find the join that inflates every invoice total
Medium · join fan-out · aggregation grain · sql correctness
invoice_line_item holds line_id, invoice_id, tenant_id, sku, rate_tier, quantity, unit_price_micros, amount_minor (bigint), currency, kind, voided_at. invoice_payment_attempt holds attempt_id, invoice_id, tenant_id, amount_minor, status (succeeded, failed, pending), created_at, and an invoice has many attempts. A finance report runs select i.invoice_id, sum(l.amount_minor), count(p.attempt_id) from invoice i join invoice_line_item l using (invoice_id) join invoice_payment_attempt p using (invoice_id) group by 1 and the totals are wrong. Say precisely what the sum now equals, and write a version that is also correct for invoices with zero attempts.
Approach
- Compute what the query actually returns before fixing it. The two joins form a Cartesian product per invoice, so each line row repeats once per attempt row:
sum(l.amount_minor)is the true total multiplied by the attempt count, andcount(p.attempt_id)is attempts times lines. Three lines and two attempts report double the money and six attempts. - Reject the reflex repair.
count(distinct p.attempt_id)does fix the count, because attempt_id is unique.sum(distinct l.amount_minor)does not fix the sum, because two legitimate lines with equal amounts collapse into one. DISTINCT inside an aggregate deduplicates values, not rows, and the difference stays invisible until two lines happen to match. - Aggregate each branch to invoice grain before joining: one CTE summing lines by invoice_id, one counting attempts by invoice_id, then join the two results. A LATERAL subquery per invoice is equivalent and sometimes plans better when the outer set is small. Either way every aggregate stays at the grain it was defined at.
- Keep invoices with no attempts by making the attempt branch a LEFT JOIN with
coalesce(attempt_count, 0). An inner join here silently drops every unpaid invoice, which is usually the exact population finance is asking about. - Push each filter to its own grain:
where l.voided_at is nullbelongs inside the line CTE, not the outer query, or it would also filter the attempt branch through the join. Put the tenant predicate on both branches, since the denormalised tenant_id is what stops a wrong join crossing tenants. - Leave yourself a standing check: an invoice total is a function of its non-voided lines and of nothing about payments, so if changing the payment filter moves the money figure, the fan-out is back.
Follow-up
- Add a third branch for credit notes applied to the invoice. Does the CTE shape still hold, and when would a single pass with
filter (where ...)be better? - Over 500k invoices this report takes minutes. Which grain would you materialise, and how do you keep it correct when a line is voided?
- The same report is needed per tenant per month. What index makes the line CTE cheap?
Webhook fan-out with per-endpoint isolation and backoff
Medium · fan-out · backpressure · circuit breaker · ordering
One domain event fans out to every matching subscription, producing a webhook_delivery row per (subscription_id, event_id, redelivery_seq). Peak unique event rate is 20k/second; attempts run five to ten times that once fan-out and retries are counted. One customer endpoint has returned 503 for six hours and its backlog holds days of events; every other customer must be unaffected. Design the delivery system: how a worker claims work, the backoff schedule, the per-endpoint circuit breaker, the queue partitioning, and whether you offer ordering per subscription. State the delivery guarantee in one sentence.
Approach
- State the guarantee first, because it determines the rest: at-least-once with a stable event_id, and the consumer documented as responsible for idempotency. Exactly-once over HTTP is not deliverable - the 200 can be lost after the customer has already committed - so any design that promises it is either lying or is really offering at-most-once.
- Partition work per subscription rather than into one global pool, with a concurrency cap per subscription. With a shared pool, the endpoint that has been dead for six hours consumes workers on retries that will fail, and every other customer's delivery latency rises: head-of-line blocking across tenants is the exact failure being designed against here.
- Claim by compare-and-set with a fencing token: UPDATE webhook_delivery SET status = 'in_flight', lease_token = $new, leased_until = now() + interval '60 seconds' WHERE delivery_id = $1 AND status IN ('pending','failed_retryable') AND (leased_until IS NULL OR leased_until < now()), and make the terminal write carry AND lease_token = $new so a paused worker's late write is rejected rather than overwriting a newer attempt. Find due work through the partial index on next_attempt_at WHERE status IN ('pending','failed_retryable'), so the scan is proportional to live rows rather than to the terminal rows that outnumber them by orders of magnitude.
- Use full jitter: sleep uniformly in [0, min(cap, base x 2^(attempt-1))]. Plain exponential backoff hands a recovering endpoint its entire backlog as one synchronised herd and knocks it over again; full jitter de-correlates it. Then check the schedule actually spans the retention you promise - with base 1 s and a 3,600 s cap, twenty attempts have an expected total elapsed time of only about 4.6 hours, so a twenty-four-hour promise needs roughly fifty-nine attempts or a larger cap.
- Trip a circuit per endpoint on consecutive failures or a failure ratio over a rolling window: stop dispatching, push next_attempt_at out or mark new deliveries dropped_circuit_open, and half-open with exactly one probe rather than a batch. Bound the backlog explicitly with a per-subscription cap or retention, and decide in advance whether a recovered endpoint receives six hours of events at full rate or a pointer telling it to fetch what it missed.
- Offer ordering only as an opt-in mode of one in-flight attempt per subscription, and price it honestly: with parallel attempts a retried event overtakes a newer one, so ordering requires serialisation, and serialisation means one slow endpoint blocks its own queue entirely. That converts a shared problem into that customer's own problem, which is the right place for it, but it is still a real cost.
Follow-up
- The endpoint recovers. Does it receive six hours of events at full rate, and what does that do to it?
- Trace the exact code path by which an event belonging to one tenant could be signed and sent to another tenant's endpoint.
- A customer insists they never received an event your row marks delivered. What evidence do you have, and what does payload_digest let you prove?
Per-tenant rate limiting that holds across one hundred pods
Medium · rate limiting · hot keys · atomicity · write skew
The same gateway enforces each tenant's plan rate limit in requests/second and its monthly quota. Traffic is heavily skewed: a handful of tenants exceed all others combined, while a small tenant's requests may all land on one of the 120 pods. Both limits must hold in aggregate rather than per pod, per credential or per region, and the check may add at most 2 ms at p99. Specify the algorithm, where the state lives, the exact operation performed per request, and the behaviour when the counter store is unreachable - separately for the rate limit and the quota.
Approach
- Do the arithmetic on per-pod buckets before discarding them, and be precise about which half fails. Giving each pod rate/N keeps the sustained aggregate correct - 120 buckets of 5 rps sum to the 600 rps limit - but a tenant whose traffic all lands on one pod is throttled at 5 rps, a hundred and twentieth of what it pays for. The over-admission comes from burst depth, not sustained rate: if each pod carries a full bucket depth B, the aggregate burst is 120B against a limit that intended to allow B.
- Move the rate limit to a shared counter accessed in exactly one atomic round trip - a token bucket or GCRA as a single server-side script returning allow/deny plus a retry-after - never a read followed by a write. Keep the key tenant-scoped and region-local; a globally shared counter costs a cross-region round trip that alone exceeds the 2 ms budget, so shard the global limit per region in proportion to observed regional share and accept that a tenant whose traffic shifts regions is briefly throttled until the shards rebalance.
- Cut round trips for the largest tenants with leased tokens: a pod leases 50 at a time and spends them locally, reducing counter round trips by 50x at the cost of bounded over-admission of pods x lease_size, which is 6,000 requests in the worst case here. Shrink the lease for tenants with small limits, where that bound would swamp the limit itself.
- Treat the quota as a different problem, not a longer window. It is a count that other in-flight requests are changing, so select-the-usage-then-insert is write skew: read-committed permits it and repeatable read permits it too, because snapshot isolation is exactly what allows two transactions to read a consistent count and both write. Enforce it inside one statement - an atomic increment whose returned value is compared, or UPDATE ... SET used = used + $1 WHERE used + $1 <= limit RETURNING - or with a constraint that makes the surplus write fail.
- Write the unavailability policy per limit rather than one policy for both. The rate limit fails open to a conservative local bucket, because it protects capacity and failing closed turns a counter blip into a total outage. The quota fails open for tenants whose last known usage was well below the cap and closed for those at or above it, since that is the only decision with revenue attached, and reconciles from the usage ledger on recovery - which is why the monthly counter must be re-seedable from that ledger rather than being the only copy.
Worked solution (30 min)
- For a 600 rps limit across 120 pods, compute the sustained rate and the aggregate burst under local buckets of depth B, then the throughput of a tenant whose traffic all lands on one pod.
- Write the shared-counter operation as a single atomic script: its inputs, its return value, and the number of network round trips per request.
- Size a lease of 50 tokens: compute the worst-case over-admission as pods x lease_size and the reduction in round trips, then decide the smallest limit at which the lease is still acceptable.
- Write the two unavailability policies side by side and the reconciliation step that runs when the counter store returns.
Expected result: A per-region shared token bucket keyed (tenant, region, window) executed as one atomic script at roughly one sub-millisecond intra-region round trip; the monthly quota enforced by a single atomic increment-and-compare that is re-seedable from the usage ledger; leases only for high-limit tenants with over-admission bounded at 120 x 50 = 6,000 requests; the rate limit failing open to a conservative local bucket and the quota failing closed only above the last known cap.
Verify it yourself
- Separate the two halves of the local-bucket claim: sustained aggregate is correct at rate/N per pod, burst is not. If your answer says local buckets let a tenant exceed its sustained rate, the arithmetic does not support it.
- Re-read your quota check and confirm no branch reads the counter and then writes it. If the predicate is not inside the write or enforced by a constraint, the race is still there.
- Compare worst-case lease over-admission against the smallest tenant limit you apply leases to. If 6,000 exceeds that tenant's per-minute allowance, the lease size is wrong for that tier.
Follow-up
- One tenant is 40% of all traffic and its counter key lands on a single shard. What do you change, and what does that cost in accuracy?
- Clients receive 429s and retry. What stops the well-behaved ones from synchronising into a herd?
- A tenant reports being throttled below its limit. Which evidence do you produce, and does your design emit it today?
One tenant's counter writes stall the whole connection pool
Medium · lock contention · connection pool · multi-tenancy
A change that made a per-tenant usage counter correct now produces site-wide latency whenever one large tenant writes: unrelated endpoints time out waiting for a connection while database CPU stays low and no statement is slow. The change wraps the counter update in a transaction that takes SELECT ... FOR UPDATE on one row, calls an external pricing service, then updates and commits. Give an ordered checklist, the arithmetic that bounds that tenant's write rate, and three repairs with the cost each one accepts.
Approach
- Separate waiting from working. Low database CPU alongside high application latency points at a queue, so instrument connection-acquisition wait separately from query execution time; that queue forms in the application and is invisible in database metrics, which is why the database looks healthy throughout.
- Confirm the lock rather than assuming it: sample waiting sessions and group by wait event, relation and tuple. Contention concentrated on one tuple belonging to one tenant is the signature; a deadlock would instead show the database aborting transactions after its detection timeout, which is not happening here.
- Do the arithmetic out loud. Throughput on a serialised row is one divided by the lock hold time, and the hold spans the external call, so a 20 ms pricing call caps that tenant near 50 writes per second no matter how many pods run. Every waiter also holds a pooled connection while it queues, so the shared pool drains and unrelated tenants fail at acquisition.
- Repair one: shrink the critical section to a single statement with the price resolved before the transaction opens. Cost is a stale price for the duration of one request and a second round trip; benefit is a hold time measured in the database's own execution time.
- Repairs two and three change where the contention lives rather than how long it is held. Sharding the counter into per-(tenant, bucket) rows and summing on read multiplies write throughput by the shard count, at the cost of an aggregate on every read and a shard count you must size against the largest tenant rather than the median. Accumulating in memory and flushing periodically removes the per-write round trip entirely, paid for with a bounded loss window on crash, which is acceptable for a rate limiter and not for a billing counter.
- Contain independently of which repair wins: a separate pool or per-tenant concurrency cap for this write class, a statement timeout low enough that a pathological query dies before it accumulates waiters, and an idle-in-transaction timeout so a stuck client cannot pin a connection and its locks.
Follow-up
- What would a genuine deadlock look like here, which two code paths would produce one, and how does the database's response differ from what you observed?
- If a transaction-pooling proxy sits in front of the database, which of your three repairs changes behaviour, and what stops working that would have worked on a direct connection?
- The counter also enforces a quota. Why is SELECT the count and then INSERT still wrong after you have fixed the contention?
Suggested practice checklist
For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.
Day 1 — Numbers before diagrams
- Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
- Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
- Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.
Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.
Day 2 — One system, from requirements to schema
- Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
- Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
- Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.
Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.
Day 3 — The consistency you are actually buying
- Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
- Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
- Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.
Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.
Day 4 — Failure is the design
- For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
- Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
- Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
- Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.
Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.
Day 5 — Scaling the hot path
- Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
- Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
- Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
- Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.
Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.
Day 6 — Keep the coding hand in, at the bar that applies to you
- Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
- Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
- Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.
Deliverable: One problem hardened to review standard, with its test list and one written review comment.
Day 7 — Defend it while being interrupted
- Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
- Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
- Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.
Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.
Risky changes and the reversal path
For anything that touched live traffic, be ready to say how you would have undone it: a flag, a staged rollout, dual writes with the old path still authoritative. Once the old column is dropped or the source rows are overwritten there is no reverse, so name what you kept a copy of and for how long.
Unblock an engineer on a job run that finished twice
Easy · mentoring · fencing tokens · lease expiry · debugging method
An engineer two weeks into the team brings you a job_run row showing status succeeded with an exit_code written by a worker declared dead ten minutes earlier; the retry attempt also shows succeeded. They have spent a day adding logging and are no closer. You have twenty minutes and you do not want to take the keyboard. Describe how you unblock someone: the question you ask first, what you let them find themselves, the concept you name and when, and how you check the next day that they own the fix rather than having watched you produce it.
Approach
- Ask what they expect rather than what they see: which statement set status to succeeded, and what did it check before writing? That question points directly at the update's WHERE clause, which is where the answer lives, and it costs them nothing to answer, so it does not read as a test.
- Let them build the timeline themselves from the row: queued_at, started_at, leased_until, finished_at and worker_id, on both the original run and the retry. Two different worker_ids with a lease expiry between them tells the whole story, and they will see it before you say it.
- Name the concept once the evidence has earned it. A lease bounds time; it does not prevent a write. The store has to reject a stale writer, which means the update carries a fencing token the row compares — update job_run set status = 'succeeded' where run_id = $1 and lease_token = $2 and status = 'running' — and a long garbage-collection pause or a brief partition is enough to produce what they are looking at.
- Point at the second, less obvious half and let them decide it: 'lost' exists in the status enum precisely so a run whose worker vanished is not recorded as failed, because failed asserts an outcome nobody observed and the system then bills and retries on that assertion. Ask them what these two rows should have said.
- Leave them with the next step rather than the patch — a test that kills the first worker after the sandbox exits and before the row is written — and say when you are available again, so the offer is real rather than polite.
- Check ownership the next day by what they produced, not by asking if it went well: a test that reproduces the window proves they understood it; a test that only asserts the new WHERE clause proves they copied it. Ask them to explain it to a third person and listen for whether the explanation is theirs.
Follow-up
- They propose a longer lease instead of a token. What do you say, and what breaks when legitimate runs last thirty minutes?
- How can you tell whether your explanation landed or they simply deferred to you?
- The same engineer hits a variant of this next month. What did you fail to teach the first time?
Reverse a webhook ordering decision after measuring its cost
Medium · reversing decisions · head-of-line blocking · at-least-once · api contracts
You argued for strict per-subscription ordering in webhook-delivery, which means one in-flight attempt per subscription. It shipped. Three months later a single unresponsive endpoint holds one subscription's queue at a six-hour backlog, and two customers report events arriving out of order anyway once their own retries are counted. Describe a decision you reversed: what you originally optimised for, the measurement that changed your mind, what the reversal cost in engineering time and customer change, and how you told the people who had already built on the original guarantee.
Approach
- State the original decision as a trade you made knowingly. Ordering across a network requires a single in-flight attempt per subscription, and its price is head-of-line blocking whenever one endpoint is slow. 'We priced it wrong' is a much stronger opening than 'we did not realise', and it is usually the true one.
- Bring the measurement that flipped it, not the anecdote: backlog age at the ninety-ninth percentile per subscription, the share of subscriptions where one slow endpoint gated an otherwise healthy queue, and the delivery throughput lost to serialisation. A reversal justified by complaints is indistinguishable from a reversal justified by fatigue.
- Name what you learned about the guarantee itself, which is the engineering content of this story. At-least-once delivery means a retried event already arrives after newer ones and the consumer already must be idempotent, so a guarantee the customer has to defend against anyway was never worth what it cost to provide.
- Describe the migration, because reversing a published contract is the hard half and the part candidates skip. Parallel attempts behind a per-subscription flag, a monotonically increasing sequence number added to the envelope so order-sensitive consumers can sort or discard, documentation that states at-least-once and unordered in those words, and a deprecation measured in quarters because the client is a pinned SDK inside a build pipeline you cannot see or redeploy.
- Give the cost in the two currencies that matter: engineer-weeks, and how many customers had to change code. Then say who you told before it shipped rather than in a changelog afterwards, and which large customer you left on the old behaviour and for how long.
- Close with the signal you now weight differently, stated as something you would do earlier next time: measuring the blocking cost on the slowest decile of endpoints before committing to the guarantee, rather than after a customer noticed.
Follow-up
- A customer insists they need ordering. What do you offer them that is not global serialisation?
- How did you choose the deprecation window given that you cannot see or redeploy the clients?
- What would have to be true for you to reverse back?
Estimate a tenant-leading index migration you have never run
Hard · estimation · online migration · index builds · uncertainty
Someone needs a date. usage_event carries an index on (occurred_at) and needs (tenant_id, occurred_at); the largest tenant holds roughly a hundred times the median tenant's rows, the table is partitioned daily with years of retention, and you have never run a migration on a table this large. Give an estimate you would defend: how you decompose the work, the two or three numbers you would go and measure first, the range and confidence you state, and what you commit to when the person asking needs a single date today.
Approach
- Refuse the bare number and then give one anyway, in the form that is actually useful: a range plus the measurement that collapses it. 'Four to eleven days; one afternoon building this index on a restored copy of the largest partition takes that to within a day' is an answer, while 'it depends' is not.
- Decompose by failure mode rather than into equal chunks, because that is where estimates go wrong. On a partitioned parent you create the index ON ONLY the parent, build each partition's index with CREATE INDEX CONCURRENTLY, then ALTER INDEX ... ATTACH PARTITION, at which point the parent index becomes valid. CONCURRENTLY does not block writes but scans each partition twice, waits out older transactions, cannot run inside a transaction block, and on failure leaves an invalid index you must drop concurrently and retry.
- Name the two unknowns that dominate and price them: build time on one restored partition of realistic size, and whether the planner actually chooses the new index for the skewed tenant, since selectivity for a tenant holding most of the rows is a different question from selectivity for the median tenant. Both are half-day measurements against a replica, and both are cheaper than being wrong by a week.
- State the assumptions the range is conditional on, because that is what makes a slip a re-estimate instead of a credibility event: no partition above a stated row count, one concurrent build at a time so it does not compete with ingest for I/O, and an ingest backlog that can absorb the added write amplification while both indexes exist.
- Budget the step nobody budgets: verification and the old index's removal. Dropping the old index is fast, but deciding it is safe to drop means confirming no plan still uses it, and that confirmation waits on real traffic across a full weekly cycle rather than on your patience.
- Answer the single-date request honestly. Commit to a date for the first checkpoint — the measured build number from the replica — and to re-estimating on that date, and say plainly what you are not committing to yet. A date with a scheduled re-estimate is worth more to the asker than a confident wrong one, and you should say why in those words.
Follow-up
- The concurrent build fails half way through the largest partition. What is the state of the database and what do you do next?
- Your estimate slips by sixty percent. Which assumption broke, and at what point would you have known?
- The person asking needs the date for a customer commitment. Does your answer change?