Interview guides · Aigen Technologies
Aigen Technologies Software Engineer Interview Questions & Guide 2026
Concurrency drills for Aigen Technologies software engineer prep: races, lock ordering, and what a check-then-act sequence assumes.
The 60-second brief
The Software Engineer role at Aigen Technologies is a pivotal position focused on building high-impact, scalable solutions that bridge the gap between advanced technology and real-world application. You will be tasked with solving complex engineering challenges, often working closely with leadership to translate technical strategy into robust product features.
Ask how many rounds there are and what each one is before you plan your weeks, because the composition is what your hours should follow. A loop with three coding rounds and one short design conversation deserves a different split from the reverse, and whoever schedules it will usually say plainly which it is.
Focus areas
- Keep money in integer minor units
- Make every write idempotent under client retries
- Bound blast radius with per-tenant concurrency limits
Overview
This role is critical to the mission of Aigen Technologies, as your contributions directly influence the efficiency and reliability of the company’s core platforms. You should expect to work in a fast-paced, high-ownership environment where your ability to navigate ambiguity and deliver clean, effective code is paramount. Whether you are working on specialized frameworks or core infrastructure, your work will be central to the company’s ongoing innovation.
The Aigen Technologies interview process
Aigen Technologies candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Initial Screening
Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.
What to demonstrate
- Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
- Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
- Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not
How to prepare
- Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
- Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
- If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window
High-Level Technical Discussions
The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.
What to demonstrate
- In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
- In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
- Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes
How to prepare
- Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
- Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
- For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first
Onsite/Remote Session
Coding rounds mostly set a floor. They decide whether you clear the bar, not where you land on the ladder. Level tends to come out of the design discussion and the ownership stories, so the question worth auditing beforehand is whether the scope you describe matches the scope of the job. Work that stops at your own service, or a story whose hard part was writing the code rather than getting several people to agree on an interface, reads a level below where you think you are interviewing, and that gap is usually resolved downwards.
What to demonstrate
- Whether the largest thing you describe owning ran end to end — the decision, the migration path, the rollout, and what you did when it went wrong — or stopped at the change you merged
- Whether design answers include what you would not build, what you would defer, and what you would measure before committing, rather than only what the boxes are
- Whether a disagreement in a story was settled with something checkable — a benchmark, a prototype, a written proposal — instead of by seniority or by waiting it out
- Whether you can say which calls you made alone and which you escalated, and why the line sat where it did
How to prepare
- Write your largest piece of owned work as a timeline of decisions — who decided what, when, and what you did when the plan broke — then delete every sentence whose subject is "we" and see how much survives
- Take one system you know well and drill the migration answer: how old and new paths run side by side under live traffic, how you compare their outputs, what the rollback is once writes are going to both, and which step you would not automate
- Map each line of the ladder in the job posting to a specific thing you have done, find the line you cannot support, and prepare the closest evidence you have plus an honest account of the gap
Where candidates lose points
Holding money in a floating-point type, or rounding it more than once
Binary floating point cannot represent 0.01 or 0.1 exactly, so sums drift and two code paths that should agree disagree by cents nobody can trace back. The fix is integer minor units or an exact decimal type end to end, with sub-cent rates expressed as scaled integers such as micro-units, because a per-request price genuinely is smaller than a cent. The second half of the trap is rounding position: rounding each line and then summing gives a different total from summing and rounding once, and half-up and half-even diverge systematically across many lines, so rounding must happen at one named place and every downstream reader must carry the rounded value rather than recompute it from quantity and rate.
Paginating a growing table with limit and offset
Two unrelated defects share the idiom. Correctness: rows inserted or deleted between page requests shift the window, so a consumer walking an export skips rows and sees others twice, which for a customer-facing sync is silent data loss rather than an error anyone notices. Cost: the database still produces and discards the skipped rows, so page N costs time proportional to N times the page size and a deep page on a large table degrades from milliseconds to seconds. Keyset pagination over a stable, unique, indexed ordering -- where (created_at, id) < ($1, $2) order by created_at desc, id desc limit $3 -- is constant-cost per page and immune to shifting, on the precondition that the cursor columns never change value for a row, which disqualifies updated_at as a cursor.
Writing code before the input contract is pinned down
Before the first line, state the types, the size bounds, whether duplicates, negatives or an empty input are possible, whether the input is sorted, whether you may mutate it, and what the function returns when nothing matches. Every one of those answers changes the code, and discovering one at minute twenty costs a rewrite you no longer have time for.
Assuming the bug is in the framework
Suspect your own code first: read the stack trace top to bottom, check which versions are actually installed rather than which ones you believe are, and reproduce in isolation before blaming a library that thousands of people run daily. When the fault really is upstream, you need that minimal reproduction to say so credibly anyway.
Questions & practice
Locate a billing reconciliation gap without rescanning ninety million events
Hard · reconciliation · dimensional-bisection · watermarks · hypothesis-testing
A tenant's sealed invoice total is 0.4% below the sum of its raw usage_event rows for the period. That tenant has 90 million events over 30 days in a table partitioned daily on ingested_at, and its rollups carry source_max_ingested_at, revision and sealed_at. Recomputing all 30 days from raw is correct, and you are not going to do it. Give the procedure that locates the divergent (workspace, sku, hour) cell, the cost of each probe, and the one query you run before any of it.
Approach
- Run the free query first. Sum raw quantity for the period restricted to
ingested_at <= source_max_ingested_atof the sealed rollups, and compare that against the unrestricted sum. The rollup stores the watermark precisely so this can be answered without a scan. If the whole 0.4% sits above the watermark, nothing is broken: it is late data, it becomes an adjustment line, and the investigation ends in one query. - Only if the gap survives that test do you bisect, and you bisect by dimension rather than by rows. Compare 30 per-day totals, then inside the offending day compare the 6 SKUs, then the workspaces, then the 24 hours. That is roughly 30 + 6 + W + 24 grouped probes, each an indexed range scan over one daily partition for one tenant, against O(N) per attempt for the naive re-fold.
- Quantify why naive is not merely slow but unusable mid-incident: at a generous 200,000 rows/second sequential, 90 million rows is about 7.5 minutes per attempt, you will want ten attempts, and every one competes for I/O on the same partitions live ingest is writing. The diagnostic worsens the backlog it is diagnosing.
- Before fetching each comparison, state what it would look like under each hypothesis. Two adjacent hours off by equal and opposite amounts is
occurred_atversusingested_atbucketing. A whole day offset by exactly N hours is a timezone applied at the wrong layer. A gap confined to one SKU in one workspace is an environment filter. The same(tenant_id, idempotency_key)present in twoingested_daypartitions is the dedup horizon losing a retry that crossed midnight. - Make the next bisection cheap by storing the aggregate you keep recomputing. A per-
(tenant_id, ingested_day)count and quantity checksum turns step two from thirty probes into one read, and it is the same number the reconciliation job already produces. - Whatever you find, the sealed period does not change value. The correction is an adjustment line pointing at the line it reverses, carrying its own
source_rollup_watermark, because the original invoice is the evidence of what the customer was charged.
Worked solution (35 min)
- Reproduce the shape locally: generate 2 million events for one synthetic tenant over 5 days, fold them, then inject three defects, namely 0.2% of events bucketed by
ingested_at, a handful of duplicateidempotency_keyvalues whose retries cross midnight, and a block of events ingested after the seal. - Write the watermark-bounded query first and record how much of the gap it explains on its own.
- Bisect by day, then SKU, then hour, recording the probe count and the rows each probe touches.
- For each located cell, write down the predicted signature before querying it, then check whether the data matches the prediction.
- Time a full re-fold of the 2 million rows and extrapolate to 90 million.
Expected result: The watermark-bounded query accounts for the post-seal block entirely and removes it from the investigation. The ingested_at bucketing appears as adjacent hours off by equal and opposite amounts. The midnight-crossing duplicates appear as one (tenant_id, idempotency_key) present in two ingested_day partitions. Bisection reaches each cell in fewer than 70 probes.
Verify it yourself
- The three injected defects sum to the original gap with no unexplained remainder.
- Each probe's row count is bounded by one day-partition for one tenant, not by the period.
- Removing the
source_max_ingested_atbound from the first query makes a gap appear, and grow, while ingest runs, even after every defect is removed.
Follow-up
- The gap is 0.4% in one direction on one day and 0.4% the other way the next day. What does that shape rule in, and what does it rule out?
- How do you distinguish a duplicate from a restatement, given
revisionandrecomputed_aton the rollup? - Ingest is still running while you investigate. What makes your two numbers comparable at all?
Fold a deduplicated usage stream into hourly rollups
Easy · aggregation · deduplication · watermarks · exact-arithmetic
You are given one day of usage_event rows, up to 250 million, each carrying event_id, tenant_id, workspace_id, environment, sku, quantity numeric(20,6), idempotency_key, occurred_at and ingested_at. Produce usage_rollup_hourly cells keyed (tenant_id, workspace_id, sku, hour_start) with quantity_sum, event_count and source_max_ingested_at. An event counts once per (tenant_id, idempotency_key). The rollup grain has no environment column, so state your filter. One pass. Give your time and space bounds, and say what the deduplication actually costs in memory.
Approach
- Bucket on
occurred_at, neveringested_at:hour_start = date_trunc('hour', occurred_at at time zone 'UTC'). The two columns answer different questions.occurred_atsays which hour the customer is billed for;ingested_atsays how current the fold is. Using the second for the first makes late data invisible instead of correctable. - The fold is trivial and the deduplication is the entire cost, so price it before designing anything clever. An exact set over
(tenant_id, idempotency_key)at 250M entries, stored as a 16-byte 128-bit hash in an open-addressed table at 0.7 load factor, needs about 357M slots at 16 bytes each, roughly 5.7 GB. The fix is partitioning byhash(tenant_id) % Pso each shard holds 1/P of the set and no tenant's keys straddle shards. - Rule out a Bloom filter as a replacement, in the right direction: a false positive reports 'already seen' for an event never seen, so you drop a real event and lose revenue with no error raised. It is usable only as a negative pre-filter in front of the exact set, where a miss is conclusive and a hit must fall through to the real lookup.
- Accumulate in scaled integers, not binary floating point.
numeric(20,6)admits values below 10^14, so one event scaled to micro-units can reach 10^20, past int64's 9.22 x 10^18; use a 128-bit or arbitrary-precision accumulator unless you first bound the per-event maximum. binary64 represents integers exactly only to 2^53, about 9.01 x 10^15, and cannot represent 0.1 at all, so two runs that sum in different orders disagree. - Carry
source_max_ingested_at = max(ingested_at)over the events folded into each cell, and countevent_countover accepted, post-dedup events. Without that watermark there is no way to prove later what a number did and did not include, which is the first question any reconciliation asks. - State the environment filter explicitly, because the rollup grain cannot record it. A fold that quietly includes
stagingbills non-production traffic; one that quietly excludes it loses a cost signal. Production-only is the billing answer, and either way it belongs in the job name and the output metadata. Complexity: O(n) time, O(distinct dedup keys) space, dominated by the dedup set rather than by the cells.
Follow-up
- A producer retries at 23:59:59 and the retry lands at 00:00:01. The unique index on the daily-partitioned table must include the partition key. What gets double-counted, and what is the smallest change that fixes it?
- The consumer acknowledges its batch before committing the fold. Which failure loses revenue now, and which arrangement duplicates instead?
- What makes a re-run over the same day produce byte-identical rollups?
Seal an hour under late data with bounded memory
Hard · watermarks · late-data · quantile-sketch · conditional-write
Metering ingest reads 256 partitions at 10,000 to 40,000 events/second. Events carry occurred_at and ingested_at, and during a producer replay the gap between them is hours. Seal each UTC hour once no more than 50 parts per million of that hour's eventual quantity can still arrive, using memory that does not grow with the size of the replay. Define the watermark, the lateness parameter and how you measure it, the structure holding open hours, and the write that performs the seal. State what an idle partition does to your watermark.
Approach
- Two clocks, two jobs. Bucket by
occurred_at, because that is the hour the customer is billed for, and advance the watermark oningested_at, because that is what the fold has consumed and whatsource_max_ingested_atrecords. Conflating them is what makes late data invisible. - The global watermark is the min over partitions of each partition's committed
ingested_at, not the max: the fold is trustworthy only as far as the slowest partition. The consequence is that one idle partition pins the watermark forever and nothing seals, so an idle partition must promote its watermark to wall clock after a stated idle timeout, and that timeout becomes a correctness parameter, because a partition that is slow rather than idle gets sealed past. - Choose the lateness L from the measured distribution of
ingested_at - occurred_at, weighted by quantity rather than by event count. The target is 50 ppm of the hour's quantity, and a replay is rare in events while carrying disproportionate mass, so an event-weighted quantile picks an L that is comfortably wrong at exactly the moment it matters. - Measure that quantile in bounded memory. A Greenwald-Khanna summary gives epsilon-approximate quantiles in O((1/epsilon) log(epsilon n)) space; a t-digest costs more per merge but has relative error that tightens at the tails, which is the half of the distribution you are reading at p99.99. Keep a separate summary per tenant class, because one tenant's batch importer is not the population.
- Hold open hours in a min-heap keyed by
hour_start. When the watermark advances, pop every hour withhour_end + L < Wand seal it: O(log H_open) per advance and O(1) amortised per event to touch its bucket. Memory is open hours multiplied by distinct(tenant, workspace, sku)keys, so cap the number of simultaneously open hours and spill the oldest intousage_rollup_hourlyasstatus='open'with arevisionbump. While an hour is open the row is upsertable, so the store is your overflow. - The seal itself is a conditional write:
update ... set status='sealed', sealed_at=now() where status='open' returning .... Two sealers race on every restart, and the loser must see zero rows and stop rather than write a second value. After the seal, an event for that hour is not an upsert but an adjustment, andsource_max_ingested_atis what proves it arrived afterwards.
Follow-up
- A replay starts during the sealing window for a period you are about to close. What do you do, and what is the customer-visible consequence of each option?
- Your measured quantity-weighted p99.99 lateness is six hours and the invoice must be issued at 02:00 UTC on the first. How do you reconcile those two numbers?
- How would you detect that L has drifted before it costs you an hour's quantity?
Paginate a tenant's delivery export without skipping rows
Easy · keyset pagination · indexing · multi-tenancy
A customer exports webhook_delivery: delivery_id (bigint identity), subscription_id, tenant_id, event_id, status, attempt_count, next_attempt_at, created_at, delivered_at, updated_at. The endpoint runs select ... where tenant_id = $1 order by created_at desc limit 100 offset $2, and customers report rows missing from exports taken while new deliveries are being inserted. Write the replacement query and the index that supports it, paging a tenant's deliveries newest first at constant cost per page. State why updated_at cannot be the cursor column.
Approach
- Name the defect precisely. OFFSET is a position in a result set that is recomputed on every request, so a row inserted ahead of the window shifts everything back by one and the next page starts after a row the client never received. Nothing errors and no identifier gap appears, so the loss is silent.
- Replace the position with a value predicate over a stable, unique, indexed ordering:
where tenant_id = $1 and (created_at, delivery_id) < ($2, $3) order by created_at desc, delivery_id desc limit 100. The row comparison is load-bearing: created_at alone is not unique, so ties straddling a page boundary are dropped or repeated, which is the same bug in a smaller window. - Index
(tenant_id, created_at, delivery_id). PostgreSQL scans a btree in either direction, so an all-DESC ORDER BY is served by an ASC index read backwards and no DESC modifiers are needed; they only matter when the ORDER BY mixes directions. Confirm the plan has no Sort node above the index scan, or the LIMIT stops being an early exit. - Price both forms: keyset is one index descent plus 100 adjacent leaf entries per page, constant regardless of depth, while OFFSET still produces and discards every skipped row, so page N costs time proportional to N times the page size and a deep page on a large table goes from milliseconds to seconds.
- Rule out updated_at as the cursor from the precondition, not from taste: a cursor column must never change value for a row already paged past. updated_at moves on every delivery attempt, so a row the client already emitted re-enters a later page and is exported twice. created_at and delivery_id are immutable, which is the whole qualification.
Worked solution (20 min)
- Load about 50k deliveries for one tenant, then walk them with the OFFSET query while a writer inserts 10 rows/second, collecting every returned delivery_id.
- Compare the distinct ids collected against the set of ids that existed when the walk started, and record the shortfall.
- Repeat the walk with the keyset query and confirm every pre-existing id is returned exactly once.
- Run
explain (analyze, buffers)on page 1 and page 500 of each form and compare shared buffer hits.
Expected result: A keyset query using the row comparison (created_at, delivery_id) < ($2, $3) with order by created_at desc, delivery_id desc limit 100, backed by (tenant_id, created_at, delivery_id) scanned backwards; plus the statement that OFFSET drops rows under concurrent insert and costs time proportional to the offset, and that updated_at is disqualified because it mutates on retry.
Verify it yourself
- Buffer counts for page 500 are the same order of magnitude as page 1.
- Every id present before the walk is returned exactly once when only inserts run concurrently.
- The query plan shows no Sort node above the index scan.
- Two rows sharing a created_at value to the microsecond are both returned across the page boundary.
Follow-up
- The client wants a snapshot as of one instant rather than a live tail. Compare a repeatable-read transaction held open, an added
created_at <= $snapshotbound, and a materialised export table. - A retention job deletes deliveries older than 90 days. What does a client mid-walk see, and does keyset pagination help at all?
- The customer wants to resume an export from yesterday's last cursor. What must be true of the cursor for that to be safe?
Migrate a live partitioned event table without blocking ingest
Hard · online migration · ddl locks · expand and contract · partitioning
usage_event is range-partitioned daily on ingested_at, holds roughly 250M rows per day across 400 live partitions, and is written at 10-40k rows/second. Two changes are required: quantity must move from double precision to numeric(20,6), and a new environment column must become NOT NULL with a default of 'production'. Ingest cannot stop. Give the ordered plan, naming for each step the lock it takes, what that lock blocks, and roughly how long it is held. Identify the one step that cannot be rolled back cleanly once traffic depends on it.
Approach
- Classify the two changes before planning anything. Adding a column with a non-volatile default has been metadata-only since PostgreSQL 11, so it is cheap. Changing double precision to numeric is not binary-coercible, so
alter column ... typerewrites every partition under ACCESS EXCLUSIVE and rebuilds its indexes; on this volume that is hours of blocked ingest and is simply not an option, which is why the plan is expand-and-contract rather than one statement. - Expand: add
quantity_numeric numeric(20,6)andenvironmentwith its default on the parent. Both are catalogue-only but both take a brief ACCESS EXCLUSIVE that cascades to partitions, so run each withlock_timeoutset to a second or two and retry on failure. A queued ACCESS EXCLUSIVE request blocks every reader behind it, which is how a metadata-only change turns into an outage. - Dual-write: deploy producer code that populates both columns on every insert, and leave it running before anything reads the new column. This is the step that cannot be reverted cleanly. Once readers depend on quantity_numeric, reverting the writer leaves rows with a null there, and the gap is only discoverable by re-reading the old column, which the readers have stopped doing.
- Backfill older partitions in batches keyed on the primary key, oldest first, committing every few thousand rows with a pause between batches, and skipping the partition still receiving writes until it rotates. Each batch is an ordinary UPDATE taking row locks only. The cost is bloat and WAL rather than blocking, so watch dead tuples and let autovacuum keep pace instead of wrapping 400 partitions in one transaction.
- Make NOT NULL cheap with the three-step form:
add constraint ... check (environment is not null) not valid(brief ACCESS EXCLUSIVE, no scan), thenvalidate constraint(SHARE UPDATE EXCLUSIVE, scans while reads and writes continue), thenset not null, which from PostgreSQL 12 uses the validated check and skips its own full scan. Do this per partition, then on the parent. - Switch and contract: move reads to the new column behind a flag, verify over a full period that both columns agree on freshly written rows, drop the old column (metadata-only), and only then remove the dual-write. Any index on the new column goes on with CREATE INDEX CONCURRENTLY per partition, since CIC is not supported on a partitioned parent: create the parent index with ONLY, build each child concurrently, then ALTER INDEX ... ATTACH PARTITION until the parent index becomes valid.
Follow-up
- A CREATE INDEX CONCURRENTLY fails halfway through the partition list. What state is the table in, how do you detect it, and what do you run?
- The producer computes quantity itself. What happens to a request already in flight when the dual-write deploy lands, and does it matter?
- Give two queries that prove the backfill is complete: one cheap enough to run every minute, one authoritative.
If you were tasked with optimizing an existing module for performance, what steps would you take?
Medium
Approach
- Say what you would check first and why it is the highest-information step.
- Clarify what is being asked and what a complete answer contains.
- Work from the requirement backwards to the design.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Walk me through how you would architect a solution for a system requiring high data throughput.
Medium
Approach
- State your assumptions explicitly before working the problem.
- Work from the requirement backwards to the design.
- Say what you would check first and why it is the highest-information step.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
How would you approach a specific technical bottleneck in our current architecture?
Medium
Approach
- Clarify what is being asked and what a complete answer contains.
- Say what you would check first and why it is the highest-information step.
- State your assumptions explicitly before working the problem.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Per-tenant rate limiting that holds across one hundred pods
Medium · rate limiting · hot keys · atomicity · write skew
The same gateway enforces each tenant's plan rate limit in requests/second and its monthly quota. Traffic is heavily skewed: a handful of tenants exceed all others combined, while a small tenant's requests may all land on one of the 120 pods. Both limits must hold in aggregate rather than per pod, per credential or per region, and the check may add at most 2 ms at p99. Specify the algorithm, where the state lives, the exact operation performed per request, and the behaviour when the counter store is unreachable - separately for the rate limit and the quota.
Approach
- Do the arithmetic on per-pod buckets before discarding them, and be precise about which half fails. Giving each pod rate/N keeps the sustained aggregate correct - 120 buckets of 5 rps sum to the 600 rps limit - but a tenant whose traffic all lands on one pod is throttled at 5 rps, a hundred and twentieth of what it pays for. The over-admission comes from burst depth, not sustained rate: if each pod carries a full bucket depth B, the aggregate burst is 120B against a limit that intended to allow B.
- Move the rate limit to a shared counter accessed in exactly one atomic round trip - a token bucket or GCRA as a single server-side script returning allow/deny plus a retry-after - never a read followed by a write. Keep the key tenant-scoped and region-local; a globally shared counter costs a cross-region round trip that alone exceeds the 2 ms budget, so shard the global limit per region in proportion to observed regional share and accept that a tenant whose traffic shifts regions is briefly throttled until the shards rebalance.
- Cut round trips for the largest tenants with leased tokens: a pod leases 50 at a time and spends them locally, reducing counter round trips by 50x at the cost of bounded over-admission of pods x lease_size, which is 6,000 requests in the worst case here. Shrink the lease for tenants with small limits, where that bound would swamp the limit itself.
- Treat the quota as a different problem, not a longer window. It is a count that other in-flight requests are changing, so select-the-usage-then-insert is write skew: read-committed permits it and repeatable read permits it too, because snapshot isolation is exactly what allows two transactions to read a consistent count and both write. Enforce it inside one statement - an atomic increment whose returned value is compared, or UPDATE ... SET used = used + $1 WHERE used + $1 <= limit RETURNING - or with a constraint that makes the surplus write fail.
- Write the unavailability policy per limit rather than one policy for both. The rate limit fails open to a conservative local bucket, because it protects capacity and failing closed turns a counter blip into a total outage. The quota fails open for tenants whose last known usage was well below the cap and closed for those at or above it, since that is the only decision with revenue attached, and reconciles from the usage ledger on recovery - which is why the monthly counter must be re-seedable from that ledger rather than being the only copy.
Worked solution (30 min)
- For a 600 rps limit across 120 pods, compute the sustained rate and the aggregate burst under local buckets of depth B, then the throughput of a tenant whose traffic all lands on one pod.
- Write the shared-counter operation as a single atomic script: its inputs, its return value, and the number of network round trips per request.
- Size a lease of 50 tokens: compute the worst-case over-admission as pods x lease_size and the reduction in round trips, then decide the smallest limit at which the lease is still acceptable.
- Write the two unavailability policies side by side and the reconciliation step that runs when the counter store returns.
Expected result: A per-region shared token bucket keyed (tenant, region, window) executed as one atomic script at roughly one sub-millisecond intra-region round trip; the monthly quota enforced by a single atomic increment-and-compare that is re-seedable from the usage ledger; leases only for high-limit tenants with over-admission bounded at 120 x 50 = 6,000 requests; the rate limit failing open to a conservative local bucket and the quota failing closed only above the last known cap.
Verify it yourself
- Separate the two halves of the local-bucket claim: sustained aggregate is correct at rate/N per pod, burst is not. If your answer says local buckets let a tenant exceed its sustained rate, the arithmetic does not support it.
- Re-read your quota check and confirm no branch reads the counter and then writes it. If the predicate is not inside the write or enforced by a constraint, the race is still there.
- Compare worst-case lease over-admission against the smallest tenant limit you apply leases to. If 6,000 exceeds that tenant's per-minute allowance, the lease size is wrong for that tier.
Follow-up
- One tenant is 40% of all traffic and its counter key lands on a single shard. What do you change, and what does that cost in accuracy?
- Clients receive 429s and retry. What stops the well-behaved ones from synchronising into a herd?
- A tenant reports being throttled below its limit. Which evidence do you produce, and does your design emit it today?
Invoice detail latency triples after an ORM relationship refactor
Easy · n+1 query · orm · latency regression
An invoice detail endpoint returned in 40 ms at p99 last week. After a refactor replaced a hand-written join with ORM relationship access it returns in 1.4 s, and the regression grows with the number of invoice_line_item rows on the invoice. Database CPU rose, but no statement in the slow-query log exceeds 3 ms. You have request traces with per-span SQL, the ORM statement log, and a staging copy of the data. Produce an ordered diagnostic checklist, the measurement that confirms the cause before any code change, and the fix.
Approach
- Count statements per request before reading any statement duration. A slow-query log hides this class by construction, because every individual query is fast and only their number is wrong; take one trace and count SQL spans.
- Establish proportionality rather than asserting it: sample invoices with 5, 20, 60 and 200 line items and plot statements per request against line count. A straight line of slope 1 through an intercept of one or two identifies a lazy relationship load, and no index or cache would move that line.
- Locate the emitting attribute access in the refactored code and check whether the same shape repeats one level deeper, for instance a tax or adjustment collection hanging off each line, which turns the cost quadratic.
- Fix with a bounded statement count: either one join that fetches invoice and lines together, or two statements where the second is WHERE invoice_id = $1 AND tenant_id = $2. Keep tenant_id in the predicate so the read stays tenant-scoped even though invoice_id already implies it.
- Choose between the two deliberately: the join duplicates the wide parent row across N children on the wire, the two-statement form avoids that for one extra round trip. Prefer the join for narrow parents and the split for wide ones.
- Pin it with a per-request statement-count assertion in a test that varies line count, because a latency assertion passes on a small fixture and would not have caught this.
Follow-up
- The endpoint now also needs per-line tax rows. Show the shape that keeps statement count constant instead of reintroducing the same defect one level down.
- How does this change if a transaction-pooling proxy sits between the service and the database, so each statement may land on a different backend session?
- The same page paginates invoices with LIMIT and OFFSET. Why is that a second, independent defect, and what replaces it?
Suggested practice checklist
For someone fluent in a dynamic language who has shipped real work but has never had to say what the runtime is doing underneath. The week is built on measuring and deliberately breaking things, because the questions that expose this background are the ones where the interviewer asks why a second time.
Day 1 — Measure before reasoning
- Take a slow piece of your own code, write down in advance where you believe the time goes, then profile it and record how wrong the guess was. The cost is usually an allocation you did not notice or an accidental quadratic membership test.
- Replace one list membership test inside a loop with a set and measure at a thousand, ten thousand and a hundred thousand elements, confirming the shape of the curve rather than only that it got faster.
- Write down the three quantities you can now measure instead of assert: wall time, peak memory, and call count for the function you suspected.
Deliverable: A before-and-after profile of real code plus a written note on the size of the gap between the guess and the measurement.
Day 2 — References, copies, and the bugs they produce
- Write the function with a mutable default argument, call it three times, and explain the accumulating result: the default is evaluated once when the function is defined, so every call shares one object.
- Build a nested structure, take a shallow copy, mutate an inner element, and show that both views changed, because a shallow copy duplicates the container and not the elements. Then fix it with a deep copy and state the cost you just accepted.
- Write two functions, one mutating its argument in place and one rebinding the local name, and predict the caller's view of each before running it. That single distinction produces most of the bugs that pass their tests.
Deliverable: Three small programs whose output you predicted correctly before running, each with a one-line statement of the rule underneath.
Day 3 — Types, once, in a language that checks them
- Port one module you have already written, roughly a hundred lines, into a statically typed language, and record every place the compiler demanded an answer your original had left implicit: a value that can be absent, a numeric width, a case never handled.
- Write the same signature in both languages and state what the static one guarantees before the program runs and what it does not, since it will not save you from a wrong algorithm or an index out of range.
- Write the difference between an interface satisfied by declaration and one satisfied structurally, with one case each where the other approach would miss the mistake.
Deliverable: One module in two languages plus a list of the questions the type checker forced you to answer.
Day 4 — Concurrency, starting with what actually runs at the same time
- Run the same CPU-bound function across four threads and four processes and measure both. Under the default CPython build the threaded version will not speed up, because only one thread executes bytecode at a time; the process version will. Check which build you are on first, since free-threaded builds remove that lock and change the result.
- Then run a blocking I/O workload across four threads and measure it speeding up, because the interpreter releases that lock around blocking calls, which is why treating threads as useless is wrong as a general claim.
- Build the lost update: two threads each incrementing a shared counter a hundred thousand times, and show a final value below the expected sum, because an increment is a load, an add and a store and the thread can be suspended between them. Fix it with a lock and then measure what the lock costs.
Deliverable: Three measurements, threads against processes on CPU work, threads on I/O work, and a demonstrated lost update, each with the mechanism written underneath.
Day 5 — Debugging as a procedure rather than an instinct
- Work one real failure as a bisection: find a revision or an input size where it is good and one where it is bad, halve repeatedly, and state the two assumptions bisection needs, that the property changes exactly once across the range and that the test is reliable.
- Minimise one failing input to the smallest version that still fails, and record how many rounds it took.
- Keep a hypothesis log for one bug in three columns, what I believe, what would disprove it, what I observed, and stop yourself the first time you are about to change two things at once.
Deliverable: One bug worked to root cause with a written hypothesis log and a minimised reproducing input.
Day 6 — Tests that catch the bug you are about to write
- Implement an LRU cache with a capacity bound, then write the three test cases that would catch an off-by-one in eviction: insert exactly capacity items and assert nothing was evicted, insert one more and assert the least recently used key is the one gone, and read an old key just before that insert so the eviction victim changes.
- Add a property test comparing your implementation against a deliberately slow reference, an ordered list scanned linearly, over a few thousand random operation sequences, because a slow reference finds the cases you would not have thought to write.
- Write one numeric test that fails under exact equality and passes with a tolerance, and state why the tolerance has to be relative rather than absolute once the magnitudes grow.
Deliverable: An LRU implementation with three boundary tests, one property test against a slow reference, and one tolerance-based numeric test.
Day 7 — Debug something broken, out loud
- Have someone plant three defects in a two-hundred-line program, an off-by-one, a shared mutable state bug, and a wrong error-handling path, then find them while narrating, under a fixed rule: state the hypothesis before touching anything.
- Time each one and record which tool found it, reading, a printed value, a debugger, or a test, because the question asked in interviews is how you would find it rather than what it was.
- Write the sentence you will use when you do not yet know the cause, one that names the next measurement instead of offering a guess.
Deliverable: A recorded debugging session with time-to-find per defect and the method that found each.
Outage answers are graded on instruments
Nobody is scoring your stamina at three in the morning. What carries weight is which signal told you something was wrong, what you measured before touching anything, what you rolled back versus what you fixed forward, and why you picked one. 'We restarted it and it went away' is a story about not knowing.
Describe a time you had to choose between two competing technical solutions; how did you decide?
Medium · behavioural and engineering judgement
Approach
- Close with what you would do differently, concretely.
- State the situation in two sentences and spend the rest on the reasoning.
- Name the disagreement and how you resolved it with evidence.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that again?
How do you handle technical debt while still meeting aggressive project deadlines?
Medium · behavioural and engineering judgement
Approach
- State the situation in two sentences and spend the rest on the reasoning.
- Close with what you would do differently, concretely.
- Give the blast radius: what could have broken, and what you measured.
Follow-up
- What would you do differently if you ran that again?
- What did you decide not to do, and why?
Resolve a review disagreement over a quota check
Easy · code review · isolation levels · write skew · disagreement
A colleague's pull request enforces a per-tenant quota by selecting the current count and then inserting when it is under the limit. You flag it as a race. They reply that the transaction already runs at repeatable read, so the snapshot makes it safe, and the tests pass. Walk through taking that disagreement to a resolution: what you write in the review, what you demonstrate rather than assert, which fix you propose and why, and what you do if they still disagree after all of it.
Approach
- Answer the claim precisely instead of restating your objection, because they have made a specific technical argument. In PostgreSQL, repeatable read is snapshot isolation; this is write skew, which snapshot isolation permits by design. Both transactions read a count that is stable within their own snapshot, insert disjoint rows that the other cannot see, and both commit, so the limit is exceeded by exactly the concurrency.
- Demonstrate rather than cite. Two psql sessions, both BEGIN ISOLATION LEVEL REPEATABLE READ, both select the count, both insert, both commit: it succeeds. Repeat at SERIALIZABLE and the second commit fails with serialization_failure, SQLSTATE 40001. That takes two minutes, ends the argument without anyone conceding a position, and leaves an artefact for the next reviewer.
- Offer the options with their costs rather than a verdict. Serialisable plus a retry loop on 40001 is correct but obliges every caller to retry and degrades under contention. An increment-and-compare on a counter row — update tenant_quota set used = used + 1 where tenant_id = $1 and used < limit returning used — is safe even at read committed, because a blocked updater re-evaluates the WHERE clause against the row version it finally locks, and zero rows returned means full. A unique or exclusion constraint that makes the surplus write fail is the third.
- Name the plausible non-fix explicitly, since it is what usually gets merged instead: folding the count into the insert as insert ... select ... where (select count(*) ...) < limit is still racy under read committed, because the subquery cannot see the other transaction's uncommitted rows. It looks atomic and is not.
- Say what you do if they still disagree: escalate the decision rather than the disagreement. Attach the reproduction, hand it to the service owner or a third reviewer, and state that you will not block the merge if the owner accepts the risk knowingly — and that you want that acceptance written down.
- Close with the general lesson worth leaving in the review thread: a passing suite is weak evidence for a concurrency claim because it runs one request at a time. Ask for a test that runs two.
Follow-up
- Write the counter-row version. Does your answer change if the quota counts child rows rather than a column?
- Under serialisable, who performs the retry, and what does the API client see if the retry also fails?
- This is the third disagreement with the same reviewer this month. What changes in how you review?