Interview guides · Nanonets
Nanonets Software Engineer Interview Guide
Timed coding practice for Nanonets software engineer candidates: full attempts under a clock, then scored against a rubric.
The 60-second brief
As a Software Engineer at Nanonets, you will play a critical role in building and scaling the next generation of Intelligent Document Processing (IDP) technology. Nanonets leverages advanced machine learning and deep learning models to help businesses automate document-heavy workflows, turning unstructured data into structured, actionable insights. In this role, you are not just writing standard application code; you are building the highly scalable, low-latency infrastructure required to ingest, process, and extract data from millions of complex documents daily.
Ask whether any round happens inside an existing repository instead of a blank file. Reading unfamiliar code, isolating a fault and making the smallest correct change is a different skill from writing a function from scratch, and it needs its own practice.
Focus areas
- Build at-least-once pipelines with explicit deduplication horizons
- Make every write idempotent under client retries
- Keep money in integer minor units
Overview
The engineering team at Nanonets tackles challenges at the intersection of system design, API performance, and machine learning deployment. Your work will directly impact the performance of core products, influencing how quickly and accurately global enterprises can process invoices, identity documents, and financial statements. Because the platform must handle heavy traffic spikes, complex asynchronous processing, and deep learning model inference in real time, your engineering decisions will directly dictate the reliability and speed of the entire system.
This position is ideal for engineers who thrive in fast-paced, high-ownership environments. You will be expected to move quickly, take complete ownership of your services, and design systems that are both highly performant and resilient. Whether you are optimizing API gateways, building complex priority-scheduling queues, or refining low-level system interactions, your contributions will be central to the company's technical and commercial success.
The Nanonets interview process
Nanonets candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
HR Introductory Call
You cannot drill a format you do not know, so put the preparation into material that travels. Three pieces of your own work, each rehearsed until you can take a follow-up you did not anticipate, will carry a conversation or a code walkthrough equally well. Specificity is what separates that from filler. A number needs its definition before it means anything: a p99 is over some window and measured at some hop, and a server-side figure excludes the queueing and network time a client would see. The number you cannot qualify is the one to leave out.
What to demonstrate
- Whether your examples carry detail only someone who did the work would hold, such as what the binding constraint actually was, which alternative you rejected and why it was worse, and what you measured on each side of the change
- Whether a number survives one follow-up, meaning you can say what it was measured over and whether it moved because of your change or merely alongside it
- Whether a failure is described with the specific change that followed it, rather than a lesson stated in general terms
- Whether your part in a team effort is stated accurately, including what other people did
How to prepare
- Write a page on each of three projects covering the constraint, the option you rejected, the measurement before and after, and what went wrong. Cut any line you cannot take a follow-up on, since you are writing the parts you will be pressed on rather than a summary.
- Recover the real figures while you still have access: request volume, data size, latency with its percentile and window, team size, timeline. Note where each came from, whether a dashboard, a design document or memory, and mark the estimates so you can say which they are out loud.
- Take your weakest project story to someone who works in a different area and have them ask why four times in succession. The point where you run out of answer is the part to go and re-read before the round.
Technical Evaluation Phase
Most of the time lost in this format is not lost to thinking. It goes to a standard-library call you half-remember, an off-by-one in a loop bound, and a debugging loop that mutates code at random until something passes. When output is wrong, stop re-reading the whole function: take the smallest input that reproduces it and walk the state through by hand, printing intermediates if the environment allows. Guessing at a fix without a failing case you understand is how a five-minute bug becomes twenty, and the clock does not pause while you do it.
What to demonstrate
- Whether you reach the right structure without a detour, and can write it from memory rather than only recall that one exists
- Whether overflow is considered where the language has fixed-width integers, since a signed 32-bit value stops at 2,147,483,647 and then wraps in Java, is undefined behaviour in C++, and does not arise in Python, whose integers grow instead
- Whether recursion depth is treated as a constraint on large inputs, given that CPython's default limit is 1000 frames and a deep recursion can exhaust the stack in any language where an iterative version would not
- Whether a failing case is isolated and explained before any edit is made to the code
How to prepare
- From an empty file and with no references open, implement the pieces you lean on most: a heap push and pop, an iterative DFS with an explicit stack, and a binary search whose midpoint is written lo + (hi - lo) / 2, which avoids the overflow that (lo + hi) / 2 can hit in a fixed-width integer type
- Time yourself on the ten library calls you look up most, such as sorting with a custom comparator, splitting and joining strings, and finding the next key at or above a value in an ordered map, until the lookup is gone
- Take a solution you know is broken and, before touching it, write one sentence naming the input, the expected value and the actual value. Repeat until you do it without deciding to.
Executive and Cultural Alignment
An unlabelled round is first an information problem, and the cheapest information is free. Whoever schedules it can usually tell you how long it runs, who will be in the room and what they work on, whether you will be writing code and in what environment, and whether anything is being sent beforehand. Ask in writing so the answer is on record, then prepare for the two or three formats those answers still leave open instead of betting on one. What separates a strong candidate is not guessing right; it is having an opening that works whichever one it turns out to be.
What to demonstrate
- Whether you can start work from an ambiguous brief, since tolerating a vague scope without stalling is the same thing the job asks for
- Whether the questions you asked beforehand were ones that change your preparation, such as duration, medium and who is joining, rather than ones whose answers you could not have acted on
- Whether you adapt when the round turns out to be something other than what you were told, instead of spending the first ten minutes visibly recalibrating
How to prepare
- Send one short scheduling message asking four things: how long, who is joining and what they work on, whether you will be writing code and where, and whether to prepare anything in advance. Treat a vague reply as real information, since it means the round is loosely structured and you will be shaping it yourself.
- Write one opening that works in any of the formats still open: restate in your own words what you have been asked to do, then ask which of two directions is more useful to them. Say it aloud until it stops sounding recited.
- Set up for the two most likely formats before the call starts, with a blank editor in the language you would choose and a shared document you can type into, so a format surprise costs you nothing in the first minutes
Where candidates lose points
Holding money in a floating-point type, or rounding it more than once
Binary floating point cannot represent 0.01 or 0.1 exactly, so sums drift and two code paths that should agree disagree by cents nobody can trace back. The fix is integer minor units or an exact decimal type end to end, with sub-cent rates expressed as scaled integers such as micro-units, because a per-request price genuinely is smaller than a cent. The second half of the trap is rounding position: rounding each line and then summing gives a different total from summing and rounding once, and half-up and half-even diverge systematically across many lines, so rounding must happen at one named place and every downstream reader must carry the rounded value rather than recompute it from quantity and rate.
One shared connection pool for every tenant and every query class
A single tenant with a large table and a missing index can occupy every connection with slow queries, and every other tenant then waits in connection acquisition -- a queue invisible in database metrics, because the database itself looks healthy while the application starves. Containment is bulkheads: separate pools or per-tenant concurrency caps for interactive requests, background jobs and exports, a statement timeout low enough that a pathological query dies before it accumulates, and an idle-in-transaction timeout so a stuck client cannot pin a connection and its locks indefinitely. One caveat worth knowing in advance: if a transaction-pooling proxy sits in front of the database, session-scoped behaviour changes, so session-level advisory locks and settings applied outside a transaction do not survive the way they do on a direct connection.
Designing for a scale nobody asked for
Ask for request rate, data size, read-to-write ratio and expected growth, then size the simplest option first; one relational instance on current hardware covers a large share of real workloads. Reaching for shards, queues and a cache tier before any number has been quoted reads as pattern-matching rather than judgement.
Check-then-act on shared state
Read, decide, write is not safe under concurrency unless the decision and the write are one atomic step: a unique constraint with conflict handling, a compare-and-set, or a row lock held for the whole transaction. Two requests can both pass the existence check before either inserts, which shows up as duplicate rows under load and never in a single-threaded test.
Questions & practice
Solve a classic graph traversal or interval-merging problem, optimizing for both time and space complexity.
Medium · data structures and algorithms
Approach
- Choose the data structure from the access pattern, not from familiarity.
- State the target complexity and say which constraint rules the naive version out.
- Walk one small example through your approach before writing the whole thing.
Follow-up
- What is the worst case, and how likely is it on real data?
- Which test case would catch an off-by-one here?
Implement a custom priority queue that supports dynamic weight adjustments based on user tiers and request types.
Medium · data structures and algorithms
Approach
- Name the brute-force solution and its complexity before improving on it.
- Choose the data structure from the access pattern, not from familiarity.
- State the target complexity and say which constraint rules the naive version out.
Follow-up
- Which test case would catch an off-by-one here?
- How does this change if the input no longer fits in memory?
Write a functional Python script during a live coding session to integrate with a mock API, process the JSON payload, and handle rate-limiting headers dynamically.
Medium · data structures and algorithms
Approach
- Walk one small example through your approach before writing the whole thing.
- Restate the input: its shape, its size, and what is guaranteed about it.
- Choose the data structure from the access pattern, not from familiarity.
Follow-up
- How does this change if the input no longer fits in memory?
- Which test case would catch an off-by-one here?
Hold a tenant to a trailing sixty-second request limit
Medium · sliding-window · two-pointer · rate-limiting · tenant-skew
The gateway must hold each tenant to R requests in any trailing 60 seconds, in aggregate across three regions and every pod, within a budget of under 10 ms added p99. Peak is 30,000 requests/second across 200,000 active tenants, and traffic is heavily skewed toward a handful of them. Give an exact single-process algorithm with its amortised per-request cost and its memory per tenant, then a bounded-memory approximation and the worst-case overshoot it actually admits. Say what the distributed version does when the counter store is unreachable.
Approach
- Exact, single process: a per-tenant deque of request timestamps. On arrival, pop from the front while
front <= now - 60s, then admit if the remaining length is below R and push. Each timestamp is pushed once and popped once, so the cost is O(1) amortised. The O(R) version is the one that re-filters the whole deque on every request. - Quote the memory. R = 1,000 across 200,000 active tenants is up to 2 x 10^8 timestamps at 8 bytes, about 1.6 GB, and that is the worst case rather than the mean, because the long tail of small tenants holds almost nothing. Skew helps you here and hurts you in the sharding decision.
- Bounded alternative, with its real bound stated: a fixed 60-second counter is O(1) memory but admits close to 2R across a 60-second span straddling a boundary. The weighted two-bucket estimate,
prev * (60 - elapsed)/60 + cur, is better on smooth traffic but assumes the previous window's arrivals were uniform; an adversary packing them at the end of that window is undercounted and can still approach 2R. Say that rather than calling it exact. - Token bucket is the usual gateway answer and a different contract: O(1) state per tenant (
tokens,last_refill), a sustained rate, and a deliberate burst allowance equal to the bucket size. Choose it when a burst is acceptable and the log when the limit is contractual. - Distributed: the limit is per tenant in aggregate, so a local bucket of R/N per pod is wrong in both directions under skew. A tenant landing on one pod is throttled at R/N, and a tenant spread evenly across pods exceeds R. The shared check must be a single atomic round trip, one script or one increment-and-compare, never read-then-write, and it must fit inside the 10 ms p99 budget.
- Decide the unavailable case in advance and write it down. Failing open keeps the product up and lets a tenant exceed its limit for the duration; failing closed converts a counter-store outage into a full outage. Most gateways fail open on rate limits and closed on authorisation, and those are two separate decisions made separately.
Worked solution (25 min)
- Implement the deque version and instrument the per-request pop count, then confirm total pops equal total pushes over a run.
- Generate a burst that places R requests in the last 100 ms of one minute and R more in the first 100 ms of the next.
- Run that burst through the exact deque, a fixed 60-second counter, and the weighted two-bucket estimate, recording admissions in the trailing 60 seconds at every instant.
- Size the memory as R x active tenants x 8 bytes at R = 1,000 and 200,000 tenants, and compare it against what a token bucket would need.
Expected result: The exact deque never admits more than R in any trailing 60-second window. The fixed counter admits close to 2R across the boundary. The weighted estimate lands between the two on this burst and approaches 2R once the previous window's requests are packed at its end.
Verify it yourself
- Total pops equal total pushes, which is what makes the amortised O(1) claim true rather than merely asserted.
- Sliding the whole burst by 30 seconds changes the fixed counter's result and leaves the deque's unchanged.
- Running two processes each limiting to R/2, against a tenant that sends everything to one of them, shows that tenant throttled at half its contractual limit.
Follow-up
- One tenant sends 40% of all traffic. What does that do to a single counter key, and what do you shard on instead?
- Quotas rather than rate limits: the check is
select used; if used < limit then insert. Name the isolation level that still permits the overshoot, and the two fixes. - How do you return an accurate
Retry-Afterfrom the exact algorithm without a second scan?
Migrate a live partitioned event table without blocking ingest
Hard · online migration · ddl locks · expand and contract · partitioning
usage_event is range-partitioned daily on ingested_at, holds roughly 250M rows per day across 400 live partitions, and is written at 10-40k rows/second. Two changes are required: quantity must move from double precision to numeric(20,6), and a new environment column must become NOT NULL with a default of 'production'. Ingest cannot stop. Give the ordered plan, naming for each step the lock it takes, what that lock blocks, and roughly how long it is held. Identify the one step that cannot be rolled back cleanly once traffic depends on it.
Approach
- Classify the two changes before planning anything. Adding a column with a non-volatile default has been metadata-only since PostgreSQL 11, so it is cheap. Changing double precision to numeric is not binary-coercible, so
alter column ... typerewrites every partition under ACCESS EXCLUSIVE and rebuilds its indexes; on this volume that is hours of blocked ingest and is simply not an option, which is why the plan is expand-and-contract rather than one statement. - Expand: add
quantity_numeric numeric(20,6)andenvironmentwith its default on the parent. Both are catalogue-only but both take a brief ACCESS EXCLUSIVE that cascades to partitions, so run each withlock_timeoutset to a second or two and retry on failure. A queued ACCESS EXCLUSIVE request blocks every reader behind it, which is how a metadata-only change turns into an outage. - Dual-write: deploy producer code that populates both columns on every insert, and leave it running before anything reads the new column. This is the step that cannot be reverted cleanly. Once readers depend on quantity_numeric, reverting the writer leaves rows with a null there, and the gap is only discoverable by re-reading the old column, which the readers have stopped doing.
- Backfill older partitions in batches keyed on the primary key, oldest first, committing every few thousand rows with a pause between batches, and skipping the partition still receiving writes until it rotates. Each batch is an ordinary UPDATE taking row locks only. The cost is bloat and WAL rather than blocking, so watch dead tuples and let autovacuum keep pace instead of wrapping 400 partitions in one transaction.
- Make NOT NULL cheap with the three-step form:
add constraint ... check (environment is not null) not valid(brief ACCESS EXCLUSIVE, no scan), thenvalidate constraint(SHARE UPDATE EXCLUSIVE, scans while reads and writes continue), thenset not null, which from PostgreSQL 12 uses the validated check and skips its own full scan. Do this per partition, then on the parent. - Switch and contract: move reads to the new column behind a flag, verify over a full period that both columns agree on freshly written rows, drop the old column (metadata-only), and only then remove the dual-write. Any index on the new column goes on with CREATE INDEX CONCURRENTLY per partition, since CIC is not supported on a partitioned parent: create the parent index with ONLY, build each child concurrently, then ALTER INDEX ... ATTACH PARTITION until the parent index becomes valid.
Follow-up
- A CREATE INDEX CONCURRENTLY fails halfway through the partition list. What state is the table in, how do you detect it, and what do you run?
- The producer computes quantity itself. What happens to a request already in flight when the dual-write deploy lands, and does it matter?
- Give two queries that prove the backfill is complete: one cheap enough to run every minute, one authoritative.
Enforce a concurrent-run quota that survives simultaneous requests
Medium · isolation levels · write skew · quotas · lock contention
A plan allows at most 20 concurrently running rows in job_run per tenant. The table holds run_id, tenant_id, workspace_id, status (queued, leased, running, succeeded, failed, timed_out, cancelled, lost), lease_token, leased_until, started_at and finished_at. Today the service runs select count(*) from job_run where tenant_id = $1 and status = 'running', compares the result to 20, then inserts. Under load a tenant exceeds the cap by exactly the number of concurrent requests. Name the anomaly, say which isolation levels do and do not prevent it, and give a version that holds, as SQL.
Approach
- Name it: write skew. Each transaction reads a predicate (the count of running rows), neither modifies what the other read, and both then insert rows that jointly violate an invariant no single row expresses. Read committed permits it. So does repeatable read, because snapshot isolation's first-updater-wins check fires only on conflicting row updates, and these are inserts touching disjoint rows.
- Enumerate the fixes with their real costs. SERIALIZABLE works: PostgreSQL's SSI tracks the predicate read and aborts one transaction with SQLSTATE 40001, which obliges the caller to retry and makes the abort rate rise with contention on a hot tenant. Folding the predicate into the write as
insert ... select ... where (select count(*) ...) < 20narrows the race to the statement's snapshot but does not close it under read committed. - Give the version that holds at read committed: serialise on a row both transactions must touch.
update tenant_concurrency set running = running + 1 where tenant_id = $1 and running < 20 returning runningupdates zero rows when the cap is reached, and zero rows is the rejection. This works because at read committed a blocked UPDATE re-evaluates its WHERE clause against the newly committed row; at repeatable read the same statement raises a serialisation error instead, so the isolation level changes the calling contract. - State the cost you just bought. That row is now a per-tenant serialisation point, so admission throughput for the tenant is bounded by one divided by the lock hold time; at a 2 ms hold that is roughly 500 admissions/second. Keep the critical section to the single UPDATE, with no network call or scheduling decision inside the transaction, and decrement in the same transaction that writes the terminal status.
- Close the leak the status enum implies: a run can end as
lost, so a crashed worker otherwise consumes a slot forever. Reconcile on a schedule againststatus = 'running' and leased_until < now(), and treat the counter as a fast path overjob_run, which stays the system of record.
Worked solution (25 min)
- Seed a tenant with 19 running rows, then fire 8 concurrent sessions each running the select-then-insert, and count the resulting running rows.
- Repeat at REPEATABLE READ and confirm the count still exceeds 20.
- Repeat at SERIALIZABLE, count the 40001 aborts, and note that without a retry loop those requests fail rather than queue.
- Implement the atomic counter UPDATE, re-run the 8-way test, and confirm exactly 20 running rows with zero over-admissions.
- Kill a worker mid-run, let the lease expire, and check whether the slot comes back without intervention.
Expected result: Write skew identified; read committed and repeatable read both permit it; serialisable prevents it at the price of 40001 retries; the working read-committed version is update tenant_concurrency set running = running + 1 where tenant_id = $1 and running < 20 returning running, whose zero-row result is the rejection, plus a reconciliation path for runs that terminate as 'lost'.
Verify it yourself
- At 8x concurrency against a cap of 20, running rows never exceed 20.
- The rejection returns zero rows rather than raising, so it is distinguishable from an error in the caller.
- Killing a worker does not permanently consume a slot.
- The counter and
select count(*) from job_run where status = 'running'agree after reconciliation.
Follow-up
- Write the retry loop for the SERIALIZABLE version. What does the caller see when it keeps aborting, and what bounds the retries?
- Two regions each keep a counter. What is the effective cap, and what does admission do when the counter store is unreachable?
- The cap changes mid-flight on a plan upgrade. Do running jobs get killed, and what does the counter row look like during the change?
Design a real-time document ingestion pipeline that can handle millions of uploads per day, ensuring data consistency and fault tolerance.
Medium · system design and architecture
Approach
- State the consistency you need, and where you are willing to be stale.
- Name the failure you are designing for, then the recovery path.
- Name the read and write paths separately; they rarely have the same bottleneck.
Follow-up
- What would you drop to keep the system up under load?
- What breaks first when traffic grows ten times?
Create a modular parser for document metadata that can easily be extended to support new document types and formats.
Medium · system design and architecture
Approach
- State the consistency you need, and where you are willing to be stale.
- Name the read and write paths separately; they rarely have the same bottleneck.
- Choose a partition key and say what query it makes expensive.
Follow-up
- How does this behave when that dependency is down for an hour?
- What breaks first when traffic grows ten times?
Implement a thread-safe in-memory cache with an LRU eviction policy, explaining how you would handle concurrent read and write operations.
Medium · system design and architecture
Approach
- Name the failure you are designing for, then the recovery path.
- Choose a partition key and say what query it makes expensive.
- State the consistency you need, and where you are willing to be stale.
Follow-up
- How does this behave when that dependency is down for an hour?
- What would you drop to keep the system up under load?
Build a resumable usage export the customer can reconcile against
Medium · pagination · cursors · late data
Customers reconcile invoices against usage_event (event_id uuid, tenant_id, workspace_id, environment, sku, quantity numeric(20,6), idempotency_key, occurred_at, ingested_at, source_service, request_id), partitioned daily on ingested_at. The current export is ?page=N&per_page=1000 ordered by occurred_at, and a customer syncing hourly reports rows that never appear in their export but do appear on their invoice. Design the replacement: the ordering, the cursor's contents, the index it requires, and the rule deciding where a page stops. State what the client does on a timeout and on a cursor older than retention.
Approach
- Separate the two defects in the current shape. Offset makes the database produce and discard N * per_page rows, so page cost grows linearly and a deep page degrades from milliseconds to seconds. Concurrent inserts also shift the window between requests, so a walker skips rows with no error raised anywhere, which for a customer sync is silent data loss.
- Order by ingestion, not by occurrence. During a replay events arrive hours out of occurred_at order, so a consumer holding an occurred_at high-water mark can never see a late event that falls below it; (ingested_at, event_id) is the only ordering under which 'everything after my cursor' is a complete statement.
- Page with a row comparison: where tenant_id = $1 and (ingested_at, event_id) > ($2, $3) order by ingested_at, event_id limit $4, backed by an index on (tenant_id, ingested_at, event_id). That seeks directly to the resume point, so every page costs the same regardless of depth.
- Trail the head of the table. With ingested_at defaulting to now(), which is transaction start time, a long insert transaction receives an earlier timestamp and becomes visible after a reader has already passed it. Cap each page at ingested_at <= now() - delta, with delta larger than the longest write transaction as bounded by statement_timeout and idle_in_transaction_session_timeout, or the export skips exactly the rows written under load.
- Make the cursor opaque and self-describing: base64 of the timestamp, the event id and a fingerprint of the filters, rejected when the filters differ from the current request. Return 410 with a
cursor_expiredcode once the cursor's partition has been dropped, so the client restarts from a known time instead of resuming into a hole. - Keep a page a pure GET with no server-side consumption, so a timeout is resolved by retrying the identical cursor.
Worked solution (30 min)
- Construct the failing case on paper: an event with occurred_at at 09:00 ingested at 14:00, and a consumer that read up to 10:00 at 11:00.
- Write the keyset query with the row comparison and the exact index it needs, then say which column of the index each predicate uses.
- Add the trailing-head predicate and pick delta from a named timeout setting rather than a round number.
- Define the cursor's encoded contents and the two error cases: filter mismatch and expired partition, with their status codes.
- Write the client's algorithm in four lines: request, persist cursor after processing the page, retry the same cursor on timeout, restart from a time on 410.
Expected result: Keyset pagination over (ingested_at, event_id) with an index on (tenant_id, ingested_at, event_id), a trailing head predicate sized above the longest write transaction, an opaque filter-bound cursor, and 410 for a cursor past retention.
Verify it yourself
- A late event with an old occurred_at is still delivered by the export.
- Page cost is independent of how many pages have already been read.
- The cursor persists across a client restart and survives retrying the same page.
- The tie-break column is unique, so no row can be skipped or repeated at a page boundary.
Follow-up
- The customer asks for a total count alongside the first page. What do you offer instead, and why is an exact count both expensive here and wrong by the time it is read?
- How would you let a customer re-read a window they have already consumed without giving up the forward-only cursor?
One tenant's counter writes stall the whole connection pool
Medium · lock contention · connection pool · multi-tenancy
A change that made a per-tenant usage counter correct now produces site-wide latency whenever one large tenant writes: unrelated endpoints time out waiting for a connection while database CPU stays low and no statement is slow. The change wraps the counter update in a transaction that takes SELECT ... FOR UPDATE on one row, calls an external pricing service, then updates and commits. Give an ordered checklist, the arithmetic that bounds that tenant's write rate, and three repairs with the cost each one accepts.
Approach
- Separate waiting from working. Low database CPU alongside high application latency points at a queue, so instrument connection-acquisition wait separately from query execution time; that queue forms in the application and is invisible in database metrics, which is why the database looks healthy throughout.
- Confirm the lock rather than assuming it: sample waiting sessions and group by wait event, relation and tuple. Contention concentrated on one tuple belonging to one tenant is the signature; a deadlock would instead show the database aborting transactions after its detection timeout, which is not happening here.
- Do the arithmetic out loud. Throughput on a serialised row is one divided by the lock hold time, and the hold spans the external call, so a 20 ms pricing call caps that tenant near 50 writes per second no matter how many pods run. Every waiter also holds a pooled connection while it queues, so the shared pool drains and unrelated tenants fail at acquisition.
- Repair one: shrink the critical section to a single statement with the price resolved before the transaction opens. Cost is a stale price for the duration of one request and a second round trip; benefit is a hold time measured in the database's own execution time.
- Repairs two and three change where the contention lives rather than how long it is held. Sharding the counter into per-(tenant, bucket) rows and summing on read multiplies write throughput by the shard count, at the cost of an aggregate on every read and a shard count you must size against the largest tenant rather than the median. Accumulating in memory and flushing periodically removes the per-write round trip entirely, paid for with a bounded loss window on crash, which is acceptable for a rate limiter and not for a billing counter.
- Contain independently of which repair wins: a separate pool or per-tenant concurrency cap for this write class, a statement timeout low enough that a pathological query dies before it accumulates waiters, and an idle-in-transaction timeout so a stuck client cannot pin a connection and its locks.
Follow-up
- What would a genuine deadlock look like here, which two code paths would produce one, and how does the database's response differ from what you observed?
- If a transaction-pooling proxy sits in front of the database, which of your three repairs changes behaviour, and what stops working that would have worked on a direct connection?
- The counter also enforces a quota. Why is SELECT the count and then INSERT still wrong after you have fixed the contention?
Suggested practice checklist
For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.
Day 1 — Rebuild the primitives by implementing them
- Implement a dynamic array with doubling growth and an operation counter, then change the growth rule to add a fixed sixteen slots instead, and time both for n of ten thousand, a hundred thousand and a million. The fixed-increment version resizes n/16 times at O(n) each, so its total work is quadratic; doubling is what makes append amortised constant.
- Implement a hash map with separate chaining and a load-factor resize, then insert ten thousand keys engineered to land in one bucket and record what happens to lookup time, so that average-case O(1) becomes a claim with a stated precondition rather than a reflex.
- For dynamic-array append and hash-map insert, write down which cost is amortised rather than worst-case, which single operation pays the whole bill, and what a system with a hard per-operation deadline would have to do instead.
Deliverable: Two working implementations plus a timing table showing the input at which each structure's advertised complexity stops holding.
Day 2 — Arrays under an invariant: two pointers, sliding window, binary search
- Solve longest-subarray-with-sum-at-most-K using a sliding window, then run it on an input containing negative numbers and watch it return the wrong answer: extending the window only moves the sum monotonically when every element is non-negative, and that precondition is the whole reason the technique works.
- Write the binary search that finds the first index satisfying a predicate rather than an exact value, put the loop invariant above the loop in a comment, and verify termination on the two inputs that break careless versions: the empty range, and a range where every element satisfies the predicate.
- Compute the midpoint as lo + (hi - lo) / 2 and write one line on why the obvious (lo + hi) / 2 is a genuine defect in a fixed-width integer type and a non-issue in a language with arbitrary-precision integers.
Deliverable: Three solved problems, each with its invariant written above the loop, plus one recorded input on which the sliding window is provably wrong.
Day 3 — Sorting, heaps, and the greedy argument that has to be proved
- Solve one top-k problem three ways, by full sort, by a size-k heap, and by quickselect, then write the values of n and k at which each becomes the right choice, along with quickselect's quadratic worst case and why a randomised pivot makes that unlikely rather than impossible.
- Implement bottom-up heapify and count sift-down steps to confirm it does linear work rather than n log n, because most nodes sit near the bottom of the tree and therefore move only a short distance.
- Take interval scheduling by earliest finishing time and write the exchange argument out in full: given any optimal schedule, swapping in the earliest-finishing interval keeps it feasible and no smaller. Then construct the weighted variant where that same greedy fails and name what has to replace it.
Deliverable: A three-way top-k comparison with measured crossover points, one written exchange argument, and one counterexample to a greedy rule that looks almost identical.
Day 4 — Recursion, memoisation, and the step to a table
- Take one problem with overlapping subproblems, such as edit distance or coin change, instrument the plain recursion with a call counter to show the blow-up, then add memoisation and re-count.
- Convert the memoised version to a bottom-up table and state the two properties you relied on: each subproblem's result depends only on its arguments, and the dependencies form a DAG you can enumerate in order.
- Rewrite one deep recursion with an explicit stack, then find the input length at which the original hits the interpreter's frame limit, which defaults to about a thousand frames in CPython, so you know when the rewrite is required rather than decorative.
Deliverable: One problem in three forms, naive, memoised and tabulated, with call counts for each and the input length at which recursion depth becomes the binding constraint.
Day 5 — Graphs, where most of the work is choosing the traversal
- Implement BFS and DFS over one adjacency list, then answer for each which finds a shortest path in an unweighted graph and which you would use to detect a cycle in a directed graph, including why the in-progress versus finished distinction matters for the second.
- Implement topological sort by in-degree, feed it a graph containing a cycle, and confirm the failure signature is that fewer than V nodes come out rather than an exception, then note that the order it produces is one of several valid ones.
- Run a shortest-path search on a graph with a single negative edge weight and show the wrong answer, then write the precondition Dijkstra actually needs, non-negative weights, because it finalises a node's distance the first time that node is popped, and name the algorithm you would switch to and its own limit.
Deliverable: A small graph library with BFS, DFS and topological sort, plus two inputs that produce documented wrong answers under the wrong algorithm choice.
Day 6 — One day for everything that is not an algorithm
- Sketch one system only to the depth a coding-heavy loop tends to reach: the endpoints, what the service stores, and the single query pattern that decides the schema. Stop at twenty-five minutes.
- Prepare the project answer for an interviewer who codes, which means rehearsing the two levels they push to: the specific thing you built, and why you chose that approach over the alternative they will name. Open with a number and be ready to say what it excludes.
- Prepare the answer to what you would do differently, choosing a real technical mistake with a specific fix rather than a complaint about process or staffing.
Deliverable: One design sketch at endpoint-and-schema depth, plus a project answer rehearsed to two levels of follow-up.
Day 7 — Solve out loud, under time
- Do three timed problems at twenty-five minutes each in a plain editor with no autocomplete and no execution until the end, then tally separately the failures that were syntax and the ones that were approach, because those two numbers call for different fixes.
- Narrate one solution from the first sentence, stating the approach and its complexity before writing any code, and rehearse the sentence you will use when you realise mid-solution that the approach is wrong.
- Re-solve from blank the two problems you were slowest on this week and compare the times against the day they first appeared.
Deliverable: A recording of one fully narrated solution and a tally that separates syntax failures from approach failures.
Know where your numbers came from
Bring the two or three numbers the story rests on and know how they were collected. A p99 whose timer starts inside your handler excludes the time a request spent queued, so it can sit flat while users wait longer. Give the window, the percentile and what the measurement left out, or drop the number.
How do you handle situations where you need to integrate with a poorly documented or unstable third-party API?
Medium · behavioural and engineering judgement
Approach
- Name the disagreement and how you resolved it with evidence.
- State the situation in two sentences and spend the rest on the reasoning.
- Close with what you would do differently, concretely.
Follow-up
- How did you know your change caused the improvement?
- What would you do differently if you ran that again?
Describe a time when a critical system failed in production. How did you diagnose the issue, mitigate the immediate impact, and prevent it from happening again?
Medium · behavioural and engineering judgement
Approach
- Name the disagreement and how you resolved it with evidence.
- Give the blast radius: what could have broken, and what you measured.
- Pick a story where you made the decision, not one where you watched it.
Follow-up
- What would you do differently if you ran that again?
- How did you know your change caused the improvement?
Own the incident where invoices undercounted metered usage
Medium · incident response · at-least-once · billing correction · postmortem
A metering consumer acknowledged each batch before committing the fold into usage_rollup_hourly. A rolling deploy restarted consumers mid-batch for two hours; roughly 1.4M usage_event rows were acknowledged and never folded, and 61 invoices sealed against the resulting rollups before anyone noticed. Take the owner's role. Describe an incident of comparable blast radius you owned: how it surfaced, the query that sized the loss, what you stopped first, and how the money was corrected. Give a wall-clock timeline and one thing you got wrong while it was still live.
Approach
- Open with the invariant that broke and the direction of the error, because they determine everything else: acknowledging before committing makes the consumer at-most-once, so this loses events rather than duplicating them, and loss raises no error anywhere. A listener who hears 'we lost revenue silently' knows immediately why detection took two hours.
- Size it with a stated reconciliation rather than an adjective: sum(quantity) from usage_event grouped by (tenant_id, sku, hour of occurred_at) over the window, against usage_rollup_hourly.quantity_sum on the same keys, filtered to environment='production' because staging and sandbox are metered but not billed. Then bisect by hour and tenant until single cells explain the gap. Say how long that ran and whether a replica could serve it while the incident was live.
- Separate mitigation from fix and say which came first. Mitigation is holding the sealing job, because a sealed row is frozen by design and every minute of sealing converts a recoverable rollup into an invoice correction. The fix is moving the acknowledgement after the commit, which re-introduces duplicates that the dedup check on (tenant_id, idempotency_key) must now absorb.
- State the correction path in the domain's own terms: sealed periods are never edited, so each affected tenant gets an adjustment line on the next invoice with kind='adjustment' and voided_by_line_id pointing at the line it reverses, priced against the same rate tier and carrying the watermark it priced against. That is four separate numbers — tenants affected, minor units, the cycle the adjustment lands in, and when customers were told.
- Close on one prevention control with its cost, not five: a per-hour reconciliation comparing raw sum to rollup sum that pages above a threshold. Name the threshold and the false-page rate you accepted, because a detector nobody will keep staffed is not prevention.
- Name a mistake you made inside the response window — the wrong first hypothesis, a mitigation that made it worse — rather than a design mistake from six months earlier. That is the part candidates rehearse away and interviewers weight heavily.
Follow-up
- Your fix moves the acknowledgement after the commit. What breaks now, and what absorbs it?
- One undercharged tenant has since churned. Do you bill them, and who decides?
- How would you have caught this in ten minutes instead of two hours, and what would that detector cost you in pages per week?