Checkmate Software Engineer Interview Guide

The 60-second brief

Candidate reports for this role do not say what Checkmate sells or which customers it serves, so this guide does not guess. What they do say is that a Software Engineer builds, maintains and scales the systems behind the product, works on existing services as well as new features, and covers the full lifecycle from requirements through deployment and monitoring. The reported duties include code reviews, paying down technical debt, and working with product managers and other engineers. Because the product itself is not described, ask your recruiter which services the team owns and prepare examples from your own stack that show the same skills.

This guide is for engineers preparing for a Software Engineer loop at Checkmate, at mid-level or senior. It covers the six reported questions: the principles of your primary language and how they affect performance, concurrency under high load, a production debugging story, clean and testable code, data-structure trade-offs, and a high-traffic system design. It also covers the evaluation areas candidates describe: memory management, Big O analysis, race conditions and deadlocks. Each topic gets a concrete way to answer it, plus a seven-day plan and a set of behavioral prompts for the project-history discussions.

Candidates describe a process of several weeks with multiple technical rounds: technical deep-dives across several domains, then discussions about past projects and problem-solving, with later rounds described as more senior. No round names or order are reported.

Focus areas

• In-depth Technical Knowledge
• General Programming Knowledge
• Core Computer Science Concepts

Overview

The reported questions are conceptual and experience-driven more than puzzle-driven. Candidates are asked to explain how their primary language behaves and how they use that knowledge to improve performance, how they handle concurrency in high-load environments, how they keep code clean, maintainable and testable, how data structures trade off, and how they would design a system for a high-traffic scenario. The listed deep-dive areas add memory management and optimization, Big O analysis of your own solutions, and race conditions and deadlocks. Together these point to an interview where you explain mechanisms from the inside, not recite definitions.

The stack is deliberately left open: the requirements mention proficiency in the core programming languages of the team's stack, strong data structures and algorithms, and experience with distributed systems. Cloud infrastructure (AWS or GCP), containers (Docker or Kubernetes) and high-traffic or startup experience are listed as extras. So the first preparation step is to choose the one language you will be questioned in and relearn its internals: how memory is allocated and reclaimed, what its threads or tasks actually share, which standard collections cost what, and where your own past code was slow and why.

Two question texts are generic and need your own context. The system design question arrives with a bracketed scenario that is left unspecified, so practise opening with scope questions and then run the same skeleton on several high-traffic cases (a read-heavy feed, a write-heavy event intake, a rate-limited public API). The data-structure question refers to the product's architecture, which you will not know in advance; answer by tying each structure to a concrete workload from a system you have built, and ask the interviewer what the workload looks like.

The loop also covers your project history. Candidates are told to be ready to defend the reasoning behind every technical decision on their resume, and one candidate review mentions shifting priorities and accumulating technical debt. That makes your production debugging story and your account of how you handled debt and changing scope worth rehearsing in detail, with numbers you can actually recall: what you measured, what you changed and what the result was.

Where to focus your preparation

Candidates report several weeks and multiple technical rounds, but no named stages, so the guide treats preparation as one block built around the reported topics.

Preparation focus

Candidates describe a series of technical deep-dives across several domains, followed by discussions of past projects and problem-solving, with later rounds described as more senior and ending in leadership conversations. They report being pushed on answers until their reasoning is clear. No round names, order or durations are reported, so confirm the sequence with your recruiter and whether you will be writing code. Approach it as one preparation block: know one language deeply, be able to reason about concurrency and memory, rehearse the design skeleton for a high-traffic scenario, and have a project history you can defend decision by decision.

What to demonstrate

• Depth in your primary language, including memory behaviour and performance
• Concurrency reasoning: shared state, race conditions and deadlocks
• Complexity analysis of your own solutions and data-structure choices
• Communicating technical decisions clearly, including when your assumptions are challenged

How to prepare

• Pick one language and write a one-page summary of how it allocates memory, schedules threads or tasks, and which of its collections suit which workload
• Write a small shared-counter or bounded-queue example with a race, then fix it and explain what the fix costs under load
• Run the same design skeleton (scope, read and write paths, storage, failure handling) on three different high-traffic scenarios
• Prepare a walkthrough of two projects from your resume that states the decision, the alternative you rejected and why

Where candidates lose points

Answering the primary-language question with a tour of syntax features instead of mechanisms that affect performance

Prepare three mechanisms in your chosen language (memory allocation and reclamation, how threads or tasks are scheduled, the cost of the standard collections) and one real case where you measured a slowdown and traced it to one of them.

Saying "I would add a lock" for concurrency without naming what is shared, where the race window is, or how a deadlock could form

Name the shared state and its owner, the invariant that is briefly false, and then the options: confine the data to one thread, use immutable values, or take locks in a fixed order. State the contention cost under high load and what you would measure.

Comparing data structures by Big O alone and ignoring memory overhead, cache behaviour, ordering needs, concurrent access and how the keys are distributed

For each structure you discuss, attach a workload from your own experience: read-to-write ratio, size, whether you need ordered iteration or range queries, and who accesses it concurrently. Give the complexity and then the practical reason it may not matter, or matters more.

Starting to draw boxes for the high-traffic design question before pinning down what the scenario is

Open with scope: who calls the system, how often, what a failed request costs, and what consistency is needed. Then describe read and write paths separately, name the data store and partition key, and say what breaks first at ten times the traffic and how the system behaves when a dependency is down.

Being unable to defend a project on your resume, or telling a production-debugging story with no hypothesis, evidence or decision of your own

For every project you list, write the main technical decision, the alternative you rejected and the result. For the debugging story, rehearse the sequence: symptom, stabilize, hypotheses, evidence, root cause, fix, verification, prevention.

Questions & practice

How do you handle concurrency and multi-threading in high-load environments?

Medium · Technical Depth and Proficiency

Approach

1. Start by classifying the work: CPU-bound work wants a pool near the core count, while I/O-bound work tolerates many more threads or an async event loop. Note that in standard CPython builds the GIL stops threads from running Python bytecode in parallel, so CPU-bound Python work goes to processes or native code.
2. Minimise shared mutable state: prefer immutable objects, thread confinement and message passing over queues. Where state must be shared, use the narrowest tool: an atomic or CAS for a counter, a concurrent map such as ConcurrentHashMap, and a lock only around the real critical section.
3. Name the race you guard against: check-then-act (read a balance, then write it) is unsafe without atomicity. Fix it with a lock, a compare-and-set, or in the database with a conditional UPDATE ... WHERE stock > 0, SELECT ... FOR UPDATE, or a unique constraint.
4. For deadlocks, cite the four Coffman conditions (mutual exclusion, hold and wait, no preemption, circular wait) and break circular wait with a global lock order, or use tryLock with a timeout. Diagnose with a thread dump (jstack) that shows each thread's held and awaited locks.
5. Under high load, bound everything: fixed-size pools with bounded queues and a rejection policy (fail fast or caller-runs) instead of an unbounded queue that grows memory until the process dies. Verify with stress tests and race detectors (go test -race, ThreadSanitizer).

Follow-up

• Two requests decrement the same inventory count at the same moment; how do you keep it from going below zero?
• How would you size this thread pool, and which metric tells you the size is wrong?
• A service hangs in production with low CPU; how do you confirm or rule out a deadlock?

Merge partitioned event streams into one ordered feed with bounded lateness

Hard · k-way merge · watermarks · out-of-order streams

The read-model service consumes 64 log partitions carrying about 4,000 events per second in total. Each partition is ordered within itself, but partitions drift by up to 30 seconds, and the activity feed must present a tenant's events in occurred_at order. Produce the merge. State its complexity, the buffer it requires in events and in bytes, what happens when one partition is idle, and what you do with an event that arrives after you have already emitted its position. Payloads average 1 KB.

Approach

1. Merge with a min-heap over the 64 partition heads keyed on (occurred_at, event_id): O(log P) per event and O(n log P) overall. The tie-break on event_id is what makes the output deterministic when two partitions carry the same millisecond, which matters because the feed is paginated and a non-deterministic order reorders pages under the reader.
2. Emitting the heap head is only correct once every partition has produced everything up to that timestamp, so the emit condition is a watermark: the minimum across partitions of the highest occurred_at seen, less the allowed lateness. Events are held until the watermark passes them, which is what turns individually ordered streams into a jointly ordered one.
3. Size the buffer from the lateness rather than guessing: 4,000 events per second times 30 seconds is 120,000 buffered events, and at 1 KB each about 120 MB of heap. That number is the real price of the ordering guarantee and belongs in front of whoever asked for it.
4. Handle the idle partition explicitly, because it fails the feed rather than corrupting it: a partition with no traffic never advances its own maximum, so the watermark freezes and output stops entirely. Either every partition emits a periodic idle marker carrying the broker's current time, or the watermark falls back to wall clock for a partition silent beyond a threshold.

Follow-up

• The lateness budget is raised to five minutes. What is the new buffer, and what besides memory changes?
• The consumer restarts. Where does it resume from, and what does the feed look like for the first 30 seconds?

Canonicalise a request body into a stable idempotency fingerprint

Medium · parsing · canonicalisation · hashing

idempotency_key.request_fingerprint is a SHA-256 over the method, path and canonicalised body, and a retry whose fingerprint differs must be rejected with 422 rather than served the stored response. Write the canonicaliser. Bodies are JSON up to 256 KB nested at most 32 levels; clients vary key order, whitespace and unicode escaping, and some send 64-bit ids as JSON numbers. Produce a deterministic byte string such that semantically identical bodies match and any semantic difference does not. State your complexity and name two normalisations you refuse to perform.

Approach

1. Parse once into a tree, then re-serialise under fixed rules: object keys sorted, array order preserved, one escaping convention, no insignificant whitespace. Parsing is O(n) and sorting keys is O(k log k) per object, so O(n log n) overall with O(depth) stack, and the 32-level cap is enforced during parsing because hostile nesting is how a canonicaliser becomes a stack overflow.
2. Sort keys by their UTF-8 bytes and say why the obvious implementation is wrong in some runtimes: a default string comparison that orders by UTF-16 code units places surrogate pairs, meaning code points from U+10000 up, below U+E000 to U+FFFF, which is not UTF-8 byte order, so two services written in different languages disagree on the same document.
3. Do not re-encode numbers through a double. IEEE-754 binary64 represents integers exactly only up to 2^53, so normalising a 19-digit id through a float changes it, and 1 against 1.0 cannot be reconciled without deciding whether they are the same value. Preserve the literal token, and require ids as strings at the API boundary if you want them comparable.
4. Reject duplicate keys rather than picking one. JSON permits them and parsers disagree, most keeping the last, so any choice you make ties the fingerprint to a parser detail that the code handling the request does not necessarily share.

Follow-up

• A client sends the same logical request with an extra field your API ignores. Same key, different fingerprint, so you return 422. Is that the right answer?
• Where does the fingerprint get computed relative to request decompression and the body-size limit?

Stop tag and share joins from fanning out a page

Medium · join fan-out · semi-joins · many-to-many

resource_tag is (resource_id, tag_id) with PK (resource_id, tag_id); resource_share is (resource_id, shared_with_user_id, permission). The tagged-and-shared listing inner-joins resource to both, filters tenant_id, tag_id = ANY($2) and shared_with_user_id = $3, orders by updated_at DESC and takes 50. Pages come back with fewer than 50 distinct resources and the total in the header is far too high. Explain the row multiplication, rewrite both the page query and the count query so each is correct, and name the index each one needs. PostgreSQL 16.

Approach

1. Do the arithmetic against the predicates that are actually there. An inner join emits one row per matching child row, and both joins are filtered: tag_id = ANY($2) admits only the requested tags, shared_with_user_id = $3 admits one user's share rows. So a resource holding three of the requested tags and shared with $3 once yields three rows, not one — the multiplier is its count of matching tags times its share rows for that single user, and that second factor is 1 unless the table admits duplicate (resource_id, shared_with_user_id) pairs. LIMIT 50 then limits rows rather than resources, and COUNT(*) counts pairs — the header is the product, not the population.
2. Reject DISTINCT as the fix. It deduplicates after the product has been built, so the planner must materialise and sort the fanned-out set before the LIMIT can apply, and it leaves any SUM or AVG in the same select list wrong.
3. Rewrite both filters as semi-joins, keeping resource as the only row source: AND EXISTS (SELECT 1 FROM resource_tag rt WHERE rt.resource_id = r.resource_id AND rt.tag_id = ANY($2)) and the same shape against resource_share. A semi-join stops at the first match per resource and preserves the driving index order, so ORDER BY updated_at DESC, resource_id DESC LIMIT 50 still stops after 50 rows.
4. Count with the same predicates and no join at all: SELECT count(*) FROM resource r WHERE r.tenant_id = $1 AND r.status = 'active' AND EXISTS (...) AND EXISTS (...). Nothing multiplies a resource, so the number is the population.

Follow-up

• The filter changes from 'any of these tags' to 'all of these tags'. Rewrite it and state what it costs relative to the ANY form.
• A resource can be shared with the same user twice under different permissions. Does your count change, and should it?

Hold a per-tenant active cap against concurrent creates

Hard · write skew · isolation levels · constraints

A tenant on the standard plan may hold at most 50 resources with status='active'. The create handler runs SELECT count(*) FROM resource WHERE tenant_id = $1 AND status = 'active', compares to 50, then inserts. Two creates arrive 3 ms apart on different instances and the tenant lands at 51. Name the anomaly, say whether PostgreSQL 16 READ COMMITTED or REPEATABLE READ prevents it and why, then give an implementation that holds the cap at READ COMMITTED with the exact statements. Finally, say what changes when the cap is 'at most one running export per tenant' on job_run.

Approach

1. Name it: write skew. The two transactions read an overlapping set and write disjoint rows, so there is no row-level conflict for the engine to detect and each commit is individually legal.
2. Rule out the levels precisely. READ COMMITTED takes a fresh snapshot per statement and takes no lock on the counted rows, so both see 49. PostgreSQL's REPEATABLE READ is snapshot isolation: it removes non-repeatable reads and phantoms within the snapshot but still admits write skew, because the anomaly is not a re-read of a changed row, it is a read of a set that a concurrent transaction invalidates. Only SERIALIZABLE closes it, by tracking the read dependency and aborting one transaction with SQLSTATE 40001 — a guarantee that exists only if the application re-runs the whole transaction from the read.
3. Convert the set predicate into a single-row conflict: keep tenant.active_resource_count and run UPDATE tenant SET active_resource_count = active_resource_count + 1 WHERE tenant_id = $1 AND active_resource_count < 50 in the same transaction as the INSERT. Zero affected rows is the cap, returned as 409. The row lock serialises the decision at any isolation level, and contention is bounded to one tenant's row — which is also the fair-scheduling unit, unlike a global counter that would convoy every tenant behind one row.
4. State the cost you just took on: a counter is a second source of truth that can drift, so every path that changes status must adjust it inside the same transaction, and a periodic reconciliation has to exist, with resource_revision as the authority for what the count should have been.

Follow-up

• A resource moves from archived back to active. Which statements change, and what breaks if the counter update and the status change land in different transactions?
• The cap becomes plan-dependent and a plan can change mid-month. Where does the number 50 live, and who reads it?

Explain the core principles of your primary programming language and how you apply them to optimize performanc

Medium · Technical Depth and Proficiency

Explain the core principles of your primary programming language and how you apply them to optimize performance.

Approach

1. Pick one language and name its execution and memory model before any tips: for Java, bytecode JIT-compiled by HotSpot with a generational garbage collector; for CPython, reference counting plus a cycle collector and the GIL; for Go, goroutines on a scheduler with escape analysis that decides stack or heap allocation.
2. Put measurement first: profile before changing anything (async-profiler or JFR for Java, py-spy or cProfile for Python, pprof for Go), find the hot path, and fix algorithmic complexity before micro-tuning. An O(n^2) loop beats any constant-factor trick once n grows.
3. Give language-specific fixes you have actually applied: avoid string += in a loop (Java needs StringBuilder; in Python use ''.join, since repeated += can go quadratic), avoid boxing in hot collections (int[] over ArrayList<Integer>), preallocate slices or lists when the size is known, and reuse buffers to cut allocation rate and GC pressure.
4. Explain data locality: contiguous arrays of primitives are cache-friendly, while pointer-heavy structures (linked lists, maps of small objects) cause cache misses even when the Big O looks the same.
5. Cover the measurement traps: JIT warm-up and dead-code elimination make naive Java microbenchmarks lie (use JMH), and a benchmark on a laptop says little about p99 latency under production load. Close with a before/after number from your own work.

Follow-up

• What does the garbage collector do when your allocation rate spikes, and how would you see that in production metrics?
• Walk me through one optimisation a profiler justified for you: what did it show, and what were the before and after numbers?
• When did an optimisation make the code harder to maintain, and was it worth keeping?

How do you approach writing clean, maintainable, and testable code?

Medium · Technical Depth and Proficiency

Approach

1. Separate decisions from side effects: keep business logic in pure functions that take values and return values, and push I/O (database, HTTP, clock, randomness) to a thin outer layer. Pure logic is testable without mocks.
2. Make dependencies explicit through constructor or parameter injection, so a test can pass a fake repository, a fixed clock or a seeded random source. Hidden globals and singletons are what make code hard to test.
3. Describe the test mix: many fast unit tests on logic, fewer integration tests against a real database (for example in a container) for queries and transactions, and a small number of end-to-end tests. Test observable behaviour rather than private methods, so refactors do not break tests.
4. Name the habits that keep code readable: functions with one reason to change, names that state intent, small diffs in review, and formatters, linters and type checkers enforced in CI so review time goes to design rather than style.
5. State the trade-offs: over-abstracting early adds indirection nobody needs, and mocking every collaborator couples tests to implementation. For untested legacy code, write characterization tests that pin current behaviour before refactoring.

Follow-up

• Here is a function that reads the current time and writes to the database; how would you restructure it to unit-test it?
• When is a mock the wrong tool, and what would you use instead?
• Your suite has grown slow and a few tests fail intermittently; what do you do first?

Can you explain the trade-offs between different data structures in the context of our product architecture?

Medium · Technical Depth and Proficiency

Approach

1. Since the product's access patterns are not given, ask for them first: which operations dominate (point lookup, range scan, insert, delete, ordered iteration, top-k), how large the data is, and whether it fits in memory. Then choose the structure from those operations.
2. Compare the core options with correct costs: a hash map gives O(1) average lookup and insert but O(n) worst case under collisions and no ordering; a balanced tree (TreeMap, red-black tree) gives O(log n) operations plus ordered iteration and range queries; a dynamic array gives O(1) indexing, amortised O(1) append and O(n) middle insertion.
3. Add the specialised structures and when they win: a binary heap for priority scheduling or top-k (O(log n) push and pop, O(1) peek); a trie for prefix lookup and autocomplete; a Bloom filter for cheap membership checks with no false negatives and a tunable false-positive rate.
4. Connect it to storage: B-tree indexes serve point and range queries for read-heavy tables, while LSM-tree stores (RocksDB, Cassandra) favour write throughput at the cost of compaction and read amplification.
5. Bring in constants and memory, not only Big O: contiguous arrays are cache-friendly, linked structures cost pointer overhead and cache misses, and a hash map's load factor trades memory for collision rate. Show one composed example, such as an LRU cache built from a hash map plus a doubly linked list for O(1) get and put.

Follow-up

• Implement an LRU cache with O(1) get and put: which two structures, and how do they stay in sync?
• Hash map lookups got slower as the data grew; what could cause that?
• When would you pick a sorted index over a hash index in a database, and what does each make expensive?

How would you design a system to handle [specific high-traffic scenario]?

Medium · Problem-Solving and System Architecture

Approach

1. The scenario in this reported question is left open, so pin it down first: peak versus average requests per second, read-to-write ratio, payload and data size, the latency target (p99, not average), and what may be stale. Do a quick back-of-envelope estimate of throughput and storage before drawing boxes.
2. Lay out the request path: CDN for static and cacheable responses, a load balancer in front of stateless application servers that scale horizontally, and a cache-aside layer (for example Redis) for hot reads with TTLs. Guard against cache stampedes with request coalescing or jittered expiry.
3. Scale the data tier deliberately: read replicas for read-heavy traffic (and say how replication lag affects read-your-own-writes), sharding by a key that spreads load evenly, and a plan for hot keys such as replicating or splitting a single popular item.
4. Absorb write spikes asynchronously: put a durable queue (Kafka, SQS) between intake and processing, make consumers idempotent with an idempotency key because delivery is at-least-once, and scale consumers on queue depth.
5. Protect the system under overload: token-bucket rate limiting, timeouts on every call, retries with exponential backoff and jitter, circuit breakers on dependencies, and graceful degradation (serve cached or partial results). Close with what you would monitor (latency percentiles, error rate, saturation) and how you would load-test the design.

Follow-up

• One item suddenly receives a large share of all traffic; what changes in your design?
• The cache cluster restarts and comes back empty; what happens to the database, and how do you prevent it from falling over?
• Clients retry on timeout; how do you make sure a write is applied only once?

Edge instances grow 400 MB per hour until the nightly restart

Hard · memory leak · heap analysis · metric cardinality

Edge API instances start at 700 MB resident and grow about 400 MB/hour; a nightly rolling restart has hidden it for weeks. Growth continues unchanged when request rate halves overnight, p99 degrades in the last hours before an instance is recycled, and heap used immediately after a forced full GC rises monotonically. The service holds no product state. Name the discriminating measurement that separates the plausible causes, give the most likely cause, and give the fix and how you would verify it.

Approach

1. Separate resident memory from live heap first, because they fail differently. Resident size can grow from fragmentation, native buffers or thread stacks while the heap is flat; heap used after a full GC rising monotonically is the measurement that says objects are reachable and not being released. You already have it, so this is retention, not fragmentation, and that closes off half the candidate list.
2. Use the rate's independence from traffic as the discriminator. Growth that continues at half the request rate rules out per-request objects that are merely slow to collect and points at a structure that grows with distinct values observed rather than with call volume. Write the candidates that have that property: a metrics registry keyed on a high-cardinality label, an unevicted cache, an interner, a per-key lock map.
3. Take two heap snapshots an hour apart and diff by retained size, reading the dominator tree, not by allocation count or instance count. Expect one root holding a map with millions of entries, then follow the reference chain to the code that inserts and never removes. Allocation profilers point at churn, which is the wrong signal here.
4. The candidate that fits this service is an observability label carrying an identifier, such as a request path recorded before templating so that /v1/resources/48213 becomes its own metric series. That grows with distinct ids seen, is independent of rate, and explains the late p99 degradation, since GC cost rises with the size of the live set.

Follow-up

• Post-GC heap is now flat but resident size still creeps. What are you looking at, and does it matter?
• How would you have detected this before an OOM, given the nightly restart masked the trend?

Suggested practice checklist

The plan is built around the six reported questions, the memory, complexity and race-condition areas candidates describe, and the distributed-systems experience the role lists. It ends with a project walkthrough and a mock, since no round sequence is reported.

Day 1 — Primary language internals and performance

• Choose the one language you will be questioned in and write a one-page summary of memory allocation and reclamation, the threading or task model, and the cost of its main collections
• Answer aloud the question about the core principles of your primary language and how you apply them to optimize performance, using one real optimization you made
• Write down the measurement you took before and after that optimization

Deliverable: A one-page language summary, a recorded spoken answer to the language question, and a before-and-after measurement for one past optimization.

Day 2 — Concurrency, race conditions and deadlocks

• Write a small shared-counter and a bounded-queue example in your language that shows a race, then fix each with a different technique (lock, confinement, atomic operation)
• Write a two-lock deadlock example and fix it with a fixed lock order; note the conditions that make deadlock possible
• Answer aloud how you handle concurrency and multi-threading in high-load environments, naming what is shared and who owns it

Deliverable: Two race examples with fixes, one deadlock example with its fix, a spoken concurrency answer and a written diagnosis for the memory-growth exercise.

Day 3 — Data structures and complexity in context

• Write a table of the collections you use most (array or list, hash map, balanced tree, heap, queue) with complexity, memory overhead and the workload each suits
• Answer aloud the data-structure trade-offs question, tying each structure to a workload from a system you have built

Deliverable: A workload-annotated collections table, a spoken trade-offs answer, and a solved stream-merge exercise with stated complexity.

Day 4 — Clean, maintainable and testable code

• Take one function you wrote that is hard to test, refactor it so its dependencies are passed in, and write three tests including an edge case
• Answer aloud how you approach writing clean, maintainable and testable code, using that refactor as your example

Deliverable: A refactored function with three tests, a spoken clean-code answer, a tested fingerprint function and a written review comment.

Day 5 — High-traffic system design

• Write a scope checklist for the bracketed high-traffic scenario: callers, request rate, failure cost, consistency need, read versus write ratio
• Design three scenarios in turn (a read-heavy feed, a write-heavy event intake, a rate-limited public API) using one skeleton: scope, read path, write path, storage and partition key, failure handling
• Answer aloud the high-traffic design question for one scenario, stating what breaks first at ten times the traffic

Deliverable: A scope checklist, three one-page designs on one skeleton, a spoken design answer and two solved SQL exercises.

Day 6 — Production debugging story and project history

• Write your production-debugging story in order: symptom, stabilization, hypotheses, evidence, root cause, fix, verification, prevention, with real numbers
• Rehearse it aloud, then answer the question about describing a time you debugged a complex production issue and what your process was
• Write a decision log for two projects on your resume: the decision, the alternative you rejected, the result, and what you would change

Deliverable: A written debugging story, a spoken version, a two-project decision log and a written answer to the reversal prompt.

Day 7 — Full mock and questions for your interviewers

• Run one spoken mock covering the language question, a concurrency question, a data-structure trade-off and a design scenario you have not practised, and note where your explanation went vague
• Fix the two weakest answers and repeat them aloud
• Ask your recruiter for the round sequence and whether you will write code, then write three questions about how the team handles code review, technical debt and changing priorities

Deliverable: A completed mock with a list of the two weakest answers and their fixes, plus the recruiter's confirmation of the format and three written questions for interviewers.

Production debugging and project ownership

Candidates report conversations about past projects and problem-solving methodology, and the one reported behavioral question asks for a production debugging story. Candidates are also told to be ready to explain the reasoning behind each technical decision on their resume. For this role that means a story you led, with the diagnosis, the decision, the measurement and the aftermath. One candidate review mentions shifting priorities and accumulating technical debt, so prepare an example about handling scope changes and debt as well.

1. Walk through a project on your resume and explain the main technical decision, the alternative you rejected and why.
2. Describe a technical decision you made and later reversed. What did you believe at the time, what changed your mind and what did the reversal cost?
3. Tell me about a time you disagreed with a colleague in code review and how it was resolved.
4. Tell me about a time priorities shifted while you were building a feature and how you handled the technical debt that resulted.
5. Describe a time you explained a technical trade-off to a product manager or other non-technical stakeholder.

Describe a time you had to debug a complex, production-level issue. What was your process?

Medium · Technical Depth and Proficiency

Approach

1. Choose an incident where you ran the investigation or made the key call, not one you only observed. Open with two sentences: what broke, who was affected and how it was noticed (an alert, a customer report, a dashboard). Spend the rest of the answer on your reasoning.
2. Walk the sequence in order: stabilize first (roll back, disable a flag, shed load) so users stop being hurt, then investigate. Say what evidence you gathered at each step, for example logs, metrics, traces, a heap or thread dump, or a diff of what changed before the first error, and which hypotheses each observation ruled out.
3. Make the cause concrete and technical. Stories that fit the reported interview areas work well: a race condition that only appeared under concurrent load, a deadlock from inconsistent lock ordering, or memory growth traced to an unbounded cache or a leaked resource. Explain how you reproduced it, or why you could not and what you did instead.
4. Cover the fix and its verification: the change itself, how you showed it worked (a metric returning to baseline, a regression test that failed before the fix) and what you added to stop a repeat, such as a test, an alert or a runbook step. Quote real numbers.
5. Avoid the common failure of a ticket-by-ticket timeline with no decision in it, or one that blames another team. State what you got wrong along the way, which shows the process was honest.

Follow-up

• How did you decide between rolling back and fixing forward, and what would have changed that call?
• How did you know this change caused the improvement and not something else deployed at the same time?
• What would you do differently the next time, and what did you change in the team's process afterwards?

Reverse your own decision and price the reversal

Medium · reversibility · measurement · migrations

Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.

Approach

1. State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
2. Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
3. Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
4. Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.

Follow-up

• What in that decision was irreversible, and did you know it was irreversible when you made it?
• How did you tell the people who had already built on top of the original decision?

Turn a code review disagreement into a decision

Easy · code review · optimistic concurrency · communication

A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.

Approach

1. Sort the disagreement before writing anything. A silently discarded write is a correctness claim about data; the choice between 409 and 412 is taste. Only the first justifies blocking a merge, and saying which one you are doing is most of the value of the comment.
2. Make the claim reproducible in the comment itself with an interleaving rather than a principle: A reads version 7, B reads version 7, B commits version 8, A's predicate matches zero rows, A is told it succeeded and A's edit is gone.
3. Offer the alternative with its cost attached: return 409 carrying the current version and the revision that won, so the client can re-read and re-apply. Note that automatic retry is not the fix, because a retry re-reads the winner's state and reapplies an intent formed against data that no longer exists.
4. Apply an escalation rule you can state: two round trips on the thread, then a call, and the service's owner decides rather than the reviewer. A reviewer who cannot be overruled is a bottleneck with extra steps.

Follow-up

• Where would you put the test that fails if someone reintroduces the swallowed zero rowcount?
• The author says clients cannot handle a 409. How do you check whether that is true?