cswork / libraryOpen practice workspace

Interview guides · Ema

Ema Machine Learning Engineer Interview Questions & Guide 2026

Ema Machine Learning Engineer interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.

The 60-second brief

At Ema, a Machine Learning Engineer is not just building standard predictive models; you are architecting the next generation of enterprise intelligence. Ema is pioneering the "Universal AI Employee," a highly secure, agentic AI platform designed to automate complex, repetitive workflows across diverse enterprise SaaS systems. This requires moving beyond simple wrapper APIs to design and deploy a sophisticated mixture of expert models, combining Large Language Models (LLMs), Small Language Models (SLMs), and custom domain-specific models.

State every complexity claim with the assumption sitting under it. Hash lookup is O(1) on average and only for a hash that spreads your actual keys; comparison-based sorting cannot beat n log n, though counting or radix sort can when the keys are bounded integers.

Focus areas

  • Bound every outbound call with a timeout
  • Choose indexes from the query's access path
  • Paginate large result sets with keyset cursors

Overview

The impact of this role is immense. You will build systems that reason, plan, retrieve information, use APIs, and execute tasks autonomously while maintaining strict enterprise-grade security and reliability. Whether you are optimizing low-latency inference, building advanced Retrieval-Augmented Generation (RAG) pipelines, or orchestrating multi-agent frameworks, your work directly defines how thousands of enterprise employees interact with AI.

This is a highly collaborative, fast-paced startup environment founded by former executives from Google, Coinbase, and Okta. Working here means solving cutting-edge AI challenges at scale, where your engineering decisions directly influence the product roadmap and the future of autonomous work.

The Ema interview process

Ema candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Standard Screening

An unlabelled round is first an information problem, and the cheapest information is free. Whoever schedules it can usually tell you how long it runs, who will be in the room and what they work on, whether you will be writing code and in what environment, and whether anything is being sent beforehand. Ask in writing so the answer is on record, then prepare for the two or three formats those answers still leave open instead of betting on one. What separates a strong candidate is not guessing right; it is having an opening that works whichever one it turns out to be.

What to demonstrate

  • Whether you can start work from an ambiguous brief, since tolerating a vague scope without stalling is the same thing the job asks for
  • Whether the questions you asked beforehand were ones that change your preparation, such as duration, medium and who is joining, rather than ones whose answers you could not have acted on
  • Whether you adapt when the round turns out to be something other than what you were told, instead of spending the first ten minutes visibly recalibrating

How to prepare

  • Send one short scheduling message asking four things: how long, who is joining and what they work on, whether you will be writing code and where, and whether to prepare anything in advance. Treat a vague reply as real information, since it means the round is loosely structured and you will be shaping it yourself.
  • Write one opening that works in any of the formats still open: restate in your own words what you have been asked to do, then ask which of two directions is more useful to them. Say it aloud until it stops sounding recited.
  • Set up for the two most likely formats before the call starts, with a blank editor in the language you would choose and a shared document you can type into, so a format surprise costs you nothing in the first minutes

Deep-Dive Technical Assessment

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

  • In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
  • In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
  • Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

  • Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
  • Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
  • For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Conversations with Leadership

An unlabelled round is first an information problem, and the cheapest information is free. Whoever schedules it can usually tell you how long it runs, who will be in the room and what they work on, whether you will be writing code and in what environment, and whether anything is being sent beforehand. Ask in writing so the answer is on record, then prepare for the two or three formats those answers still leave open instead of betting on one. What separates a strong candidate is not guessing right; it is having an opening that works whichever one it turns out to be.

What to demonstrate

  • Whether you can start work from an ambiguous brief, since tolerating a vague scope without stalling is the same thing the job asks for
  • Whether the questions you asked beforehand were ones that change your preparation, such as duration, medium and who is joining, rather than ones whose answers you could not have acted on
  • Whether you adapt when the round turns out to be something other than what you were told, instead of spending the first ten minutes visibly recalibrating

How to prepare

  • Send one short scheduling message asking four things: how long, who is joining and what they work on, whether you will be writing code and where, and whether to prepare anything in advance. Treat a vague reply as real information, since it means the round is loosely structured and you will be shaping it yourself.
  • Write one opening that works in any of the formats still open: restate in your own words what you have been asked to do, then ask which of two directions is more useful to them. Say it aloud until it stops sounding recited.
  • Set up for the two most likely formats before the call starts, with a blank editor in the language you would choose and a shared document you can type into, so a format surprise costs you nothing in the first minutes

Where candidates lose points

Shipping a migration and the code that depends on it as a single change

During any rolling deploy, and for as long as a rollback remains possible, old and new code execute against the same schema at the same time. A migration that drops or renames a column breaks every instance that has not restarted yet, and code that requires a column the migration has not applied breaks every instance that restarted early. The discipline is expand then contract: add the new column nullable, write both shapes, backfill in batches, move reads across once the backfill is verified, and only then stop writing the old shape and drop it - four deploys, usually spread over days. It feels disproportionate until the first rollback, at which point it is the only reason the previous version still runs.

Running a schema change as though the lock lasts as long as the statement

In PostgreSQL an ALTER TABLE that needs an ACCESS EXCLUSIVE lock must first wait for every open transaction touching that table, and while it waits, later queries needing a conflicting lock queue behind it rather than overtaking it. A DDL statement that would execute in milliseconds, issued while a thirty-second analytics query is open, therefore stalls all traffic on that table for thirty seconds: the outage length is set by the longest open transaction, not by the change. The defences are specific and worth knowing by name - set lock_timeout low and retry rather than queue, add columns without a volatile default so no table rewrite occurs (from version 11 a non-volatile default is a metadata-only change), build indexes with CREATE INDEX CONCURRENTLY while accepting that it cannot run inside a transaction block and leaves an invalid index behind if it fails, and add constraints as NOT VALID followed by a separate VALIDATE CONSTRAINT, which takes a weaker lock.

Arguing past a hint

When the interviewer asks what happens for a particular input or floats a different data structure, stop and take it seriously; it is almost always a correction rather than idle curiosity. Talking over it converts a recoverable wrong turn into a data point about how you handle review.

A cache with no invalidation story

Say how an entry goes stale, how long you can serve it stale, and what happens when many requests miss the same key at the same instant. One popular key expiring under load sends every concurrent request to the origin together; single-flight coalescing, jittered expiry, or serving stale while revalidating are the standard answers.

Questions & practice

How do you design a system that dynamically routes user queries to either a cheap SLM or an expensive LLM based on task complexity?

Medium · machine learning fundamentals

Approach

  1. State the learning problem: the label, the unit of prediction and how the model is used.
  2. Name the simplest model that could work and what would make you move past it.
  3. Say how you would validate it, and where leakage could enter the split.

Follow-up

  • What changes if the classes are heavily imbalanced?
  • Where could label leakage enter this setup?

Write a Python function to implement a custom beam search decoder for a language model.

Medium · machine learning fundamentals

Approach

  1. Name the simplest model that could work and what would make you move past it.
  2. Pick the metric from the cost of each error type, not from habit.
  3. Say how you would validate it, and where leakage could enter the split.

Follow-up

  • Where could label leakage enter this setup?
  • What changes if the classes are heavily imbalanced?

Implement an efficient, thread-safe cache in Python for storing vector embedding results.

Medium · machine learning fundamentals

Approach

  1. Say how you would validate it, and where leakage could enter the split.
  2. Name the simplest model that could work and what would make you move past it.
  3. State the learning problem: the label, the unit of prediction and how the model is used.

Follow-up

  • How would you know the model is overfitting?
  • What changes if the classes are heavily imbalanced?

Explain the difference between fine-tuning an LLM using LoRA versus full parameter fine-tuning. When would you choose one over the other?

Medium · machine learning fundamentals

Approach

  1. State the learning problem: the label, the unit of prediction and how the model is used.
  2. Say how you would validate it, and where leakage could enter the split.
  3. Name the simplest model that could work and what would make you move past it.

Follow-up

  • How would you know the model is overfitting?
  • Where could label leakage enter this setup?

Solve a medium-to-hard algorithmic problem involving graph traversal (often mapping to agentic planning or dependency resolution).

Medium · coding and algorithms

Approach

  1. Choose the data structure from the access pattern, not from familiarity.
  2. State the target complexity and say which constraint rules the naive version out.
  3. Name the brute-force solution and its complexity before improving on it.

Follow-up

  • Which test case would catch an off-by-one here?
  • How does this change if the input no longer fits in memory?

Diff a projection against the primary without per-row point reads

Hard · reconciliation · range hashing · throttling

The listing projection has drifted and some rows show a stale version. The primary holds 40,000,000 resource rows across 12,000 tenants while serving 1,200 writes and 14,000 reads per second. The obvious repair, reading each resource row and comparing its version against the projection, is correct and would eventually finish. Explain precisely why it is unacceptable here, then give a diff that finds the differing rows, state its complexity, and make it safe to run against a live primary. Replication lag is usually under 100 ms and is not bounded.

Approach

  1. Quantify the naive cost rather than calling it slow: 40,000,000 point reads at even 0.5 ms each is over five hours serialised, and the only lever is concurrency, which is exactly what you cannot spend. The primary's pool is sized for the write path, and 40,000,000 random reads evict the buffer cache that sustains the 85 percent cache hit rate, so the audit degrades the system it is auditing.
  2. Replace random access with one ordered pass per side. Both sides can be read in (tenant_id, resource_id) order, which is a sequential scan on each and a merge join in O(n) time and O(1) memory. For a dense diff that is the whole answer, and it reads the primary once instead of 40,000,000 times.
  3. For the expected sparse case, compare range hashes instead of rows: partition the key space, compute per range an order-independent aggregate over hash(resource_id, version), compare aggregates, and descend only into ranges that differ. With d differing rows and branching factor B, at most d ranges mismatch per level, so the drill-down examines O(d log_B(n/d)) ranges and reads full rows only in mismatching leaves.
  4. Aggregate with a sum modulo 2^64 or a multiset hash, never XOR. XOR is order-independent but self-cancelling, so two rows wrong in the same way, or a row duplicated on one side, leave the range aggregate matching and the range is declared clean.
  5. Pin the comparison to a point in time or it reports lag as drift: consider only rows whose updated_at is older than now minus a lag margin, and re-check each candidate mismatch individually before repairing. At 1,200 writes per second a diff without this reports thousands of false positives, and an unattended repairer would then overwrite live rows with stale values.
  6. Make the run resumable and throttled: batch by range key, persist the last completed range, and watch a signal such as replica lag or primary CPU, pausing rather than pressing on. A reconciliation that cannot be stopped and resumed gets killed halfway and restarted from zero, which is how a repair becomes an incident.

Worked solution (35 min)

  1. Compute the naive cost explicitly at 40,000,000 reads and 0.5 ms each, then at 100 concurrent, and state what those connections do to a pool already carrying 1,200 writes per second.
  2. Write the merge-join version over (tenant_id, resource_id) and state its memory.
  3. Define the range aggregate: the range key, the per-row hash input, and the combining function, with one sentence excluding XOR.
  4. Work an example with 40,000,000 rows, branching factor 256 and 5 differing rows, and count the ranges examined.
  5. Add the watermark filter and the resume point, and name the throttle signal the loop watches.

Expected result: A rejection of per-row point reads backed by the time cost and the cache-eviction argument, a single ordered merge join as the dense-case answer at O(n) time and O(1) memory, a range-hash drill-down examining O(d log_B(n/d)) ranges using a sum or multiset hash rather than XOR, a watermark excluding recently written rows, and a resumable throttled run loop.

Verify it yourself

  • Injecting one known stale row makes exactly one leaf range mismatch and the drill-down reaches it.
  • Duplicating a row on one side is detected, which is precisely the case XOR would miss.
  • Running the diff twice against an idle system reports zero mismatches both times.

Follow-up

  • The diff reports 900 stale rows. How do you decide between patching those rows and rebuilding the projection from resource_revision?
  • Same job, but the projection lives in a search index that cannot be scanned in key order. What changes?
  • How would you run this continuously at low cost instead of only as incident response?

Given a large dataset of unstructured text, write an optimized ETL pipeline using PyTorch/TensorFlow datasets to tokenize and batch the data for training.

Medium · data, sql and pipelines

Approach

  1. Say which index the query would use, and what makes it unusable.
  2. Name the grain you start from and join outward from it.
  3. Handle the rows that do not match: that is usually the actual question.

Follow-up

  • What happens to this when the table is ten times larger?
  • How would you run this migration without downtime?

Write the update path that detects a concurrent edit

Medium · optimistic concurrency · transactional outbox · audit history

resource carries version INT NOT NULL DEFAULT 1. resource_revision holds revision_id, resource_id, version, actor_user_id, change_kind, patch JSONB, request_id, created_at with UNIQUE (resource_id, version). outbox_event holds aggregate_type, aggregate_id, aggregate_version, event_type, payload, status. A PUT carries the version the client read. Write the exact statements for the single transaction that applies the edit, records the revision and enqueues 'resource.updated', and give the handler's branch on zero affected rows. Then say what PostgreSQL 16 does under READ COMMITTED when two of these updates hit one row at once.

Approach

  1. One transaction, three writes, no network call inside it: UPDATE resource SET title = $3, version = version + 1, updated_at = now() WHERE resource_id = $1 AND tenant_id = $4 AND version = $2; then INSERT the resource_revision row at version $2 + 1; then INSERT the outbox_event row at the same aggregate_version. The event goes to a table rather than a broker because no transaction spans both.
  2. Branch on the affected-row count before doing anything else. Zero has three causes — stale version, wrong tenant, row gone — so re-read once and map to 409 carrying the current version, or 404 for an id outside the caller's tenant, which also stops the endpoint confirming that another tenant's id exists.
  3. State the engine behaviour instead of assuming it. Under READ COMMITTED the second UPDATE blocks on the row lock, and when the first commits PostgreSQL re-evaluates the WHERE clause against the newly committed row, so the version predicate now fails and the statement reports zero rows. Under REPEATABLE READ the identical collision raises SQLSTATE 40001 instead, so the handler must fold both shapes into one conflict response.
  4. Keep UNIQUE (resource_id, version) even though the predicate already serialises writers. It is what makes a lost update unwritable if any other path ever reaches the revision table, and it converts a logic bug into 23505 rather than into a silently missing history row.
  5. Refuse to auto-retry the whole PUT. A retry re-reads the winner's state and reapplies an intent formed against data that no longer exists — the silent overwrite the version token was added to detect. Return the conflict; merge field-wise only if the patches are provably disjoint.
  6. Note that now() is the transaction timestamp in PostgreSQL, so resource.updated_at, the revision's created_at and the outbox row share one instant, which is what later makes reconciliation between the three tables unambiguous.

Worked solution (25 min)

  1. Write the three statements plus the rowcount branch and confirm they sit in one BEGIN/COMMIT with no outbound call between them.
  2. Run two clients that both read version 7 and apply their updates 5 ms apart; assert one 200 and one 409.
  3. Assert resource.version = 8, exactly one resource_revision row at version 8, and one outbox_event row at aggregate_version 8.
  4. Repeat at REPEATABLE READ and record the different failure shape (SQLSTATE 40001) the handler must also map to 409.
  5. Delete the version predicate and re-run: both writes commit and the first edit disappears with no error raised anywhere.

Expected result: Exactly one of the two commits. version = 8 with one revision row and one outbox row at version 8, the loser gets 409 carrying version 8, and no state exists in which the resource moved but the revision or the outbox row is missing.

Verify it yourself

  • COUNT(*) of resource_revision for that resource equals resource.version, with no gap in the version sequence.
  • Kill the process between the UPDATE and the outbox INSERT — the transaction rolls back and no event describes a change that never happened.
  • Under a 50-way race, resource.version equals the number of 2xx responses rather than the number of requests.

Follow-up

  • A client sends the version it read ten minutes ago and the resource has moved three versions. What is in your 409 so it can resolve the conflict without a full re-fetch?
  • Two editors, two disjoint fields, no overlap. Does your answer still refuse the second write, and should it?
  • Every write now touches a second hot table. How do you keep the outbox insert and its partial index from becoming the write bottleneck at 1.2k writes/second?

What strategies would you use to handle rate limits and cold-start latencies when deploying models across hybrid cloud environments?

Medium · ml system design and production

Approach

  1. Name what you would monitor after launch and what triggers a retrain.
  2. Fix the product goal and the online metric before choosing any model.
  3. Separate the offline training path from the online serving path.

Follow-up

  • How would you detect drift before the metric drops?
  • What happens when a feature is missing at serving time?

What are the primary techniques you would use to reduce LLM inference latency in a production environment?

Medium · ml system design and production

Approach

  1. Name what you would monitor after launch and what triggers a retrain.
  2. Fix the product goal and the online metric before choosing any model.
  3. Separate the offline training path from the online serving path.

Follow-up

  • What happens when a feature is missing at serving time?
  • How would you detect drift before the metric drops?

Describe how reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO) can be applied to align an agent's reasoning style.

Medium · system design

Approach

  1. State the consistency you need, and where you are willing to be stale.
  2. Fix the scope first: who calls this, how often, and what they do when it fails.
  3. Choose a partition key and say what query it makes expensive.

Follow-up

  • What would you drop to keep the system up under load?
  • How does this behave when that dependency is down for an hour?

Describe a scenario where an agent fails during tool use (e.g., an API returns a 500 error). How should the agentic framework handle this autonomously?

Medium

Approach

  1. Say what you would check first and why it is the highest-information step.
  2. Clarify what is being asked and what a complete answer contains.
  3. Work from the requirement backwards to the design.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Keep one unresponsive destination from stalling all webhook delivery

Medium · fan-out · circuit breaker · timeouts · bulkheads

Egress delivery sends about 1.5k webhooks/second to 40k destinations, with a per-destination concurrency cap of 4 and a 10-second connect-plus-read timeout. One destination begins accepting connections and never responding; within the hour 150 destinations behave the same way. Design the delivery path so unrelated destinations are unaffected: the pool structure, the timeouts, the retry policy, the per-destination circuit, and what is recorded so a retry is not a second effect at the receiver. State how many in-flight slots the degraded destinations hold and why that number decides the design.

Approach

  1. Start with the number, and with the law that produces it. In-flight work is arrival rate times time in service, so 1.5k/second against a healthy 200 ms response needs about 300 concurrent slots. Per destination the same product applies, ceilinged by the concurrency cap: at the fleet average of 0.0375 deliveries/second per destination (1.5k spread over 40k) a 10-second timeout is 0.375 slots. A destination that has queued retries behind it is a different regime - every slot refills the instant an attempt expires, so it sits pinned at its cap of 4 - and 150 of those hold 600 slots, more than a pool sized for healthy traffic, entirely consumed by endpoints that will never answer. The per-destination cap bounds one endpoint and says nothing about the aggregate, which is exactly why it alone is not containment.
  2. Contain with bulkheads and an admission bound rather than a larger pool. Cap total in-flight per pool and shard destinations across pools by a hash of destination id, so a correlated group - one provider, one region - cannot exceed its pool's share. A delivery refused admission and re-queued with backoff is strictly better than one holding a slot on behalf of a receiver that is not listening.
  3. Treat the timeout as two timeouts, and be exact about what shortening one buys. Connect and read are separate failures and both must be shorter than the budget of whatever is waiting. Occupancy is min(cap, arrival rate x timeout), so dropping the read ceiling from 10 seconds to 3 cuts a merely slow destination's occupancy proportionally, 0.375 slots to 0.11 at the fleet average. It does not cut the 4 slots held by one of the 150: a destination with a retry backlog arrives far above cap/timeout - 0.4/second at a 10-second timeout, 1.33/second at 3 - so it stays pinned at the cap either way and only the slot-seconds per attempt fall. What that does buy is detection rate: 3.3x more failures observed per second on the same four slots, which is how fast the circuit reaches its threshold. Pick the value from the measured latency distribution of successful deliveries, with their high percentile as the floor, not from a round number.
  4. Add a circuit per destination, counting a timeout as a failure. Once open, fail fast without taking a slot - that is the whole point, converting 4 held slots into zero. Half-open on a schedule with exactly one probe and close only if the probe succeeds, so a permanently dead endpoint costs one request per interval instead of a growing retry queue.
  5. Make retries safe and non-synchronising. Back off with full jitter, sleeping a random value in [0, min(cap, base x 2^attempt)], because a fixed delay re-synchronises every failed delivery to one destination into a simultaneous burst. Delivery is at-least-once, so the payload carries the event id under the signature and the receiver deduplicates on it; record the attempt against (destination, event id) rather than a bare success flag, so a lost response does not become a second business effect on the other side.

Worked solution (25 min)

  1. Compute healthy in-flight from rate times latency, then slots held by 150 destinations at the cap and the full timeout, and compare both against one pool size.
  2. Write the pool sharding rule and the admission bound, and state what a refused delivery does next.
  3. Pick connect and read timeouts from the success-latency distribution, then compute min(cap, arrival rate x timeout) for an average destination and for one with a retry backlog, and say which of the two the shorter timeout actually moves.
  4. Write the circuit's state machine with its open threshold, probe interval and close condition, and the backoff formula with full jitter.

Expected result: Destinations sharded across bounded pools with a total in-flight cap per pool, separate connect and read timeouts chosen from the success-latency distribution, a per-destination circuit that fails fast while open and probes once per interval, and full-jitter backoff on retry. A shorter read timeout lowers occupancy only for destinations arriving below cap/timeout; a dead destination with a retry backlog stays pinned at its cap of 4 whatever the timeout, and the circuit opening is what takes it to zero. Deliveries carry a signed event id the receiver deduplicates on, and attempts are recorded per (destination, event id). Degraded destinations hold at most their pool's share rather than 600 slots of a shared pool.

Verify it yourself

  • Slots held by degraded destinations are bounded by the pool shard, not by how many destinations degrade.
  • An open circuit reduces slots held by that destination to zero between probes.
  • Lowering the read timeout does not reduce successful deliveries, verified against the measured success-latency distribution, and does not by itself reduce the slots a backlogged dead destination holds.
  • A redelivered event is discarded by the receiver on its event id rather than applied twice.

Follow-up

  • The destination is not dead - it answers in 9.5 seconds with a 200. Does a failure-rate circuit open? Should anything shed that traffic, and on what signal?
  • One destination requires deliveries in order. What does a per-destination concurrency of 4 do to that guarantee, and what would you change to offer it?
  • A destination has been parked six hours with 900k undelivered events. What does resuming look like, and is delivering the whole backlog the right call?

One log partition stops advancing while the others drain

Easy · poison message · head-of-line blocking · consumer lag · projections

Search results for a subset of tenants are hours stale; the rest are current. The projection consumer reports lag of zero on 15 of 16 partitions and 400,000 on one. Its error rate is flat and its CPU is idle. outbox_event has no pending rows older than a second, so the relay has published everything it holds. Identify the mechanism, give the ordered checks, and state what you do in the first ten minutes versus what you change permanently.

Approach

  1. Read the lag distribution first. A slow consumer lags everywhere; zero on fifteen partitions and 400,000 on one is not throughput. Idle CPU on the stuck partition means the consumer is not advancing its offset at all, which points at one message it cannot get past rather than at a rate problem.
  2. Exonerate the producer before touching the consumer. No pending outbox rows older than a second means the relay published, so the event exists in the log. This separates never sent from sent and never applied, which are different code paths and usually different owners.
  3. Read the message at the stuck offset and the handler's log lines for its event_id. A flat error rate with no progress has two explanations and you must distinguish them: the handler is throwing and the retry loop is swallowing it, or the handler is blocking on something and never returning. Idle CPU with no error lines favours the second.
  4. Mitigate before diagnosing further. Move the offending event to a dead-letter store and commit the offset past it. Adding consumers does nothing here, because a partition is consumed by exactly one member of the group, and the blast radius is every aggregate hashed to that partition, not only the aggregate that produced the bad event.
  5. Fix permanently by bounding handler attempts and dead-lettering on exhaustion, so no single message can stop a partition. Then replay the dead-lettered event once the handler is fixed: it carries aggregate_id and aggregate_version, so a consumer that discards versions it has already applied can absorb the replay, and resource_revision is the fallback if the event itself is unusable.

Follow-up

  • The dead-lettered event carried aggregate_version 7 and the projection had applied 6. What must the replay do differently if 8 and 9 landed in the meantime?
  • How do you show staleness to the user while the partition is behind, given the API already returns the projection's watermark?
  • What changes if the message is poison because a previous deploy wrote a payload shape the current code cannot parse?

Suggested practice checklist

For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.

Day 1 — Rebuild the primitives by implementing them

  • Implement a dynamic array with doubling growth and an operation counter, then change the growth rule to add a fixed sixteen slots instead, and time both for n of ten thousand, a hundred thousand and a million. The fixed-increment version resizes n/16 times at O(n) each, so its total work is quadratic; doubling is what makes append amortised constant.
  • Implement a hash map with separate chaining and a load-factor resize, then insert ten thousand keys engineered to land in one bucket and record what happens to lookup time, so that average-case O(1) becomes a claim with a stated precondition rather than a reflex.
  • For dynamic-array append and hash-map insert, write down which cost is amortised rather than worst-case, which single operation pays the whole bill, and what a system with a hard per-operation deadline would have to do instead.

Deliverable: Two working implementations plus a timing table showing the input at which each structure's advertised complexity stops holding.

Day 2 — Arrays under an invariant: two pointers, sliding window, binary search

  • Solve longest-subarray-with-sum-at-most-K using a sliding window, then run it on an input containing negative numbers and watch it return the wrong answer: extending the window only moves the sum monotonically when every element is non-negative, and that precondition is the whole reason the technique works.
  • Write the binary search that finds the first index satisfying a predicate rather than an exact value, put the loop invariant above the loop in a comment, and verify termination on the two inputs that break careless versions: the empty range, and a range where every element satisfies the predicate.
  • Compute the midpoint as lo + (hi - lo) / 2 and write one line on why the obvious (lo + hi) / 2 is a genuine defect in a fixed-width integer type and a non-issue in a language with arbitrary-precision integers.

Deliverable: Three solved problems, each with its invariant written above the loop, plus one recorded input on which the sliding window is provably wrong.

Day 3 — Sorting, heaps, and the greedy argument that has to be proved

  • Solve one top-k problem three ways, by full sort, by a size-k heap, and by quickselect, then write the values of n and k at which each becomes the right choice, along with quickselect's quadratic worst case and why a randomised pivot makes that unlikely rather than impossible.
  • Implement bottom-up heapify and count sift-down steps to confirm it does linear work rather than n log n, because most nodes sit near the bottom of the tree and therefore move only a short distance.
  • Take interval scheduling by earliest finishing time and write the exchange argument out in full: given any optimal schedule, swapping in the earliest-finishing interval keeps it feasible and no smaller. Then construct the weighted variant where that same greedy fails and name what has to replace it.

Deliverable: A three-way top-k comparison with measured crossover points, one written exchange argument, and one counterexample to a greedy rule that looks almost identical.

Day 4 — Recursion, memoisation, and the step to a table

  • Take one problem with overlapping subproblems, such as edit distance or coin change, instrument the plain recursion with a call counter to show the blow-up, then add memoisation and re-count.
  • Convert the memoised version to a bottom-up table and state the two properties you relied on: each subproblem's result depends only on its arguments, and the dependencies form a DAG you can enumerate in order.
  • Rewrite one deep recursion with an explicit stack, then find the input length at which the original hits the interpreter's frame limit, which defaults to about a thousand frames in CPython, so you know when the rewrite is required rather than decorative.

Deliverable: One problem in three forms, naive, memoised and tabulated, with call counts for each and the input length at which recursion depth becomes the binding constraint.

Day 5 — Graphs, where most of the work is choosing the traversal

  • Implement BFS and DFS over one adjacency list, then answer for each which finds a shortest path in an unweighted graph and which you would use to detect a cycle in a directed graph, including why the in-progress versus finished distinction matters for the second.
  • Implement topological sort by in-degree, feed it a graph containing a cycle, and confirm the failure signature is that fewer than V nodes come out rather than an exception, then note that the order it produces is one of several valid ones.
  • Run a shortest-path search on a graph with a single negative edge weight and show the wrong answer, then write the precondition Dijkstra actually needs, non-negative weights, because it finalises a node's distance the first time that node is popped, and name the algorithm you would switch to and its own limit.

Deliverable: A small graph library with BFS, DFS and topological sort, plus two inputs that produce documented wrong answers under the wrong algorithm choice.

Day 6 — One day for everything that is not an algorithm

  • Sketch one system only to the depth a coding-heavy loop tends to reach: the endpoints, what the service stores, and the single query pattern that decides the schema. Stop at twenty-five minutes.
  • Prepare the project answer for an interviewer who codes, which means rehearsing the two levels they push to: the specific thing you built, and why you chose that approach over the alternative they will name. Open with a number and be ready to say what it excludes.
  • Prepare the answer to what you would do differently, choosing a real technical mistake with a specific fix rather than a complaint about process or staffing.

Deliverable: One design sketch at endpoint-and-schema depth, plus a project answer rehearsed to two levels of follow-up.

Day 7 — Solve out loud, under time

  • Do three timed problems at twenty-five minutes each in a plain editor with no autocomplete and no execution until the end, then tally separately the failures that were syntax and the ones that were approach, because those two numbers call for different fixes.
  • Narrate one solution from the first sentence, stating the approach and its complexity before writing any code, and rehearse the sentence you will use when you realise mid-solution that the approach is wrong.
  • Re-solve from blank the two problems you were slowest on this week and compare the times against the day they first appeared.

Deliverable: A recording of one fully narrated solution and a tally that separates syntax failures from approach failures.

Risky changes and the reversal path

For anything that touched live traffic, be ready to say how you would have undone it: a flag, a staged rollout, dual writes with the old path still authoritative. Once the old column is dropped or the source rows are overwritten there is no reverse, so name what you kept a copy of and for how long.

Give an example of how you mentored a senior engineer or influenced a cross-functional team's technical roadmap.

Medium · behavioural and collaboration

Approach

  1. Name the disagreement and how you resolved it with evidence.
  2. State the situation in two sentences and spend the rest on the reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • What would you do differently if you ran that again?
  • How did you know your change caused the improvement?

Reverse your own decision and price the reversal

Medium · reversibility · measurement · migrations

Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.

Approach

  1. State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
  2. Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
  3. Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
  4. Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.
  5. Finish on the process change: the smallest experiment that would have produced the same measurement in a day, and why you did not run it the first time.

Follow-up

  • What in that decision was irreversible, and did you know it was irreversible when you made it?
  • How did you tell the people who had already built on top of the original decision?
  • What do you now measure before committing to a change of this size?

Narrate an outage you owned from page to postmortem

Hard · incident response · blast radius · postmortems

Pick an incident you personally drove, ideally one where writes were affected rather than reads. In six to eight minutes: state the symptom as it first appeared on a dashboard, the blast radius you established before you knew the cause, the mitigation you applied and when, the mechanism you eventually proved, and the follow-up that would prevent a repeat. Bring numbers: error rate, tenants affected, minutes to mitigate, minutes to resolve. If you cannot name what you measured, choose a different incident.

Approach

  1. Open on the signal rather than the cause: which metric at which percentile moved, on which service, at what time, so the listener follows the same evidence you had rather than a conclusion you already reached.
  2. Separate mitigation from diagnosis out loud. State what you did to stop the bleeding (flag off, shed traffic, drain a lease, roll back a deploy) and say plainly that you did it before the mechanism was known, because those are two jobs with different deadlines.
  3. Establish blast radius in countable terms: how many tenants, how many writes, and crucially whether the effect was loss or only delay. An append-only revision table or a pending outbox row means the change survived and the projection was merely behind, which is a repair rather than a data-loss incident.
  4. Prove the mechanism instead of asserting it. Name the trace span that grew, the plan that flipped to a sequential scan, the lease that expired, plus one alternative you ruled out and the signal that stayed flat while you ruled it out.
  5. Close on the durable fix and its cost, distinguishing what landed that week from what needed an expand-and-contract migration across several deploys, and say which of the two you actually finished.

Follow-up

  • What would you do differently in the first five minutes, given the same dashboard and no more information?
  • Which follow-up action did you deliberately not take, and why was dropping it the right call?
  • How did you convince yourself the mitigation was safe to apply while the cause was still unknown?