Interview guides · Glance
Glance Machine Learning Engineer Interview Questions & Guide 2026
Glance Machine Learning Engineer interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.
The 60-second brief
As a Machine Learning Engineer at Glance, you sit at the epicenter of one of the most innovative digital content platforms globally. Your work directly powers the personalized lock screen experience that millions of users interact with every day. You are not just building models; you are crafting intelligent, large-scale systems that bridge the gap between human curiosity and high-quality, real-time entertainment.
Ask whether any round happens inside an existing repository instead of a blank file. Reading unfamiliar code, isolating a fault and making the smallest correct change is a different skill from writing a function from scratch, and it needs its own practice.
Focus areas
- Detect concurrent edits instead of losing writes
- Bound every outbound call with a timeout
- Make every write idempotent under retry
Overview
This role is unique because it demands a synthesis of classical machine learning, large-scale recommendation systems, and the emerging frontier of agentic AI. You will be responsible for the end-to-end lifecycle of ML products, from conceptualizing ranking algorithms and deep learning architectures to deploying robust, self-improving pipelines. Your contributions will directly influence user retention, content discovery, and the overall business trajectory of Glance.
Working here requires a blend of deep technical craftsmanship and a highly collaborative spirit. You will interact with cross-functional teams comprising physicists, economists, and data scientists, all focused on solving complex problems in identity-less ecosystems. If you thrive in environments where experimentation, production-scale impact, and forward-looking AI research converge, this position offers a rare opportunity to define the future of mobile content.
Where to focus your preparation
Treat the sections below as preparation areas and confirm the format with your recruiter.
Preparation focus
No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.
What to demonstrate
- Breadth across SQL, experimentation and product reasoning
- Ability to state assumptions before choosing a method
How to prepare
- Drill the practice exercises below and time yourself
- Prepare three quantified stories about decisions you drove
Where candidates lose points
Assuming an isolation level prevents the anomaly you actually have
Isolation levels are named by the SQL standard but implemented differently, so any claim about one is only true of a named engine. PostgreSQL defaults to READ COMMITTED, where every statement takes a fresh snapshot, so two statements inside one transaction can legitimately disagree about the same row. Its REPEATABLE READ is snapshot isolation: it removes non-repeatable and phantom reads but permits write skew, where two transactions each read a set, each conclude their own write is safe, both commit, and the combined result violates a constraint that no single row expresses. Only SERIALIZABLE closes that, and it closes it by aborting a transaction with a serialization failure (SQLSTATE 40001), which means the guarantee is theoretical unless the application has a retry loop. InnoDB's REPEATABLE READ is a different mechanism again - plain SELECTs read a consistent snapshot while locking reads and writes see the latest committed row - so a read-modify-write inside one transaction can act on a value that the transaction's own earlier SELECT never returned.
Running a schema change as though the lock lasts as long as the statement
In PostgreSQL an ALTER TABLE that needs an ACCESS EXCLUSIVE lock must first wait for every open transaction touching that table, and while it waits, later queries needing a conflicting lock queue behind it rather than overtaking it. A DDL statement that would execute in milliseconds, issued while a thirty-second analytics query is open, therefore stalls all traffic on that table for thirty seconds: the outage length is set by the longest open transaction, not by the change. The defences are specific and worth knowing by name - set lock_timeout low and retry rather than queue, add columns without a volatile default so no table rewrite occurs (from version 11 a non-volatile default is a metadata-only change), build indexes with CREATE INDEX CONCURRENTLY while accepting that it cannot run inside a transaction block and leaves an invalid index behind if it fails, and add constraints as NOT VALID followed by a separate VALIDATE CONSTRAINT, which takes a weaker lock.
Hardcoding to the sample inputs
Solve the stated problem rather than the two examples; special-casing a literal to make a sample pass is obvious immediately and reads as either a misunderstanding or an attempt to fake progress. If you genuinely cannot generalise yet, say which part is a stub and what would replace it.
Optimising an axis nobody named
Ask which resource is actually scarce here: wall-clock latency, throughput, memory footprint, cost per request, or engineering time. Shaving a constant factor off an in-memory step is wasted effort when the same function makes a blocking remote call inside the loop.
Questions & practice
Compare and contrast collaborative filtering, content-based approaches, and hybrid models in the context of a cold-start problem.
Medium · machine learning fundamentals
Approach
- Say how you would validate it, and where leakage could enter the split.
- Name the simplest model that could work and what would make you move past it.
- Pick the metric from the cost of each error type, not from habit.
Follow-up
- What changes if the classes are heavily imbalanced?
- How would you know the model is overfitting?
How would you implement a ranking system that balances user engagement metrics with business-driven content diversity?
Medium · machine learning fundamentals
Approach
- State the learning problem: the label, the unit of prediction and how the model is used.
- Pick the metric from the cost of each error type, not from habit.
- Say how you would validate it, and where leakage could enter the split.
Follow-up
- How would you know the model is overfitting?
- Where could label leakage enter this setup?
Explain the trade-offs between exploration and exploitation in a multi-armed bandit setup for content delivery.
Medium · machine learning fundamentals
Approach
- Pick the metric from the cost of each error type, not from habit.
- Name the simplest model that could work and what would make you move past it.
- State the learning problem: the label, the unit of prediction and how the model is used.
Follow-up
- What changes if the classes are heavily imbalanced?
- How would you know the model is overfitting?
Merge partitioned event streams into one ordered feed with bounded lateness
Hard · k-way merge · watermarks · out-of-order streams
The read-model service consumes 64 log partitions carrying about 4,000 events per second in total. Each partition is ordered within itself, but partitions drift by up to 30 seconds, and the activity feed must present a tenant's events in occurred_at order. Produce the merge. State its complexity, the buffer it requires in events and in bytes, what happens when one partition is idle, and what you do with an event that arrives after you have already emitted its position. Payloads average 1 KB.
Approach
- Merge with a min-heap over the 64 partition heads keyed on (occurred_at, event_id): O(log P) per event and O(n log P) overall. The tie-break on event_id is what makes the output deterministic when two partitions carry the same millisecond, which matters because the feed is paginated and a non-deterministic order reorders pages under the reader.
- Emitting the heap head is only correct once every partition has produced everything up to that timestamp, so the emit condition is a watermark: the minimum across partitions of the highest occurred_at seen, less the allowed lateness. Events are held until the watermark passes them, which is what turns individually ordered streams into a jointly ordered one.
- Size the buffer from the lateness rather than guessing: 4,000 events per second times 30 seconds is 120,000 buffered events, and at 1 KB each about 120 MB of heap. That number is the real price of the ordering guarantee and belongs in front of whoever asked for it.
- Handle the idle partition explicitly, because it fails the feed rather than corrupting it: a partition with no traffic never advances its own maximum, so the watermark freezes and output stops entirely. Either every partition emits a periodic idle marker carrying the broker's current time, or the watermark falls back to wall clock for a partition silent beyond a threshold.
- Choose the late-event policy from what the projection is keyed on. The projection upserts on (aggregate_id, aggregate_version) and discards a version it has already applied, so a late event is safe to apply out of order and correctness never depended on the merge at all. Apply it, recompute the affected feed page, and count lateness so the 30-second budget can be re-derived from data rather than folklore.
- Say what the merge does not buy: ordering is guaranteed within one aggregate by the log's partitioning, and no watermark makes the cross-aggregate order authoritative. Two events from different aggregates in the same millisecond have no true order, so the feed's order is a presentation choice that must be stable rather than correct.
Worked solution (35 min)
- Write the heap comparator on (occurred_at, event_id) and the per-partition head refill.
- Write the watermark computation and the emit-loop condition, then list which buffered events are held at a chosen instant.
- Compute the buffer at 4,000 events per second, 30 seconds and 1 KB per event, and state what fraction of a worker's heap that represents.
- Add the idle-partition marker and trace the watermark with one silent partition, both with and without the marker.
- Write the late-event path and name the key that makes applying it safe.
Expected result: A 64-way min-heap merge at O(n log P) with a deterministic (occurred_at, event_id) comparator, a watermark of the per-partition minimum less 30 seconds gating emission, a stated buffer of 120,000 events and roughly 120 MB, idle markers so a silent partition cannot freeze the watermark, and a late-event policy justified by the projection's idempotency on (aggregate_id, aggregate_version).
Verify it yourself
- Feeding the partitions in an adversarial interleaving produces the same output sequence as feeding them in timestamp order.
- Silencing one partition stalls output without the idle marker and does not stall it with the marker.
- Applying a late event twice leaves the projection byte-identical, which is what permits the out-of-order path to exist.
Follow-up
- The lateness budget is raised to five minutes. What is the new buffer, and what besides memory changes?
- The consumer restarts. Where does it resume from, and what does the feed look like for the first 30 seconds?
- One partition is ten minutes behind because its producer is slow. Do you stall the feed or emit without it?
Keep soft-deleted accounts from blocking re-registration
Easy · soft delete · unique constraints · engine differences
app_user holds user_id, tenant_id, email CITEXT, password_hash (NULL for SSO principals), email_verified_at, auth_version, status ('invited','active','suspended','deactivated'), created_at, updated_at, deleted_at. Two live accounts for one address inside a tenant must be impossible, but an address freed by a soft delete must be reusable, and the same tenant may delete and re-register it repeatedly. Write the uniqueness DDL for PostgreSQL 16, then the equivalent for MySQL 8 where partial indexes do not exist, and say what each permits once three deleted rows already hold that address.
Approach
- Start from what is actually unique: not (tenant_id, email), but (tenant_id, email) among live rows. PostgreSQL says that directly — CREATE UNIQUE INDEX app_user_live_email ON app_user (tenant_id, email) WHERE deleted_at IS NULL. A full constraint over the same two columns burns the address permanently the first time someone deletes an account.
- Keep case-insensitivity in the type or the index, never in the application: CITEXT as given, or UNIQUE (tenant_id, lower(email)) as an expression index where the extension is unavailable. A case-sensitive unique column is exactly how two accounts for one human appear.
- For MySQL 8 the predicate has to move inside the key: add a discriminator column that is a constant 0 while the row is live and is set to user_id on delete, with UNIQUE (tenant_id, email, deleted_marker). Live rows share the constant and still collide; deleted rows differ from each other and stop colliding.
- State the NULL variant and its dependency: leaving the marker NULL for deleted rows also works, because a unique index treats NULLs as distinct — true in MySQL, and true in PostgreSQL only under the default NULLS DISTINCT, which PostgreSQL 15 lets you reverse. Check the polarity against the three existing deleted rows: constant-on-live is what preserves the collision you want, and reversing it silently admits duplicate live accounts.
- Say what a soft delete must do besides setting deleted_at: increment auth_version so existing tokens stop validating, leave resource.owner_user_id and resource_revision.actor_user_id intact, and accept that the address is retained — erasure is a different requirement answered by scrubbing the column, not by a DELETE that would break those references.
Worked solution (20 min)
- Create the PostgreSQL partial unique index, insert a live row, soft delete it, and insert the same address again.
- Repeat the delete-and-reinsert cycle three times and confirm three deleted rows coexist with exactly one live row.
- Write the MySQL form with the discriminator, then deliberately reverse the polarity so live rows carry NULL, and show two live duplicates commit.
- Attempt a second live insert on both engines and map the resulting 23505 / ER_DUP_ENTRY to the 409 the handler should return.
Expected result: PostgreSQL rejects only the second live row and admits unlimited deleted ones. MySQL behaves identically with UNIQUE (tenant_id, email, deleted_marker) where live = 0; with the polarity reversed, two live rows commit and nothing afterwards can tell them apart.
Verify it yourself
- Group live rows by (tenant_id, lower(email)) after a randomised insert/delete run — no group exceeds one.
- Inserting 'User@x.test' after 'user@x.test' is rejected; if it is accepted, the case handling is in application code rather than in the index.
- EXPLAIN a login lookup and confirm the partial index is chosen — it is only usable by queries that carry the deleted_at IS NULL predicate, so a query without it falls back to a scan.
Follow-up
- A deleted account re-registers with the same address the next day. Do the old resource rows follow the new user_id, and how does the API keep the two principals apart?
- How do you honour an erasure request while resource_revision.actor_user_id still references this table?
- What changes if a user may hold membership in two tenants?
Stop tag and share joins from fanning out a page
Medium · join fan-out · semi-joins · many-to-many
resource_tag is (resource_id, tag_id) with PK (resource_id, tag_id); resource_share is (resource_id, shared_with_user_id, permission). The tagged-and-shared listing inner-joins resource to both, filters tenant_id, tag_id = ANY($2) and shared_with_user_id = $3, orders by updated_at DESC and takes 50. Pages come back with fewer than 50 distinct resources and the total in the header is far too high. Explain the row multiplication, rewrite both the page query and the count query so each is correct, and name the index each one needs. PostgreSQL 16.
Approach
- Do the arithmetic against the predicates that are actually there. An inner join emits one row per matching child row, and both joins are filtered: tag_id = ANY($2) admits only the requested tags, shared_with_user_id = $3 admits one user's share rows. So a resource holding three of the requested tags and shared with $3 once yields three rows, not one — the multiplier is its count of matching tags times its share rows for that single user, and that second factor is 1 unless the table admits duplicate (resource_id, shared_with_user_id) pairs. LIMIT 50 then limits rows rather than resources, and COUNT(*) counts pairs — the header is the product, not the population.
- Reject DISTINCT as the fix. It deduplicates after the product has been built, so the planner must materialise and sort the fanned-out set before the LIMIT can apply, and it leaves any SUM or AVG in the same select list wrong.
- Rewrite both filters as semi-joins, keeping resource as the only row source: AND EXISTS (SELECT 1 FROM resource_tag rt WHERE rt.resource_id = r.resource_id AND rt.tag_id = ANY($2)) and the same shape against resource_share. A semi-join stops at the first match per resource and preserves the driving index order, so ORDER BY updated_at DESC, resource_id DESC LIMIT 50 still stops after 50 rows.
- Count with the same predicates and no join at all: SELECT count(*) FROM resource r WHERE r.tenant_id = $1 AND r.status = 'active' AND EXISTS (...) AND EXISTS (...). Nothing multiplies a resource, so the number is the population.
- Attach the tags for display after the page has been cut — LEFT JOIN LATERAL (SELECT array_agg(rt.tag_id) FROM resource_tag rt WHERE rt.resource_id = p.resource_id) ON TRUE over the 50 returned rows. Aggregate over the page, never over the tenant.
- Index both directions and say which query each serves: PK (resource_id, tag_id) serves the lateral lookup, (tag_id, resource_id) serves the EXISTS probe by tag, and resource_share needs (shared_with_user_id, resource_id) for the same reason. An index covering one direction only leaves the other as a scan.
Follow-up
- The filter changes from 'any of these tags' to 'all of these tags'. Rewrite it and state what it costs relative to the ANY form.
- A resource can be shared with the same user twice under different permissions. Does your count change, and should it?
- Where does the correct total come from when the tenant holds 4M resources and the header must not cost 200 ms?
How would you design a recommendation system for a lock-screen platform with strict latency requirements?
Medium · ml system design and production
Approach
- Separate the offline training path from the online serving path.
- Say where features come from at serving time and how they match training.
- Fix the product goal and the online metric before choosing any model.
Follow-up
- How would you roll the new model out safely?
- How would you detect drift before the metric drops?
Publish rate-limit and deadline semantics the edge actually enforces
Easy · rate limiting · backpressure · retries
The edge API serves about 3k requests/second steady and 9k at peak against a 400 ms p99 budget, with an explicit bounded concurrency limit per instance. Limits exist per principal and per tenant. Callers are a partner integration running nightly bulk loads and a browser app. Specify the counting algorithm and window, which limit a request is charged against, the headers a well-behaved client reads, the status and body when a limit is hit, how that differs from the response when an instance is shedding load, and what each caller does with each.
Approach
- Choose the counter and name its failure mode. Fixed windows admit nearly twice the limit across a boundary - a full burst at the end of one window and another at the start of the next. A token bucket states sustained rate and burst separately, which is exactly what a nightly bulk load needs. A sliding-window counter is more faithful and costs more state per key. State the choice and the burst it permits.
- Charge each request against both keys and reject on the stricter. The tenant limit protects the shared primary, which absorbs roughly 1.2k writes/second in total; the per-principal limit stops one credential inside a tenant from consuming that tenant's whole allowance. The tenant is the fairness unit for the same reason it is the leading column of every index.
- Advertise limit, remaining and reset for the binding key on every response, not only on rejections, so a client can pace before it is refused. Pick one naming scheme - the RateLimit-* draft fields or an X-prefixed set - document the units, and never change them afterwards.
- Separate two rejections that look identical to a naive client. 429 means this caller exceeded its own share and Retry-After is a real schedule it should obey. 503 means the instance is at its concurrency bound and shedding, which is a statement about the server; a fleet-wide 503 retried on a fixed delay resynchronises every client into one stampede, so full jitter is mandatory there and the delay is the client's guess, not ours.
- Make shedding cheap and early - before the token is verified against the database, before any downstream call - because a rejection that costs as much as the work relieves nothing. Drop requests whose client deadline has already elapsed rather than serving them; the caller has stopped listening and the work is pure cost.
- Write the caller behaviours down: the bulk loader paces against
remainingand treats a 429 as a defect in its own pacing; the browser surfaces the wait and must never retry a 429 inside a render loop, which turns one limited user into a self-inflicted flood.
Worked solution (20 min)
- Write the bucket parameters for both keys: sustained rate, burst size, and the refill interval, with the arithmetic that ties them to the 3k/9k figures.
- Draft the three response headers and one example 429 body carrying a code, the limit that bound, and Retry-After.
- Write the 429-versus-503 decision as a two-line rule an on-call engineer can apply to a log line.
- State where in the request pipeline the rejection happens and which work it skips.
Expected result: A token bucket per principal and per tenant with named burst, headers on every response, 429 with Retry-After for share exhaustion, 503 with jittered client backoff for concurrency shedding, and rejection placed before authentication's database read.
Verify it yourself
- A client that respects
remainingnever sees a 429 under steady traffic at its configured rate. - The cost of a shed request is bounded and does not include a database round trip.
- Peak throughput times the limiter's per-request state cost is affordable at 9k requests/second.
Follow-up
- One tenant stays under its limit and still degrades everyone else during a backfill. What changes - the limiter, the worker concurrency caps, or both?
- How are counters kept correct across 20 to 40 stateless instances, and what does your answer cost per request?
Listing latency scales with page size, not with filters
Easy · n+1 · latency · query batching
The tenant listing endpoint reads resource filtered by tenant_id and status, ordered by updated_at DESC, and returns each row plus the owner's display name from app_user and the actor of that resource's latest resource_revision. p99 is 55 ms at 10 rows per page and 1.4 s at 200. Database telemetry shows 401 statements per request, each under 1 ms, and nothing in the slow-query log. Diagnose the cause and give the fix, stating the statement count per request and the p99 you expect afterwards.
Approach
- Read the counters before forming a theory. 401 statements for 200 rows is one driver query plus two per row, and sub-millisecond execution with an empty slow-query log rules out a bad plan. The time is round trips, which is why it is invisible in every per-query metric and scales with rows returned rather than with filter selectivity.
- Name the two per-row statements from their normalised text: a single-row app_user lookup by user_id, and a resource_revision lookup by resource_id ordered by version DESC LIMIT 1. Confirm by dropping those two response fields and watching the statement count fall to one. That locates the calls in the serialisation layer, not the repository.
- Check that the arithmetic accounts for the whole gap. Measure one round trip to the replica in isolation; 400 trips at roughly 3 ms of network plus 0.2 ms of execution is about 1.3 s on top of a 55 ms baseline, which matches. If the multiplication had fallen short, the N+1 would only be part of the story and you would keep looking.
- Batch both lookups. Collect owner_user_ids and resource_ids from the driver query, then issue WHERE tenant_id = $1 AND user_id = ANY($2) for the users, and PostgreSQL's SELECT DISTINCT ON (resource_id) ... WHERE resource_id = ANY($2) ORDER BY resource_id, version DESC for the latest revision, which the UNIQUE (resource_id, version) index serves directly. On an engine without DISTINCT ON, use a lateral join or a row_number window. Three statements per request at any page size.
- Keep the tenant predicate in the batched query. The per-row version was implicitly scoped because its ids came from tenant-scoped rows; a batched user_id = ANY(...) with no tenant_id is an unscoped read that behaves correctly only as long as the id list is trustworthy.
- Re-measure at 10, 50 and 200 rows and confirm the statement count is constant. Latency should now track bytes returned.
Follow-up
- The page size is capped at 200 today. What breaks first if it is raised to 2,000, and is it still this bug?
- How do you stop the next N+1 from reaching production, given that no individual query is slow and the endpoint's tests pass?
- The latest-revision actor is only used to render an avatar. Make the case for denormalising it onto resource, and name the write anomaly that introduces.
Suggested practice checklist
Roughly ninety minutes on weeknights with one longer weekend block. The plan cuts scope rather than compressing everything, on the assumption that one thing finished per night beats four half-started.
Day 1 — Fix the scope and take a cold baseline
- Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list and keep it visible all week.
- Take one twenty-five-minute coding problem and one fifteen-minute design prompt cold, and write the single sentence naming what blocked each, because those two sentences decide where the remaining evenings go.
- Set the week's rule: one thing finished every night, including the night you only have forty minutes.
Deliverable: A one-page scope with a not-doing list and two cold attempts, each carrying one sentence on what blocked it.
Day 2 — One pattern, written three times from blank
- Choose the single pattern most likely to appear in your loop and write it three times from an empty file rather than editing the previous attempt.
- On the third pass, write the invariant as a comment before the loop body and the complexity before the first line of code.
- Stop at ninety minutes even if the third version is imperfect, and write the one thing you would fix given another hour.
Deliverable: Three independent implementations of the same pattern plus a note on what changed between them.
Day 3 — One design, only to the depth you can defend
- Take one system shape and go only as far as requirements, interface and data model, refusing to draw a box you could not survive a follow-up about.
- Attach one number to each non-functional requirement, deriving it rather than asserting it, and write the assumption the number rests on.
- Write the one tradeoff you are choosing against and the observation that would make you reverse it.
Deliverable: One design at interface-and-schema depth with derived numbers and one written reversible tradeoff.
Day 4 — Only the fundamentals you will have to defend
- Write, in under two hundred words each, the answers to the two questions that follow almost any implementation: why this structure and not the obvious alternative, and what happens to this code at a hundred times the input.
- Write what an index actually costs: faster lookups on the indexed columns against a write that now maintains a second structure, plus the cases where the planner declines to use it anyway, low selectivity, or a predicate wrapping the column in a function.
- Delete any answer you cannot deliver aloud in under a minute, since an answer that needs reading is not an answer you have.
Deliverable: Three written answers, each under two hundred words and each timed aloud.
Day 5 — Your own work, timed
- Write a ninety-second and a four-minute version of your main project and time both aloud rather than reading them.
- Prepare the two follow-ups that always come: what you would do differently, and how you knew it worked.
- Put one number in the first sentence and be ready to say exactly where it came from and what it excludes.
Deliverable: Two timed narratives with one defensible number in the opening line.
Day 6 — The one full rehearsal, in the weekend block
- Run a sixty-minute mock covering a coding round and a design round in one sitting with no break, because sustained attention is the thing evenings have not trained.
- Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
- Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.
Deliverable: Mock notes naming three failure moments with a specific fix written under each.
Day 7 — Taper
- Write the twenty-minute warm-up you will actually do on the morning: one problem you can already solve from a blank file, one design you can narrate, and nothing you have never seen.
- Re-read only your own notes from this week and open no new material.
- Write the logistics down: the editor or shared document you will be working in, whether execution and lookups are permitted, and the sentence you will use when you do not know something.
Deliverable: A one-page card holding the design structure, the project numbers, and the logistics.
Disagreement needs an outcome you did not choose
Conflict answers where you were right and everyone came round are the weakest ones. Stronger: the evidence you went and collected, what would have changed your mind, and what you did in the weeks after the call went against you. Implementing a design you argued against, properly, is a specific and checkable behaviour.
How do you handle model drift in a high-volume, real-time recommendation environment?
Medium · behavioural and collaboration
Approach
- Name the disagreement and how you resolved it with evidence.
- Pick a story where you made the decision, not one where you watched it.
- Give the blast radius: what could have broken, and what you measured.
Follow-up
- What did you decide not to do, and why?
- How did you know your change caused the improvement?
Unblock an engineer without taking the keyboard
Easy · mentoring · leases · at-least-once
A teammate has spent two days on a job handler that occasionally writes duplicate rows. They are certain the queue is delivering twice by mistake. You suspect a lease expiring under a slow handler, so the job is running concurrently with itself. Describe how you have unblocked someone in this position: what you asked before offering a hypothesis, what you showed them rather than told them, and what you left them owning. Then say what you would do if their theory turned out to be the right one.
Approach
- Ask before diagnosing, and ask for things answerable from data they already have: the attempt count on the job rows that produced duplicates, the handler's observed duration against its lease expiry, and whether the duplicate rows share a natural key that a unique constraint could have caught.
- Teach the shape rather than the answer. A lease cannot distinguish a dead worker from a slow one, so a handler that outruns its lease is running twice by design, and deploys deliver the other half by killing handlers mid-run on every rollout. Both of their candidate theories produce identical duplicate rows, which is why the evidence has to come from timings rather than from argument.
- Hand over a checklist they execute: a natural key on every write the handler performs so the second copy collides rather than appends, the record of intent written before any external effect, a lease heartbeat while running, and the metric that shows it working.
- Keep ownership with them deliberately. Pair on the first write, then step back; if you finish it yourself you have closed one ticket and left the same person stuck on the next redelivery.
- Close on the systemic gap that let two days pass, which is usually a missing dashboard for attempt counts or an undocumented at-least-once contract, and fix that rather than only the bug.
Follow-up
- How would you distinguish a genuine double-delivery from a lease expiry using only the data already stored?
- Their handler calls an external endpoint before recording that it did. What do you tell them to change first?
- What do you do the third time the same person brings you the same class of bug?
Turn a code review disagreement into a decision
Easy · code review · optimistic concurrency · communication
A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.
Approach
- Sort the disagreement before writing anything. A silently discarded write is a correctness claim about data; the choice between 409 and 412 is taste. Only the first justifies blocking a merge, and saying which one you are doing is most of the value of the comment.
- Make the claim reproducible in the comment itself with an interleaving rather than a principle: A reads version 7, B reads version 7, B commits version 8, A's predicate matches zero rows, A is told it succeeded and A's edit is gone.
- Offer the alternative with its cost attached: return 409 carrying the current version and the revision that won, so the client can re-read and re-apply. Note that automatic retry is not the fix, because a retry re-reads the winner's state and reapplies an intent formed against data that no longer exists.
- Apply an escalation rule you can state: two round trips on the thread, then a call, and the service's owner decides rather than the reviewer. A reviewer who cannot be overruled is a bottleneck with extra steps.
- Close in writing wherever the decision lands, so the next reader finds the reasoning in the code or the ticket instead of in a collapsed review thread.
Follow-up
- Where would you put the test that fails if someone reintroduces the swallowed zero rowcount?
- The author says clients cannot handle a 409. How do you check whether that is true?
- How do you handle the same review comment when the author is more senior than you and in a hurry?