cswork / libraryOpen practice workspace

Interview guides · Acko General Insurance

Acko General Insurance Software Engineer Interview Guide

Acko General Insurance Software Engineer interview guide: Splitwise-style machine coding, schema and concurrency questions, plus a 7-day plan.

The 60-second brief

Acko General Insurance is an insurer. Candidates describe a business built around digital flows where customers buy policies and file claims, with engineering sitting between older insurance processes and user-facing software. The Software Engineer role covers backend services and core platform functionality on a cloud-native footing. Candidates list REST APIs as the interface style, and AWS, distributed systems and microservices as nice-to-have experience. The reported interview questions use an expense-sharing app as the vehicle rather than an insurance scenario, so the domain you need is money, groups and balances rather than policies.

This guide is for candidates interviewing for a Software Engineer role at Acko General Insurance. It covers the machine coding and design topic that candidates report: designing a Splitwise-like system, handling concurrent expense updates, shaping a schema for group balances and individual debts, keeping code maintainable when features such as currency conversion arrive, and weighing an in-memory store against a relational database. It does not cover rounds that nobody has reported.

No round-by-round sequence is reported. Candidates describe a lean process in which machine coding is the central component, and the five reported questions all sit under machine coding and system design.

Focus areas

  • Machine Coding
  • Requirements Gathering
  • Splitwise-like Application Core Functionalities

Overview

Every reported question for this loop circles one system: a Splitwise-like expense-sharing application. The questions ask you to design its core functionality, handle concurrent expense updates, shape a schema for group balances and individual debts, keep the code maintainable when currency conversion is added, and compare an in-memory store with a relational database. Candidates also report two concrete scenarios: an expense-sharing application with equal, exact and percentage-based splits, and the API endpoints a user would call to see their total outstanding balance. That makes this a loop where one well-prepared domain pays off many times.

Preparation therefore starts with the money model. Represent amounts in integer minor units, decide where the leftover cent goes when 100.00 is split three ways, and reject exact splits that do not add up to the total or percentages that do not add up to 100. Then derive balances: a net figure per user per group, built from an append-only list of expenses and settlements, plus a way to reduce who owes whom into few transfers. Once that core runs in your chosen language, the remaining questions become variations on it: what happens when two people edit the same group at once, what the tables look like, and which parts change when a second currency appears.

Candidates describe the process as lean, with machine coding as the central piece, and the role requirements they cite name Java, Python or Go as example object-oriented languages, plus clean, testable code and RESTful API experience. Tips that circulate among candidates are to clarify requirements at the start, talk through your reasoning, design for extension, mention scale briefly and favour readable code. Since no round order is reported, ask your recruiter whether you will write and run code, in which language, and in what environment, and then rehearse that exact setup.

Where to focus your preparation

Only one stage is reported, a general preparation focus, because candidates have not described a round order for this role. Confirm the sequence and format with your recruiter before you plan around it.

Preparation focus

No named rounds are reported for this role. What candidates do report is a lean process in which machine coding carries the most weight, with questions grouped under machine coding and system design. Expect to turn a loosely stated requirement, such as a Splitwise-like application with equal, exact and percentage-based splits, into classes, interfaces and working logic, then defend the design: how balances are computed, how concurrent edits are handled, which storage you would pick and how a new feature such as currency conversion would fit. Approach it by clarifying requirements first, getting a small end-to-end path running before adding features, and keeping the split logic behind an interface so it can be extended. Ask the recruiter whether code must run, which languages are allowed and how the stages are ordered.

What to demonstrate

  • A working, readable solution in an object-oriented language, with sensible class and interface boundaries
  • Whether you pin down requirements and edge cases, such as split validation and rounding, before writing code
  • How you reason about shared state, storage and extension when the design is pushed past the first version

How to prepare

  • Build the Splitwise-style core from a blank file: users, groups, expenses, equal/exact/percentage splits and a per-user balance view, with unit tests
  • Write the schema for group balances and individual debts, then the query that returns one user's net balance
  • Practise the concurrent-update answer for the expense-editing path, naming the shared state and the lock or version check you would use
  • Prepare a short script of requirement questions and a verbal outline of the in-memory versus relational trade-off

Where candidates lose points

Using floating-point numbers for money, so a 100.00 expense split three ways loses or invents a cent

Store amounts as integer minor units, give the leftover units to specific participants by a stated rule, and assert in a test that the shares always sum to the original total. For percentage splits, reject inputs that do not total 100.

Starting to code the Splitwise-like application before settling what it must do

Spend the opening minutes asking: must exact splits sum to the total, can a user be in several groups, are settlements recorded, is there more than one currency, who can edit an expense. State your assumptions out loud and build only what they require.

Hard-wiring split types into one method with if/else branches, so currency conversion or a fourth split type means editing the core

Put each split type behind a common interface with its own validation, wrap amounts in a Money type that carries a currency, and hide exchange rates behind a small provider interface. Be ready to point at the exact seam where a new feature plugs in.

Answering the concurrent-update question with only "use a lock"

Name the shared state (the group's expense list and balances), describe the lost-update case with two edits, then pick a mechanism: a per-group lock in process, or a version column or row lock in a database. Add idempotency keys for retried requests, and say how you would test it.

Treating the in-memory versus relational comparison as one-sided, either "memory is faster" or "SQL is safer"

Compare on durability across restarts, constraints and transactions, query flexibility, behaviour with more than one app instance, and memory limits. Conclude with a recommendation tied to the requirements, and note that a repository interface lets you start in memory and move later.

Questions & practice

How would you handle concurrent expense updates in a multi-user environment?

Medium · Machine Coding and System Design

Approach

  1. Name the race first. Two requests read the same balance, each adds its own delta and writes it back, so one update is lost. The same thing happens when two users edit one expense at the same moment.
  2. For edits to one expense, use optimistic locking with a version column: UPDATE expense SET ..., version = version + 1 WHERE id = ? AND version = ?. If zero rows are affected, someone else won, so reload and either retry or return a 409 to the client.
  3. For balances, write the expense, its splits and the balance deltas in one transaction. Use an atomic UPDATE balance SET amount = amount + ? (or SELECT ... FOR UPDATE) instead of read-modify-write in application code. Lock rows in a fixed order, such as by user id, so A-B and B-A updates cannot deadlock.
  4. Another option is an append-only ledger. Each expense inserts ledger rows and balances are derived with SUM, so concurrent inserts never overwrite each other. The cost is heavier reads, or a materialised balance kept in the same transaction.
  5. In an in-memory machine-coding version, guard each group's state with a lock from a ConcurrentHashMap (computeIfAbsent per group id), or use synchronized methods on the BalanceSheet. Attach an idempotency key to each add-expense request so a client retry does not double-charge.

Follow-up

  • When would you choose pessimistic locking over optimistic locking here, and what changes under heavy contention on one group?
  • A mobile client times out and resends the same expense. How do you make sure it is recorded only once?
  • How would you detect and recover from a deadlock between two balance updates?

Explain the trade-offs between using an in-memory data store versus a relational database for this application

Medium · Machine Coding and System Design

Explain the trade-offs between using an in-memory data store versus a relational database for this application.

Approach

  1. An in-memory store (HashMaps in a machine-coding solution, or Redis) gives fast reads and writes and no schema to manage. A plain in-process map loses everything on restart and is not shared when the app runs as more than one instance.
  2. Redis can persist data through RDB snapshots or an append-only file, but each has a durability window depending on fsync settings. MULTI/EXEC does not roll back when one command fails, and Redis has no foreign keys or cross-row constraints.
  3. A relational database gives ACID transactions, so an expense, its splits and the balance changes commit together. It also gives foreign keys and constraints, and joins and aggregates for balance and history queries. The cost is higher latency per operation and schema migrations.
  4. For money owed between people, keep the relational database as the system of record. Cache derived data, such as per-user totals, in memory or Redis, and invalidate or update it after the database transaction commits.
  5. In the machine-coding round, put storage behind a repository interface and start with an in-memory implementation. Explain how a SQL-backed implementation would replace it without changing the services.

Follow-up

  • An expense is edited after its balance was cached. How do you keep the cache consistent with the database?
  • What happens to your in-memory version if the process crashes halfway through adding an expense?
  • At what point, if ever, would a document or key-value store be the right primary store for this app?

Archive a resource graph without breaking live references or recursing

Medium · graph traversal · topological order · tenant isolation

Resources reference other resources within a tenant; for the largest tenant the reference table holds up to 2,000,000 nodes and 8,000,000 edges. Archiving a resource must archive everything reachable from it that nothing outside the set still references, refuse when a live external referrer exists, and terminate when references form cycles, which they legitimately do. Produce the archive order and the refusal list, targeting O(V+E). Say what stops the traversal crossing a tenant boundary, and why recursion is the wrong control structure at this size.

Approach

  1. Load the subgraph with the tenant predicate on both endpoints of the edge, not only on the side you started from. Scoping the left table alone is the classic cross-tenant leak: one mis-entered edge then pulls another tenant's resources into the traversal and, worse, into the archive.
  2. Traverse iteratively with an explicit stack. A 2,000,000-node graph can hold a chain deep enough to exhaust a native stack in the low tens of thousands of frames, and that failure is a process crash rather than an error you can return.
  3. Treat cycles as data rather than corruption: compute strongly connected components with Tarjan in O(V+E) using its own explicit stack, then condense. The condensation is a DAG, so a topological order over it gives the archive order, and every member of a component archives in one transaction because no order within a cycle is valid.
  4. Decide refusals with reverse edges. A candidate is archivable only if every in-edge originates inside the candidate set, so build the transpose or count in-degrees restricted to the visited set, and emit each blocked resource with the id of the external referrer, which is the only part of the answer an operator can act on.

Follow-up

  • The graph is read in one query and the archive writes a minute later. What can change in between, and how do you make the write safe?
  • The candidate set is 400,000 resources. Is that one transaction, and if not, what does a half-finished archive look like to a reader?

Propose a database schema for tracking group balances and individual debts.

Medium · Machine Coding and System Design

Approach

  1. List the core tables: users; groups; group_members(group_id, user_id) with a composite primary key; expenses(id, group_id, paid_by, amount_minor BIGINT, currency CHAR(3), split_type, created_by, created_at); expense_splits(expense_id, user_id, owed_minor) with primary key (expense_id, user_id); settlements(id, group_id, from_user, to_user, amount_minor).
  2. Choose between derived and stored balances. Derived means a user's net in a group is SUM(paid) minus SUM(owed) plus settlements, which is always correct but costs more per read. Stored means a balances(group_id, user_a, user_b, amount) table with user_a < user_b and a signed amount, updated in the same transaction as the expense.
  3. Use integer minor units or DECIMAL for money, never FLOAT. Add foreign keys and CHECK (amount_minor > 0). A CHECK cannot compare values across rows, so enforce 'splits sum to the total' in the transaction that writes them.
  4. Index for the reads you know about: expenses(group_id, created_at) for the group feed, and expense_splits(user_id) for a user's outstanding balance. When aggregating, join payments and splits in separate subqueries, or the one-to-many join will inflate the sums.
  5. Cover the edge cases: multiple payers on one expense (an expense_payers table), soft deletes or reversal rows so history stays auditable, and expenses between two people outside any group (nullable group_id).

Follow-up

  • Write the query that returns a user's total outstanding balance across all their groups.
  • How would the schema change to support expenses in different currencies?
  • If you keep a stored balances table, how do you check that it still matches the expense history?

Find version gaps and relay lag with window functions

Medium · window functions · event tables · frame semantics

outbox_event holds event_id, aggregate_type, aggregate_id, aggregate_version, event_type, payload, status ('pending','published','dead'), attempts, created_at, published_at. A projection is missing rows and you must decide whether the relay skipped events or the consumer dropped them. Write three queries over the last seven days: one listing every aggregate_id whose published aggregate_version sequence has a hole, one giving per-day counts with a running total, and one returning the newest published event per aggregate. For each, say where the window function is evaluated relative to WHERE and LIMIT. PostgreSQL 16.

Approach

  1. Gaps: compute lead(aggregate_version) OVER (PARTITION BY aggregate_id ORDER BY aggregate_version) in a subquery, then filter next_version <> aggregate_version + 1 in the outer query. Window functions are evaluated after WHERE, GROUP BY and HAVING and before the outer ORDER BY and LIMIT, so the predicate cannot sit in the same WHERE clause and PostgreSQL 16 has no QUALIFY.
  2. Say what the seven-day filter does to the answer: it truncates every partition, so the first row per aggregate has no predecessor inside the window and a hole spanning the boundary is invisible. Widen the window, or join to resource.version as the authority for the true maximum.
  3. Running total: SELECT date_trunc('day', created_at) AS d, count() AS n, sum(count()) OVER (ORDER BY date_trunc('day', created_at) ROWS UNBOUNDED PRECEDING). An aggregate inside a window call is legal because grouping runs before windowing. The grouping key is unique per row here so ROWS and RANGE agree, but write the frame anyway — over ungrouped rows with tied timestamps the default RANGE frame pulls in every peer row and the total jumps.
  4. Newest per aggregate: DISTINCT ON (aggregate_id) ... ORDER BY aggregate_id, aggregate_version DESC is the cheap PostgreSQL-only form when an index matches that order; row_number() OVER (PARTITION BY aggregate_id ORDER BY aggregate_version DESC) = 1 is the portable form and needs a subquery for the same evaluation-order reason as the gap query.

Follow-up

  • Relay failover redelivers events. Does a duplicate break the gap query, and how would you detect one from this table alone?
  • Turn the gap check into a continuous monitor rather than a query someone runs after an incident. What does it watch?

Design the core functionalities of a system like Splitwise.

Medium · Machine Coding and System Design

Approach

  1. Pin the scope out loud before coding: users, groups, adding an expense with equal, exact and percentage splits, viewing balances and settling up. Then name the classes: User, Group, Expense (payer, total, participants), Split, BalanceSheet and an ExpenseService that coordinates them.
  2. Store money as integer minor units (paise or cents) or BigDecimal, never double. An equal split of 100 across 3 people gives 33, 33 and 34, so say which participant absorbs the remainder.
  3. Validate each split type before touching balances: exact amounts must sum to the total and percentages must sum to 100. Reject the expense with a clear error rather than partly applying it.
  4. Keep a net balance per pair of users (or per user within a group), updated on every expense and settlement. Each split adds what the participant owes the payer, and the payer's own share cancels out.
  5. For simplifying debts, compute each user's net position, then repeatedly match the largest creditor with the largest debtor using two heaps. That is O(n log n) and needs at most n-1 payments. Finding the true minimum number of payments is NP-hard, so say the greedy result is good enough.

Follow-up

  • How would you add a new split type, such as shares or itemised amounts, without editing the existing split logic?
  • How do you handle editing or deleting an expense that has already changed several balances?
  • Walk me through your debt-simplification output for a group where A owes B 50 and B owes C 50.

How do you ensure your code remains maintainable as new features (e.g., currency conversion) are added?

Medium · Machine Coding and System Design

Approach

  1. Put each split type behind a SplitStrategy interface with validate() and compute() methods, and choose the implementation through a registry or factory keyed by split type. A new type then means a new class, and ExpenseService does not change (the open/closed principle).
  2. Model money as a Money value object holding minor units and a currency code, with arithmetic that refuses to mix currencies. Currency conversion then has one place to live, and minor units vary by currency (JPY has none, INR and USD have two).
  3. For currency conversion, store the original amount and currency, plus the exchange rate and converted amount at the time of the expense. Old balances then do not change when rates move. Decide whether balances stay per currency or are converted to a group base currency, and state the trade-off.
  4. Inject dependencies through interfaces, such as an ExchangeRateProvider and a repository for storage. Tests can then use fixed rates and an in-memory store, and the real implementations can be swapped in without touching business logic.
  5. Keep rounding rules explicit and in one place, for example BigDecimal with RoundingMode.HALF_EVEN, or a documented rule for who absorbs the leftover minor unit. Write a unit test per strategy for totals that do not divide evenly.

Follow-up

  • The exchange-rate provider is down when a user adds a foreign-currency expense. What does your code do?
  • Should a group's balances be shown per currency or in one base currency, and who decides?
  • Show me where a new 'split by shares' type would plug into your class design.

Read latency spikes on a sixty-second sawtooth

Medium · cache stampede · single-flight · ttl

The cached listing read path serves about 14k reads/second at an 85% hit rate. p99 sits at 35 ms for 57 seconds, jumps to 900 ms for 3, and repeats. During each spike the primary shows several hundred identical listing queries starting within the same millisecond, all carrying one large tenant's id. Cache entries use a 60-second TTL. Give the mechanism, the ordered checks, the fix, and the correctness hazard your fix must not introduce.

Approach

  1. Match the period to a configured number before theorising about load. A spike every 60 seconds against a 60-second TTL is an entry expiring, and you confirm it by correlating spike timestamps with the entry's write time rather than with the traffic curve. If the period had matched a cron or a GC interval instead, this is a different investigation.
  2. Establish the concurrency of the miss. Several hundred identical queries in one millisecond means the miss path has no coalescing: every request that arrives between expiry and repopulation recomputes. The herd size is that key's arrival rate times its recompute time, so at 1.2k reads/second for the hot key and a 250 ms recompute you expect about 300 concurrent misses, which matches what is observed.
  3. Add single-flight on the miss path so one caller per key recomputes under a short-lived lock while the rest wait for its result. Prefer stale-while-revalidate where the read tolerates it: return the expired value immediately and refresh asynchronously, which removes the latency spike rather than serialising it into a queue of waiters.
  4. De-synchronise the keys. Write TTLs with jitter, for example 60 seconds plus or minus 10%, so a deploy or a mass invalidation does not align every key on the same second and turn a per-key herd into a fleet-wide one.

Follow-up

  • The same sawtooth appears on a key that is invalidated on write rather than expired. Is that the same bug?
  • How does your answer change if the recompute takes 4 seconds instead of 250 ms?

Suggested practice checklist

The plan builds one Splitwise-style solution over seven days, adding the pieces the reported questions ask about in turn: split logic, balances and API, schema, concurrency, then extensibility and storage trade-offs, ending in a full timed run.

Day 1 — Confirm the format and write the requirements checklist

  • Write to your recruiter asking for the sequence of stages, whether you will write and run code, which languages are allowed and what tools you get
  • Write a requirements checklist for a Splitwise-like application covering users, groups, payer, equal/exact/percentage splits, balance views, settling up and currency
  • For each checklist item, write the assumption you would state aloud and one edge case it creates

Deliverable: A list of format questions for the recruiter and a one-page requirements checklist with your stated assumptions and edge cases.

Day 2 — Model the domain and the split strategies

  • Define User, Group, Expense, Split and a Money type holding integer minor units and a currency
  • Implement equal, exact and percentage split strategies behind one interface, each validating its own input
  • Write tests for 100.00 split three ways, exact shares that miss the total and percentages that do not reach 100

Deliverable: A runnable module with three split strategies, a Money type and a passing test file covering rounding and invalid input.

Day 3 — Balances, settlement and the API surface

  • Compute each user's net balance per group from an append-only list of expenses and settlements
  • Implement debt reduction with a max-heap of creditors and a max-heap of debtors, matching the largest of each, which gives at most n-1 transfers but not always the fewest
  • Sketch REST endpoints for adding an expense, listing group balances, viewing a user's total outstanding balance and recording a settlement, with request and response shapes

Deliverable: A balance service with tests, plus an endpoint list showing method, path, request body and response for each of the four operations.

Day 4 — Schema for group balances and debts

  • Write tables for users, groups, group membership, expenses, expense splits and settlements, with keys, foreign keys and a check that split rows sum to the expense amount where practical
  • Write the query returning one user's net balance in a group, and decide whether balances are always derived or also cached

Deliverable: DDL for the schema, the net-balance query, and a short written decision on derived versus cached balances.

Day 5 — Concurrent expense updates

  • List the shared state in your solution and write out the lost-update case where two users edit the same group
  • Add a per-group lock to the in-process version and write a multi-threaded test that fails without it and passes with it
  • Write the database-level answer: a version column with compare-and-set, SELECT ... FOR UPDATE on the group row, and an idempotency key for retried requests

Deliverable: A threaded test that reproduces the race and then passes, and a written comparison of in-process lock, optimistic version check and row lock.

Day 6 — Extensibility and storage trade-offs

  • Add currency conversion to your Day 2 code through a rate-provider interface and count how many existing classes you had to change
  • Write a comparison of an in-memory store against a relational database for this application, covering durability, constraints, concurrency and scale-out
  • Answer both questions aloud, then listen back and cut anything that is not tied to the Splitwise requirements

Deliverable: A diff showing currency conversion added with minimal edits to the split strategies, and a one-page in-memory versus relational comparison with your recommendation.

Day 7 — Full timed run and behavioral rehearsal

  • Starting from a blank file and a time limit you set yourself, build the Splitwise-style core while asking your requirement questions aloud and explaining each class as you add it
  • Afterwards, answer the schema and concurrency questions out loud without notes

Deliverable: A working, tested solution from the timed run, a recording or notes of your spoken answers, and a list of three weak points to fix.

Behavioral prompts for a product-engineering role

No behavioral round is reported for this role. Candidates do describe the work as writing code, taking part in code reviews, turning requirements from product managers into technical specifications and discussing trade-offs with senior engineers, so these prompts rehearse stories from that kind of work. Each story should name the decision you made, the alternatives you rejected and the outcome.

  1. Tell me about a time you turned a vague product requirement into a technical design. What did you ask first, and what did you leave out of version one?
  2. Describe a code review in which you disagreed with a senior engineer about a trade-off. What did each side argue and how was it resolved?
  3. Tell me about a design you built that had to absorb a feature nobody planned for. What made the change easy or hard?
  4. Describe a bug involving two users or two processes changing the same data. How did you find it, and what did you change to prevent it?

Estimate work you have never done and defend the range

Hard · estimation · backfills · expand-contract

You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.

Approach

  1. Decompose into independently deployable steps before estimating anything: add the column nullable, write both shapes, backfill in batches, verify, move reads, stop writing the old shape, drop it. That is four deploys spread over days, and the calendar estimate is dominated by them rather than by the loop's runtime.
  2. Do the arithmetic aloud for the part that has arithmetic in it: batch size times number of batches times per-batch duration, at a write rate the primary can absorb alongside roughly 1.2k writes per second of production traffic. The loop is throttled by replication lag and lock waits, not by how fast it can issue statements.
  3. Price the schema step by its lock rather than its statement duration. In PostgreSQL an ALTER TABLE taking ACCESS EXCLUSIVE waits for every open transaction on that table while later queries queue behind it, so a millisecond change issued during a thirty-second analytics query stalls that table for thirty seconds. Adding a nullable column with a non-volatile default avoids a rewrite from version 11; a new index wants CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an invalid index behind if it fails.
  4. Express the answer as a range whose endpoints each trace to a stated assumption, then name the cheapest experiment that collapses it, which is almost always running one real batch against the real table and multiplying.

Follow-up

  • How do you verify the backfill genuinely finished, given rows written by production traffic while it ran?
  • Where does the backfill resume from after a worker is killed mid-batch, and what makes that resume point trustworthy?

Tell callers you do not own that their integration breaks

Medium · deprecation · compatibility · stakeholders

A field in a write endpoint's response must change shape. You own the endpoint; you do not own the four internal callers or the outbound webhook consumers who read it. Describe a deprecation you were responsible for: what you shipped first, how you established who was actually reading the field, the window you gave and what set its length, what you did about the consumer who never moved, and how you decided removal was safe. Name the signal you used, not the announcement you sent.

Approach

  1. Establish the reader set empirically rather than from a wiki of owners: per-field usage counters keyed by principal, or access logs attributed to a consumer. State the blind spot of whichever you pick, since a consumer that reads the field only on a monthly job will not appear in a week of logs.
  2. Ship additive first. Populate the new field alongside the old one so no reader is forced to move, which is also what keeps a rolling deploy safe, because old and new instances answer the same requests at the same time and a rollback must still find the old shape present.
  3. Set the window from the slowest legitimate consumer's release cadence, not from your calendar, and decide separately what to do for a consumer with no release process at all, such as an external webhook endpoint you can only email.
  4. Convert silence into evidence before you rely on it: a short, low-traffic removal window that makes a still-dependent consumer fail visibly and loudly while you are watching, rather than at three in the morning after you have moved on.

Follow-up

  • How would you detect a consumer that reads the field only during a monthly export?
  • One caller refuses to move and has a commercial relationship behind it. What changes in your plan and what does not?

Unblock an engineer without taking the keyboard

Easy · mentoring · leases · at-least-once

A teammate has spent two days on a job handler that occasionally writes duplicate rows. They are certain the queue is delivering twice by mistake. You suspect a lease expiring under a slow handler, so the job is running concurrently with itself. Describe how you have unblocked someone in this position: what you asked before offering a hypothesis, what you showed them rather than told them, and what you left them owning. Then say what you would do if their theory turned out to be the right one.

Approach

  1. Ask before diagnosing, and ask for things answerable from data they already have: the attempt count on the job rows that produced duplicates, the handler's observed duration against its lease expiry, and whether the duplicate rows share a natural key that a unique constraint could have caught.
  2. Teach the shape rather than the answer. A lease cannot distinguish a dead worker from a slow one, so a handler that outruns its lease is running twice by design, and deploys deliver the other half by killing handlers mid-run on every rollout. Both of their candidate theories produce identical duplicate rows, which is why the evidence has to come from timings rather than from argument.
  3. Hand over a checklist they execute: a natural key on every write the handler performs so the second copy collides rather than appends, the record of intent written before any external effect, a lease heartbeat while running, and the metric that shows it working.
  4. Keep ownership with them deliberately. Pair on the first write, then step back; if you finish it yourself you have closed one ticket and left the same person stuck on the next redelivery.

Follow-up

  • How would you distinguish a genuine double-delivery from a lease expiry using only the data already stored?
  • Their handler calls an external endpoint before recording that it did. What do you tell them to change first?