Interview guides · Abzer Technology Solutions

Abzer Technology Solutions Software Engineer Interview Questions & Guide 2026

Abzer Technology Solutions software engineer prep on databases: indexes, query plans, and the anomalies an isolation level still permits.

The 60-second brief

A Software Engineer at Abzer Technology Solutions plays a key role in maintaining the technical integrity and operational efficiency of the company's core systems. Whether you are working as a Control-M Developer or a Level 3 Application Support Engineer, your work directly affects the company's ability to deliver reliable, high-performance solutions to its clients. You are the bridge between complex system architecture and seamless user experience.

Allocate preparation against your weakest link rather than your favourite topic. A strong algorithm habit usually comes with weak out-loud explanation of tradeoffs, and years of shipping usually come with rusty from-scratch implementation under a clock.

Focus areas

Scope every query and cache key by tenant
Evolve APIs without breaking pinned SDK clients
Make every write idempotent under client retries

Overview

This role is inherently collaborative and intellectually demanding. You will navigate intricate software environments, troubleshoot critical production issues, and contribute to the evolution of the company's technical stack. The position lets you influence the stability and scalability of the company's services, and suits engineers who work well in fast-paced, high-impact environments.

Where to focus your preparation

Treat the sections below as preparation areas and confirm the format with your recruiter.

Preparation focus

No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.

What to demonstrate

Breadth across SQL, experimentation and product reasoning
Ability to state assumptions before choosing a method

How to prepare

Drill the practice exercises below and time yourself
Prepare three quantified stories about decisions you drove

Where candidates lose points

Serialising a tenant's writes through select ... for update on a single counter row

It is the first change that makes a counter correct, and it caps that tenant's write throughput at roughly one divided by the lock hold time. A transaction that takes the lock, makes a network call and then commits holds it for the entire round trip: at 2 ms that is about 500 writes per second for the whole tenant, and the largest tenants are exactly the ones that exceed it. The damage then spreads, because every waiter holds a database connection while it queues, so one hot tenant drains the shared pool and the symptom presents as a site-wide latency incident rather than as a lock problem. The repairs are to shrink the critical section to a single statement, to shard the counter into per-(tenant, hour) or per-(tenant, bucket) rows and sum on read, or to batch in memory and flush periodically while accepting the bounded loss that batching implies.

Checking a quota with a select and then writing

Under read-committed isolation, two concurrent transactions both observe a count below the limit and both insert, so the limit is exceeded by exactly the concurrency. Repeatable read does not rescue it either: it provides a stable snapshot, and this is write skew, which snapshot isolation permits by design. The options are serialisable isolation, which detects the conflict and aborts one transaction with a serialisation failure and therefore obliges the caller to retry; a single statement with the predicate inside the write; or a constraint that makes the surplus insert fail outright. The reason this pattern survives review is that it is correct in every test that runs one request at a time.

Writing code before the input contract is pinned down

Before the first line, state the types, the size bounds, whether duplicates, negatives or an empty input are possible, whether the input is sorted, whether you may mutate it, and what the function returns when nothing matches. Every one of those answers changes the code, and discovering one at minute twenty costs a rewrite you no longer have time for.

Issuing one query per row of a result set

Fetch related rows in a single batched query keyed by the ids you already hold, or join them into the original query. A per-row round trip multiplies network latency by the row count, and it looks perfectly fine against the ten rows in your development database.

Questions & practice

Fold a deduplicated usage stream into hourly rollups

Easy · aggregation · deduplication · watermarks · exact-arithmetic

You are given one day of usage_event rows, up to 250 million, each carrying event_id, tenant_id, workspace_id, environment, sku, quantity numeric(20,6), idempotency_key, occurred_at and ingested_at. Produce usage_rollup_hourly cells keyed (tenant_id, workspace_id, sku, hour_start) with quantity_sum, event_count and source_max_ingested_at. An event counts once per (tenant_id, idempotency_key). The rollup grain has no environment column, so state your filter. One pass. Give your time and space bounds, and say what the deduplication actually costs in memory.

Approach

Bucket on occurred_at, never ingested_at: hour_start = date_trunc('hour', occurred_at at time zone 'UTC'). The two columns answer different questions. occurred_at says which hour the customer is billed for; ingested_at says how current the fold is. Using the second for the first makes late data invisible instead of correctable.
The fold is trivial and the deduplication is the entire cost, so price it before designing anything clever. An exact set over (tenant_id, idempotency_key) at 250M entries, stored as a 16-byte 128-bit hash in an open-addressed table at 0.7 load factor, needs about 357M slots at 16 bytes each, roughly 5.7 GB. The fix is partitioning by hash(tenant_id) % P so each shard holds 1/P of the set and no tenant's keys straddle shards.
Rule out a Bloom filter as a replacement, in the right direction: a false positive reports 'already seen' for an event never seen, so you drop a real event and lose revenue with no error raised. It is usable only as a negative pre-filter in front of the exact set, where a miss is conclusive and a hit must fall through to the real lookup.
Accumulate in scaled integers, not binary floating point. numeric(20,6) admits values below 10^14, so one event scaled to micro-units can reach 10^20, past int64's 9.22 x 10^18; use a 128-bit or arbitrary-precision accumulator unless you first bound the per-event maximum. binary64 represents integers exactly only to 2^53, about 9.01 x 10^15, and cannot represent 0.1 at all, so two runs that sum in different orders disagree.
Carry source_max_ingested_at = max(ingested_at) over the events folded into each cell, and count event_count over accepted, post-dedup events. Without that watermark there is no way to prove later what a number did and did not include, which is the first question any reconciliation asks.
State the environment filter explicitly, because the rollup grain cannot record it. A fold that quietly includes staging bills non-production traffic; one that quietly excludes it loses a cost signal. Production-only is the billing answer, and either way it belongs in the job name and the output metadata. Complexity: O(n) time, O(distinct dedup keys) space, dominated by the dedup set rather than by the cells.

Follow-up

A producer retries at 23:59:59 and the retry lands at 00:00:01. The unique index on the daily-partitioned table must include the partition key. What gets double-counted, and what is the smallest change that fixes it?
The consumer acknowledges its batch before committing the fold. Which failure loses revenue now, and which arrangement duplicates instead?
What makes a re-run over the same day produce byte-identical rollups?

Parse and verify a timestamped multi-signature webhook header

Easy · parsing · hmac · constant-time-compare · replay-protection

An inbound webhook carries a signature header of at most 1 KiB shaped t=<unix seconds>,v1=<64 hex chars>, with up to five v1 values during secret rotation and possibly unknown scheme keys. You hold the raw request body bytes and the currently active signing secrets. Write the parser and the verifier: accept when any active secret reproduces a signature and the timestamp is within a five-minute tolerance in either direction, reject otherwise. Single left-to-right pass over the header, no regular expression. State what is inside the MAC and why.

Approach

Parse in one scan: split on ,, then on the first = only, since a value may itself contain = under a future scheme. Accept t exactly once and treat a second t as a reject rather than last-wins. Push every v1 onto a short list and ignore any other key, so a v2 can be introduced later without breaking this verifier.
Say what is signed: HMAC-SHA256 over the exact byte string <t>.<raw body bytes>, yielding 32 bytes or 64 hex characters. The timestamp sits inside the MAC because otherwise an attacker replays yesterday's body with its still-valid signature and only has to edit the header timestamp.
Hash the bytes as received. Verifying against a re-serialised JSON body is the usual defect: key order, whitespace and number formatting all change the bytes while the parsed objects compare equal, so signatures fail for honest senders and the popular 'fix' is to stop checking.
Compare in constant time over fixed-length digests. Decode the hex to 32 bytes, accumulate acc |= a[i] ^ b[i] across the whole length, and test acc == 0 at the end. Evaluate every candidate without an early exit; at five candidates that is five HMACs over the body, linear in body size and negligible beside the network.
Apply the tolerance as a two-sided bound, rejecting when |now - t| > 300 seconds. A sender whose clock runs ahead of yours is an ordinary case, and an unbounded future timestamp is a free replay window.
Complexity: O(L) over the header producing k candidates, plus k HMACs at O(|body|) each. Space is O(k) beyond the body itself. Do the cheap rejections, including the tolerance check, before any cryptography runs.

Worked solution (15 min)

Write the grammar on one line before coding: header := field (',' field)*, field := key '=' value, split on the first = only.
Implement the parser to return {t: int, v1: [hex, ...]}, rejecting a missing t, a duplicate t, any v1 that is not 64 hex characters, and a header over 1 KiB, all before any cryptography runs.
Implement the verifier: for each active secret compute HMAC-SHA256(secret, f'{t}.'.encode() + raw_body), compare it in constant time against each parsed v1, and OR the results with no early exit.
Test with a valid signature; the same body with t moved 400 seconds into the past; the same body with t 400 seconds into the future; a header carrying an unknown v2= alongside a valid v1; and a body re-serialised with different JSON key order.

Expected result: The valid case accepts. Both out-of-tolerance cases reject, including the future one. The unknown v2 field is ignored and the v1 still verifies. The re-serialised body fails, which is correct and is exactly why the raw bytes must be retained.

Verify it yourself

Timing the comparison over 10,000 mismatching signatures that differ at the first character versus the last shows no separation you can measure.
Removing t from the MAC input makes an old captured body with its original signature pass. Write that test, watch it pass, then put t back.
Every rejection except the signature mismatch itself returns before an HMAC is computed.

Follow-up

The body is 40 MB. What changes about where you verify, and what can you do before the whole body has arrived?
A customer reports that signatures fail for exactly the requests whose body contains a non-ASCII character. What is your first hypothesis?
How do you rotate the signing secret with no failed deliveries, and how long do both secrets stay live?

Order a job dependency graph and find its critical path

Medium · topological-sort · dag-longest-path · cycle-detection · critical-path

A workspace defines up to 50,000 jobs with up to 200,000 dependency edges and an estimated duration_seconds per job. Given the edge list, reject the graph if it contains a cycle and name one cycle's nodes; otherwise return a valid execution order, the earliest possible completion time with unlimited workers, and the set of jobs whose slack is zero. Then say which single job to shorten in order to cut the completion time, and by exactly how much. State the complexity of each part.

Approach

Kahn's algorithm for the order: compute indegrees, seed a queue with zero-indegree nodes, emit and decrement. O(V + E), which at 50,000 and 200,000 is milliseconds. If fewer than V nodes are emitted, the graph contains a cycle.
Kahn detects a cycle but cannot name one. The nodes left with indegree above zero contain every cycle, so run one DFS restricted to that residual subgraph with three-colour marking and report the stack slice from the grey node the back edge points at. That is the difference between a usable error message and 'dependency cycle detected'.
Earliest completion with unlimited workers is the longest path, which is NP-hard on a general graph and linear on a DAG. State the precondition, then relax in topological order: earliest_finish[v] = duration[v] + max(earliest_finish[u] for u in preds(v)), taking the max over an empty predecessor set as zero. The makespan T is the maximum over all nodes. O(V + E).
Second pass in reverse topological order for latest_finish, then slack[v] = latest_finish[v] - earliest_finish[v]. Zero-slack nodes form the critical path, and there can be several disjoint critical paths, so return the set rather than one chain. slack[v] = 0 is exactly the statement that some longest path runs through v; equivalently, the longest path through v has length T - slack[v].
The speed-up bound is the point of the question, and the obvious form of it is wrong. Shortening a zero-slack job v by d, with 0 <= d <= duration[v], cuts the makespan by min(d, T - L_avoid(v)), where L_avoid(v) is the longest path in the graph with v deleted: the longest path that avoids v, not the second-longest path overall. The two coincide only when the runner-up path misses v. Counterexample: A of 10 s feeds both B of 5 s and C of 4 s, so T = 15 s and the second-longest path is 14 s, yet shortening A by 10 s leaves a makespan of 5 s. The realised gain is the full 10 s, because both paths ran through A and shrank together, while min(10, 15 - 14) predicts 1 s. The reason is structural: shortening v reduces every path through v by d and leaves every other path alone, so the new makespan is max(T - d, L_avoid(v)).
Compute L_avoid(v) the direct way: delete v and re-run the same forward relaxation, O(V + E) per candidate. The cheaper equivalent skips the deletion, since L_avoid(v) only ever matters through that max: set duration[v] := 0, recompute the makespan as T0(v) = max(T - duration[v], L_avoid(v)), and the gain is min(d, T - T0(v)), which is identical for every d <= duration[v]. Only zero-slack jobs are candidates, because shortening a job with positive slack changes the completion time not at all. One relaxation is milliseconds at this size, so ranking a critical set in the hundreds costs O(k(V + E)) and is worth doing exactly; a critical set in the tens of thousands is not, and there you evaluate a shortlist, longest jobs first, and say that the answer is the best of that shortlist rather than the optimum.

Follow-up

Only m workers are available. What happens to your answer, and what can you still promise about the schedule you produce?
Edges arrive incrementally as the customer edits the pipeline. How do you detect a cycle at insert time without re-running Kahn over 250,000 elements?
Durations are estimates. How would you express completion time as a distribution, and what breaks about the critical path once you do?

Decide which facts an invoice line copies instead of joining

Medium · denormalisation · immutable records · foreign keys · rounding

invoice_line_item already denormalises tenant_id, which is reachable through invoice_id, and stores amount_minor even though quantity times unit_price_micros would recompute it. A reviewer asks you to normalise both away, and separately asks whether the tenant's legal name and billing address should be copied onto the invoice header. Decide each case. For every field you keep denormalised, name the read pattern or the invariant that justifies it, the anomaly the copy can develop, and the mechanism that prevents that anomaly here.

Approach

Split the question into two kinds of copy, because they fail differently. A copy of a currently mutable fact is a cache: it drifts and needs invalidation. A copy of a fact frozen at write time is not a cache at all, it is the record of what happened, and normalising it away destroys information the source no longer holds.
Keep tenant_id on the line. It costs 8 bytes, it leads every index on the table so no read is ever accidentally cross-tenant, and it turns a wrong join into an empty result rather than another tenant's money. Prevent the drift structurally: a unique constraint on invoice (invoice_id, tenant_id) plus a composite foreign key from the line on (invoice_id, tenant_id) makes a mismatched pair impossible, so the database enforces agreement instead of a code review.
Keep amount_minor. Rounding must happen exactly once, at a named site, with a stated mode (half-even here). If readers recompute from quantity and unit_price_micros, every reader owns a rounding decision, and half-up and half-even diverge systematically across thousands of lines rather than cancelling out. A check constraint can bound the stored value but deliberately cannot re-derive it.
Copy the legal name and billing address onto the invoice header, written once and never updated. The statement must show what was true when it was sealed, and the tenant record will change afterwards. This is a snapshot for the same reason source_rollup_watermark is stored per line: without it, nobody can reconstruct what the customer was told.
Name the read pattern that pays for all of it. Rendering, dispute response and export are per-tenant, per-period reads over thousands of lines that would otherwise join back to slowly changing dimensions that no longer hold the historical value. The write side is a once-per-period batch, so the extra columns cost nothing that matters.
Concede the case where the reviewer is right: a mutable operational attribute such as the tenant's current plan name has no business on a line. If a report wants it, join. If a statement needs the plan as of the period, that is another snapshot and it belongs on the header with the rest.

Worked solution (25 min)

Write the DDL: unique (invoice_id, tenant_id) on invoice, the composite FK from the line, and a comment on each denormalised column saying whether it is a snapshot or a cache.
Attempt to insert a line whose tenant_id differs from its invoice's and confirm the foreign key rejects it.
Rename a tenant, re-render a sealed invoice, and confirm the rendered name is the one stored on the header.
Recompute amount_minor from quantity times unit_price_micros for a thousand synthetic lines rounding half-up, sum both ways, and record the divergence from the stored half-even values.

Expected result: tenant_id kept and enforced by a composite foreign key to (invoice_id, tenant_id); amount_minor kept because rounding happens once with a stated mode; legal name and billing address snapshotted onto the invoice header and never updated; mutable operational attributes left normalised. Each decision paired with its read pattern, its anomaly, and the mechanism that prevents the anomaly.

Verify it yourself

A line cannot be attached to an invoice belonging to another tenant, and the rejection comes from the database rather than the application.
Re-rendering a sealed invoice after editing the tenant record produces identical output.
Rounding per line and rounding once at the end differ measurably over a thousand lines, which is the argument for storing the value.
No column on the line is recomputed by any reader.

Follow-up

Write the composite foreign key and the unique constraint it requires on the parent. What does it cost on every line insert, and what does it do to a bulk load?
A tenant is renamed after being invoiced. Which rows change, and what does the customer see on last quarter's PDF?
Where does currency live, and what breaks if a tenant's billing currency changes between two periods?

Explain why the metering dashboard scans every daily partition

Medium · indexing · partition pruning · sargability · tenant skew

usage_event is range-partitioned daily on ingested_at and holds tenant_id, workspace_id, environment, sku, quantity numeric(20,6), occurred_at and ingested_at. The only relevant index is on (occurred_at). A dashboard runs select sku, sum(quantity) from usage_event where tenant_id = $1 and date_trunc('hour', occurred_at) >= $2 and environment = 'production' group by sku, and EXPLAIN shows a sequential scan of every partition. Give each distinct reason, rewrite the predicate so an index can serve it, propose the index, and state the write cost its column order adds.

Approach

Separate the three causes rather than blaming one. First, date_trunc('hour', occurred_at) wraps the column, so the predicate is not sargable against a btree on the bare column. Second, pruning keys off ingested_at while the query constrains occurred_at, so no partition can be excluded. Third, even made sargable, (occurred_at) is not tenant-leading, so for one tenant among thousands the scan reads the whole time range and discards nearly all of it.
Rewrite the bound carefully, because the obvious rewrite is only conditionally equivalent. date_trunc('hour', x) >= $2 equals x >= $2 only when $2 is already hour-aligned; for an arbitrary $2 it means x >= date_trunc('hour', $2) + interval '1 hour'. Normalise the parameter in the caller and leave the column bare.
Restore pruning with a second, redundant predicate on the partition key: ingested_at >= $2 - interval '<late-data horizon>'. State both sides of it. It prunes to a handful of partitions, and it silently omits any event whose ingest lagged past that horizon, which is precisely what a producer replay produces. Either document the horizon as a stated bound, or partition on occurred_at and move the problem into the dedup window instead.
Propose (tenant_id, occurred_at) include (sku, quantity) per partition. A partial index where environment = 'production' mostly saves size rather than selectivity, since production dominates the three environments; take it if non-production is a meaningful share and skip it otherwise.
Price the write path honestly. At roughly 250M rows/day each extra index is another insert plus WAL per row, and a tenant-leading key scatters inserts across one hot leaf per active tenant instead of appending to a single rightmost leaf, so page dirtying and random I/O both rise. An INCLUDE payload widens every leaf entry and enlarges the index accordingly.
Add the index-only-scan caveat before someone reports it as a regression: on a freshly appended table the visibility map is not yet set for recent pages, so the INCLUDE columns still cost heap fetches until autovacuum has been through, and the newest hour is exactly the data the dashboard reads.

Follow-up

CREATE INDEX CONCURRENTLY is not supported on a partitioned parent. Give the sequence that gets this index onto 400 existing partitions without blocking ingest.
One tenant holds 200 times the median row count and the dashboard still times out for them with the index in place. What changes?
Should this read hit usage_rollup_hourly instead? State what that costs in freshness and what the watermark lets you promise.

Can you explain the lifecycle of an object in Java?

Medium

Approach

Work from the requirement backwards to the design.
Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Explain the core differences between an interface and an abstract class.

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer contains.
Say what you would check first and why it is the highest-information step.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

How do you implement inheritance in your current projects to reduce code redundancy?

Medium

Approach

Clarify what is being asked and what a complete answer contains.
Work from the requirement backwards to the design.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Describe a scenario where you utilized polymorphism to improve system flexibility.

Medium

Approach

Say what you would check first and why it is the highest-information step.
Work from the requirement backwards to the design.
State your assumptions explicitly before working the problem.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Build a resumable usage export the customer can reconcile against

Medium · pagination · cursors · late data

Customers reconcile invoices against usage_event (event_id uuid, tenant_id, workspace_id, environment, sku, quantity numeric(20,6), idempotency_key, occurred_at, ingested_at, source_service, request_id), partitioned daily on ingested_at. The current export is ?page=N&per_page=1000 ordered by occurred_at, and a customer syncing hourly reports rows that never appear in their export but do appear on their invoice. Design the replacement: the ordering, the cursor's contents, the index it requires, and the rule deciding where a page stops. State what the client does on a timeout and on a cursor older than retention.

Approach

Separate the two defects in the current shape. Offset makes the database produce and discard N * per_page rows, so page cost grows linearly and a deep page degrades from milliseconds to seconds. Concurrent inserts also shift the window between requests, so a walker skips rows with no error raised anywhere, which for a customer sync is silent data loss.
Order by ingestion, not by occurrence. During a replay events arrive hours out of occurred_at order, so a consumer holding an occurred_at high-water mark can never see a late event that falls below it; (ingested_at, event_id) is the only ordering under which 'everything after my cursor' is a complete statement.
Page with a row comparison: where tenant_id = $1 and (ingested_at, event_id) > ($2, $3) order by ingested_at, event_id limit $4, backed by an index on (tenant_id, ingested_at, event_id). That seeks directly to the resume point, so every page costs the same regardless of depth.
Trail the head of the table. With ingested_at defaulting to now(), which is transaction start time, a long insert transaction receives an earlier timestamp and becomes visible after a reader has already passed it. Cap each page at ingested_at <= now() - delta, with delta larger than the longest write transaction as bounded by statement_timeout and idle_in_transaction_session_timeout, or the export skips exactly the rows written under load.
Make the cursor opaque and self-describing: base64 of the timestamp, the event id and a fingerprint of the filters, rejected when the filters differ from the current request. Return 410 with a cursor_expired code once the cursor's partition has been dropped, so the client restarts from a known time instead of resuming into a hole.
Keep a page a pure GET with no server-side consumption, so a timeout is resolved by retrying the identical cursor.

Worked solution (30 min)

Construct the failing case on paper: an event with occurred_at at 09:00 ingested at 14:00, and a consumer that read up to 10:00 at 11:00.
Write the keyset query with the row comparison and the exact index it needs, then say which column of the index each predicate uses.
Add the trailing-head predicate and pick delta from a named timeout setting rather than a round number.
Define the cursor's encoded contents and the two error cases: filter mismatch and expired partition, with their status codes.
Write the client's algorithm in four lines: request, persist cursor after processing the page, retry the same cursor on timeout, restart from a time on 410.

Expected result: Keyset pagination over (ingested_at, event_id) with an index on (tenant_id, ingested_at, event_id), a trailing head predicate sized above the longest write transaction, an opaque filter-bound cursor, and 410 for a cursor past retention.

Verify it yourself

A late event with an old occurred_at is still delivered by the export.
Page cost is independent of how many pages have already been read.
The cursor persists across a client restart and survives retrying the same page.
The tie-break column is unique, so no row can be skipped or repeated at a page boundary.

Follow-up

The customer asks for a total count alongside the first page. What do you offer instead, and why is an exact count both expensive here and wrong by the time it is read?
How would you let a customer re-read a window they have already consumed without giving up the forward-only cursor?

A rare job-run overwrite that logging makes disappear

Hard · fencing tokens · leases · concurrency

About one job run in fifty thousand records billable_seconds matching no observed sandbox lifetime, and a few rows show worker_id changing after finished_at was already set. It does not reproduce: debug logging around the terminal write made it vanish for two weeks before it returned. Runs last from 200 ms to 30 minutes, the lease is 60 seconds and is renewed while a run executes. Give an ordered checklist, the mechanism, and a fix that makes the illegal write impossible rather than merely rarer.

Approach

Mine the evidence instead of chasing a repro: select rows where updated_at is later than finished_at, or where a terminal status was written twice, and join them to attempt history to recover both writer identities. The defect has already happened tens of times and the rows are the recording.
State the signature before measuring it. If the cause is a lease that expired while the original worker was stalled, affected runs should cluster where the gap between the last renewal and the terminal write exceeds the lease, and should correlate with worker pause metrics rather than with workload shape.
Read the disappearance honestly. Logging inside the window changed the timing and lowered the probability; it is evidence about how narrow the window is, not a fix. Reproduce by widening the window on purpose, shortening the lease and injecting a pause between sandbox exit and the terminal write, rather than by adding more instrumentation.
Name the mechanism precisely: the lease expires during a stall such as a long garbage-collection pause or a brief partition, the run is re-dispatched, and the original worker then wakes and writes its terminal state over the new attempt's row. A lease alone cannot stop this, because the check and the write are separated by the stall.
Fix by fencing the write itself: UPDATE job_run SET status = $2, finished_at = $3, billable_seconds = $4 WHERE run_id = $1 AND status = 'running' AND lease_token = $5, with zero rows affected interpreted as having been fenced rather than as success. The token lives on the row so the store arbitrates, not the worker's memory.
Keep the state machine honest: a retry inserts a new row pointing at parent_run_id rather than resetting the old one, and a run whose worker vanished terminates as lost with billable_seconds null, because recording failure asserts an outcome nobody observed and then bills and retries on that assertion.

Follow-up

The supervisor also emits a usage event on completion. What does the fenced worker do about the event it already emitted, and how does metering absorb it?
Lease renewal is itself a network call. What happens when a renewal times out, and how does the worker decide whether it still holds the lease?
Why is lengthening the lease past the longest legitimate run the wrong lever, and what breaks if you do it anyway?

Suggested practice checklist

For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.

Day 1 — Rebuild the primitives by implementing them

Implement a dynamic array with doubling growth and an operation counter, then change the growth rule to add a fixed sixteen slots instead, and time both for n of ten thousand, a hundred thousand and a million. The fixed-increment version resizes n/16 times at O(n) each, so its total work is quadratic; doubling is what makes append amortised constant.
Implement a hash map with separate chaining and a load-factor resize, then insert ten thousand keys engineered to land in one bucket and record what happens to lookup time, so that average-case O(1) becomes a claim with a stated precondition rather than a reflex.
For dynamic-array append and hash-map insert, write down which cost is amortised rather than worst-case, which single operation pays the whole bill, and what a system with a hard per-operation deadline would have to do instead.

Deliverable: Two working implementations plus a timing table showing the input at which each structure's advertised complexity stops holding.

Day 2 — Arrays under an invariant: two pointers, sliding window, binary search

Solve longest-subarray-with-sum-at-most-K using a sliding window, then run it on an input containing negative numbers and watch it return the wrong answer: extending the window only moves the sum monotonically when every element is non-negative, and that precondition is the whole reason the technique works.
Write the binary search that finds the first index satisfying a predicate rather than an exact value, put the loop invariant above the loop in a comment, and verify termination on the two inputs that break careless versions: the empty range, and a range where every element satisfies the predicate.
Compute the midpoint as lo + (hi - lo) / 2 and write one line on why the obvious (lo + hi) / 2 is a genuine defect in a fixed-width integer type and a non-issue in a language with arbitrary-precision integers.

Deliverable: Three solved problems, each with its invariant written above the loop, plus one recorded input on which the sliding window is provably wrong.

Day 3 — Sorting, heaps, and the greedy argument that has to be proved

Solve one top-k problem three ways, by full sort, by a size-k heap, and by quickselect, then write the values of n and k at which each becomes the right choice, along with quickselect's quadratic worst case and why a randomised pivot makes that unlikely rather than impossible.
Implement bottom-up heapify and count sift-down steps to confirm it does linear work rather than n log n, because most nodes sit near the bottom of the tree and therefore move only a short distance.
Take interval scheduling by earliest finishing time and write the exchange argument out in full: given any optimal schedule, swapping in the earliest-finishing interval keeps it feasible and no smaller. Then construct the weighted variant where that same greedy fails and name what has to replace it.

Deliverable: A three-way top-k comparison with measured crossover points, one written exchange argument, and one counterexample to a greedy rule that looks almost identical.

Day 4 — Recursion, memoisation, and the step to a table

Take one problem with overlapping subproblems, such as edit distance or coin change, instrument the plain recursion with a call counter to show the blow-up, then add memoisation and re-count.
Convert the memoised version to a bottom-up table and state the two properties you relied on: each subproblem's result depends only on its arguments, and the dependencies form a DAG you can enumerate in order.
Rewrite one deep recursion with an explicit stack, then find the input length at which the original hits the interpreter's frame limit, which defaults to about a thousand frames in CPython, so you know when the rewrite is required rather than decorative.

Deliverable: One problem in three forms, naive, memoised and tabulated, with call counts for each and the input length at which recursion depth becomes the binding constraint.

Day 5 — Graphs, where most of the work is choosing the traversal

Implement BFS and DFS over one adjacency list, then answer for each which finds a shortest path in an unweighted graph and which you would use to detect a cycle in a directed graph, including why the in-progress versus finished distinction matters for the second.
Implement topological sort by in-degree, feed it a graph containing a cycle, and confirm the failure signature is that fewer than V nodes come out rather than an exception, then note that the order it produces is one of several valid ones.
Run a shortest-path search on a graph with a single negative edge weight and show the wrong answer, then write the precondition Dijkstra actually needs, non-negative weights, because it finalises a node's distance the first time that node is popped, and name the algorithm you would switch to and its own limit.

Deliverable: A small graph library with BFS, DFS and topological sort, plus two inputs that produce documented wrong answers under the wrong algorithm choice.

Day 6 — One day for everything that is not an algorithm

Sketch one system only to the depth a coding-heavy loop tends to reach: the endpoints, what the service stores, and the single query pattern that decides the schema. Stop at twenty-five minutes.
Prepare the project answer for an interviewer who codes, which means rehearsing the two levels they push to: the specific thing you built, and why you chose that approach over the alternative they will name. Open with a number and be ready to say what it excludes.
Prepare the answer to what you would do differently, choosing a real technical mistake with a specific fix rather than a complaint about process or staffing.

Deliverable: One design sketch at endpoint-and-schema depth, plus a project answer rehearsed to two levels of follow-up.

Day 7 — Solve out loud, under time

Do three timed problems at twenty-five minutes each in a plain editor with no autocomplete and no execution until the end, then tally separately the failures that were syntax and the ones that were approach, because those two numbers call for different fixes.
Narrate one solution from the first sentence, stating the approach and its complexity before writing any code, and rehearse the sentence you will use when you realise mid-solution that the approach is wrong.
Re-solve from blank the two problems you were slowest on this week and compare the times against the day they first appeared.

Deliverable: A recording of one fully narrated solution and a tally that separates syntax failures from approach failures.

'We rewrote it in a better framework' is not a story

A migration is a cost you chose to pay, not an achievement. The story is what the old system made expensive, what you measured before committing, what kept serving traffic during the cutover, and what you would have done if the numbers had come back flat. Without those, a rewrite reads as taste.

Reverse a webhook ordering decision after measuring its cost

Medium · reversing decisions · head-of-line blocking · at-least-once · api contracts

You argued for strict per-subscription ordering in webhook-delivery, which means one in-flight attempt per subscription. It shipped. Three months later a single unresponsive endpoint holds one subscription's queue at a six-hour backlog, and two customers report events arriving out of order anyway once their own retries are counted. Describe a decision you reversed: what you originally optimised for, the measurement that changed your mind, what the reversal cost in engineering time and customer change, and how you told the people who had already built on the original guarantee.

Approach

State the original decision as a trade you made knowingly. Ordering across a network requires a single in-flight attempt per subscription, and its price is head-of-line blocking whenever one endpoint is slow. 'We priced it wrong' is a much stronger opening than 'we did not realise', and it is usually the true one.
Bring the measurement that flipped it, not the anecdote: backlog age at the ninety-ninth percentile per subscription, the share of subscriptions where one slow endpoint gated an otherwise healthy queue, and the delivery throughput lost to serialisation. A reversal justified by complaints is indistinguishable from a reversal justified by fatigue.
Name what you learned about the guarantee itself, which is the engineering content of this story. At-least-once delivery means a retried event already arrives after newer ones and the consumer already must be idempotent, so a guarantee the customer has to defend against anyway was never worth what it cost to provide.
Describe the migration, because reversing a published contract is the hard half and the part candidates skip. Parallel attempts behind a per-subscription flag, a monotonically increasing sequence number added to the envelope so order-sensitive consumers can sort or discard, documentation that states at-least-once and unordered in those words, and a deprecation measured in quarters because the client is a pinned SDK inside a build pipeline you cannot see or redeploy.
Give the cost in the two currencies that matter: engineer-weeks, and how many customers had to change code. Then say who you told before it shipped rather than in a changelog afterwards, and which large customer you left on the old behaviour and for how long.
Close with the signal you now weight differently, stated as something you would do earlier next time: measuring the blocking cost on the slowest decile of endpoints before committing to the guarantee, rather than after a customer noticed.

Follow-up

A customer insists they need ordering. What do you offer them that is not global serialisation?
How did you choose the deprecation window given that you cannot see or redeploy the clients?
What would have to be true for you to reverse back?

Disclose a cross-tenant webhook delivery to affected customers

Medium · cross-tenant leak · disclosure · blast radius · authorisation checks

An enqueue path took the subscription from one lookup and the payload from another. For nineteen minutes, webhook_delivery rows were created whose tenant_id did not match the subscription's tenant, and eleven payloads were signed and sent to four endpoints belonging to other customers. You hold payload_digest, delivery timestamps and response codes. Describe how you handle a disclosure of this kind: what the records prove, what they cannot prove, what you say before you know everything, the one code change that closes it, and which parts you personally drove.

Approach

Bound the population before saying anything externally. The affected set is deliveries in the window where the event's tenant and the subscription's tenant differ; the ones that actually left are those with delivered_at set and a 2xx in last_response_code. Attempted and delivered are two different counts and a disclosure has to use the right one in the right sentence.
Separate what the records prove from what they do not, and say both halves rather than the flattering one. They prove which payloads were signed, where they went, and — through payload_digest — exactly which bytes. They do not prove what the receiving system did with them, and they do not bound the window more precisely than your deploy timestamps do.
Communicate on the facts you hold, with the scope stated as an upper bound: 'at most eleven payloads, four recipient endpoints, these fields, this window' is more useful and more honest than waiting a day for certainty. The field list matters more than the event count, because a customer cannot assess exposure from 'an event'.
Name the code change precisely, because this class never originates in the delivery worker. Compare the event's tenant against the subscription's tenant at enqueue and again immediately before the payload is signed, and make the second comparison drop the delivery rather than log a warning. Say why one check is insufficient: the enqueue check protects against the bug you know about, the pre-signing check protects the boundary itself.
Run the history question in parallel and say so: a query over historical deliveries for the same mismatch tells you whether this was nineteen minutes or a year, and you would rather find the second case yourself than have a customer find it after your disclosure.
Split the response into workstreams with owners — recipients asked to delete, affected customers notified, the check landed with a test, history swept — and say which you personally drove and which you handed off. Claiming all four is not credible and claiming none is not ownership.

Follow-up

The historical sweep finds two more instances from last year. What changes in what you have already told people?
Who approves the wording, and what do you do when you are asked to soften the scope?
A customer asks you to prove a redelivery contained the same bytes as the original. What do you show them?

Unblock an engineer on a job run that finished twice

Easy · mentoring · fencing tokens · lease expiry · debugging method

An engineer two weeks into the team brings you a job_run row showing status succeeded with an exit_code written by a worker declared dead ten minutes earlier; the retry attempt also shows succeeded. They have spent a day adding logging and are no closer. You have twenty minutes and you do not want to take the keyboard. Describe how you unblock someone: the question you ask first, what you let them find themselves, the concept you name and when, and how you check the next day that they own the fix rather than having watched you produce it.

Approach

Ask what they expect rather than what they see: which statement set status to succeeded, and what did it check before writing? That question points directly at the update's WHERE clause, which is where the answer lives, and it costs them nothing to answer, so it does not read as a test.
Let them build the timeline themselves from the row: queued_at, started_at, leased_until, finished_at and worker_id, on both the original run and the retry. Two different worker_ids with a lease expiry between them tells the whole story, and they will see it before you say it.
Name the concept once the evidence has earned it. A lease bounds time; it does not prevent a write. The store has to reject a stale writer, which means the update carries a fencing token the row compares — update job_run set status = 'succeeded' where run_id = $1 and lease_token = $2 and status = 'running' — and a long garbage-collection pause or a brief partition is enough to produce what they are looking at.
Point at the second, less obvious half and let them decide it: 'lost' exists in the status enum precisely so a run whose worker vanished is not recorded as failed, because failed asserts an outcome nobody observed and the system then bills and retries on that assertion. Ask them what these two rows should have said.
Leave them with the next step rather than the patch — a test that kills the first worker after the sandbox exits and before the row is written — and say when you are available again, so the offer is real rather than polite.
Check ownership the next day by what they produced, not by asking if it went well: a test that reproduces the window proves they understood it; a test that only asserts the new WHERE clause proves they copied it. Ask them to explain it to a third person and listen for whether the explanation is theirs.

Follow-up

They propose a longer lease instead of a token. What do you say, and what breaks when legitimate runs last thirty minutes?
How can you tell whether your explanation landed or they simply deferred to you?
The same engineer hits a variant of this next month. What did you fail to teach the first time?