Interview guides · Broadcom

Broadcom Software Engineer Interview Questions & Guide 2026

Take-home style practice for Broadcom software engineer candidates: a small service, a fixed budget, a written tradeoff.

The 60-second brief

As a Software Engineer at Broadcom, you play a pivotal role in designing, implementing, and optimizing the critical software layers that power modern global enterprise infrastructure. Broadcom operates at the intersection of complex hardware silicon and high-performance software systems. Engineers here develop everything from low-level firmware, microcode, and device drivers to enterprise-grade virtualization platforms, network management tools, and cloud-native backend applications.

Treat capacity estimation as a conversion skill rather than a table to memorise: turn a user count and an action rate into requests per second and bytes per day, then name the component that number breaks first. The figure only matters if it changes the design.

Focus areas

Evolve APIs without breaking pinned SDK clients
Build at-least-once pipelines with explicit deduplication horizons
Scope every query and cache key by tenant

Overview

The impact of this role reaches across data centers, telecommunication networks, cloud service providers, and global enterprises. Whether you are optimizing packet processing speeds for next-generation Ethernet switches, developing driver interfaces for high-speed storage controllers, or engineering microservices for massive infrastructure management, your code directly influences performance, throughput, and system stability at scale.

Working as a Software Engineer at Broadcom demands technical rigor, strong software design fundamentals, and a deep understanding of how code interacts with underlying systems and hardware. It is an environment built for engineers who thrive on solving challenging low-level or high-throughput computing problems, conducting thorough debugging, and writing highly efficient, resilient code.

The Broadcom interview process

Broadcom candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

The title covers product work, platform work, infrastructure, mobile and frontend, and those are different jobs with different loops behind them. A screening call is the cheapest place to find out which one the seat is, and asking reads as experienced rather than fussy. The questions that separate them: what the team is on call for, what the last three projects were, and whether any round happens inside an existing repository instead of a blank file. Then say which of that you have done and which you have not. Claiming the whole posting is the fastest way to be found out one round later.

What to demonstrate

Whether you can locate your experience inside one flavour of the role honestly instead of claiming the entire requirements list
Whether you name what you have not done, which an experienced screener reads as a level signal and can plan the loop around
Whether what you want next matches what the seat is: someone who wants greenfield work landing on a team that mostly operates an existing system is a hire that leaves within the year

How to prepare

Mark every line of the posting as done, adjacent or new, and write one sentence for each adjacent line naming the closest thing you actually built
Split your last two years into rough percentages across feature work, operating and debugging live systems, and design or review, so a question about scope gets numbers rather than adjectives
Bring three questions that discriminate between seats: what the team is paged for, how much of the work is changing existing code versus standing up something new, and what shipped in the last quarter

Technical Interviews

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Behavioral Interviews

This round is deciding whether a change you make without supervision can be allowed to reach production. It is scored on what you knew at the moment you decided, not on how it turned out, so a story that opens with the result and works backwards reads as luck retold as judgement. Say what the options were, what you did not know, what you did to shrink the unknown before committing, and what you accepted as the worst plausible case. The detail that separates answers is a bound: how many users, how much data, and for how long, if you had been wrong.

What to demonstrate

Whether the reasoning you give was available at the time you decided rather than after the result came in, since a story whose deciding evidence arrived later describes an outcome and not a judgement
Whether you can put units on the exposure (users, rows, minutes of degraded service) and whether the containment you chose actually bounded it: a canary bounds the request path it fronts, while a background job writing to a shared table reaches every user regardless of which version served their requests
Whether the reversal path existed before you shipped or was improvised during the incident, and whether it restores state or only stops further damage

How to prepare

For your three largest changes, write down the one thing you would have had to be wrong about for it to fail, and what your best estimate of it was on the day you shipped. If you never held an estimate, that is the gap the follow-up questions will find
Write the undo procedure for one of those changes as it existed at the time, then mark which steps restore data and which only stop new damage. Turning a flag off or reverting a deploy ends the new writes; rows already written come back only from a copy you kept, and a dropped column comes back empty unless something outside the schema holds the values
Rehearse one story from the decision point forward and stop before the outcome, then have someone ask what you would do next. If the story only works with the ending attached, it is an anecdote rather than a decision you can defend

Where candidates lose points

Paginating a growing table with limit and offset

Two unrelated defects share the idiom. Correctness: rows inserted or deleted between page requests shift the window, so a consumer walking an export skips rows and sees others twice, which for a customer-facing sync is silent data loss rather than an error anyone notices. Cost: the database still produces and discards the skipped rows, so page N costs time proportional to N times the page size and a deep page on a large table degrades from milliseconds to seconds. Keyset pagination over a stable, unique, indexed ordering -- where (created_at, id) < ($1, $2) order by created_at desc, id desc limit $3 -- is constant-cost per page and immune to shifting, on the precondition that the cursor columns never change value for a row, which disqualifies updated_at as a cursor.

Treating a timed-out write as a failed write

A timeout says the response did not arrive, not that the work did not happen; the server may well have committed and then lost the connection. Retrying a non-idempotent create after a timeout is the standard way to end up with two of something, and those duplicates land precisely when the system is already degraded and least able to absorb them. The discipline is to treat a timeout as unknown: either the write carries an idempotency key so the retry is safe by construction, or the client re-reads authoritative state before deciding what to do, and the interface says unknown rather than showing a failure that invites a second click.

Check-then-act on shared state

Read, decide, write is not safe under concurrency unless the decision and the write are one atomic step: a unique constraint with conflict handling, a compare-and-set, or a row lock held for the whole transaction. Two requests can both pass the existence check before either inserts, which shows up as duplicate rows under load and never in a single-threaded test.

Finishing a solution without stating its complexity

Give time and space in the same breath as the code, and define n explicitly when there are two sizes, since n nodes and m edges are not interchangeable. Space is the half that gets skipped: count the auxiliary structures you allocate and the recursion stack at its deepest, not only the answer you hand back.

Questions & practice

Solve a dynamic programming challenge to find the optimal path or value combination given constraints.

Medium · data structures and algorithms

Approach

Walk one small example through your approach before writing the whole thing.
Name the brute-force solution and its complexity before improving on it.
State the target complexity and say which constraint rules the naive version out.

Follow-up

Which test case would catch an off-by-one here?
What is the worst case, and how likely is it on real data?

Implement a custom queue using primitive arrays or linked nodes and discuss thread safety.

Medium · data structures and algorithms

Approach

Walk one small example through your approach before writing the whole thing.
Choose the data structure from the access pattern, not from familiarity.
Restate the input: its shape, its size, and what is guaranteed about it.

Follow-up

How does this change if the input no longer fits in memory?
Which test case would catch an off-by-one here?

Given an integer array, find the smallest contiguous subarray that matches a target sum condition.

Medium · data structures and algorithms

Approach

Walk one small example through your approach before writing the whole thing.
State the target complexity and say which constraint rules the naive version out.
Choose the data structure from the access pattern, not from familiarity.

Follow-up

How does this change if the input no longer fits in memory?
Which test case would catch an off-by-one here?

Write a function to reverse a single linked list and find its middle node in a single traversal.

Medium · data structures and algorithms

Approach

Name the brute-force solution and its complexity before improving on it.
Restate the input: its shape, its size, and what is guaranteed about it.
Choose the data structure from the access pattern, not from familiarity.

Follow-up

Which test case would catch an off-by-one here?
How does this change if the input no longer fits in memory?

What is Clock Domain Crossing (CDC), and how do you design an asynchronous FIFO to safely handle cross-domain data?

Medium · languages, concurrency and fundamentals

Approach

Identify the window where an invariant is briefly untrue.
Distinguish a value from a reference to it, and say which one you handed out.
Name what is shared across threads and what owns each piece of state.

Follow-up

Where could this allocate more than you expect?
How would you prove the race exists rather than suspect it?

Describe the classic producer-consumer problem and implement a synchronization mechanism using mutexes and condition variables.

Medium · languages, concurrency and fundamentals

Approach

Say what the runtime actually does before reasoning about the code.
Distinguish a value from a reference to it, and say which one you handed out.
Name what is shared across threads and what owns each piece of state.

Follow-up

Where could this allocate more than you expect?
What happens if two callers reach this at the same time?

Locate a billing reconciliation gap without rescanning ninety million events

Hard · reconciliation · dimensional-bisection · watermarks · hypothesis-testing

A tenant's sealed invoice total is 0.4% below the sum of its raw usage_event rows for the period. That tenant has 90 million events over 30 days in a table partitioned daily on ingested_at, and its rollups carry source_max_ingested_at, revision and sealed_at. Recomputing all 30 days from raw is correct, and you are not going to do it. Give the procedure that locates the divergent (workspace, sku, hour) cell, the cost of each probe, and the one query you run before any of it.

Approach

Run the free query first. Sum raw quantity for the period restricted to ingested_at <= source_max_ingested_at of the sealed rollups, and compare that against the unrestricted sum. The rollup stores the watermark precisely so this can be answered without a scan. If the whole 0.4% sits above the watermark, nothing is broken: it is late data, it becomes an adjustment line, and the investigation ends in one query.
Only if the gap survives that test do you bisect, and you bisect by dimension rather than by rows. Compare 30 per-day totals, then inside the offending day compare the 6 SKUs, then the workspaces, then the 24 hours. That is roughly 30 + 6 + W + 24 grouped probes, each an indexed range scan over one daily partition for one tenant, against O(N) per attempt for the naive re-fold.
Quantify why naive is not merely slow but unusable mid-incident: at a generous 200,000 rows/second sequential, 90 million rows is about 7.5 minutes per attempt, you will want ten attempts, and every one competes for I/O on the same partitions live ingest is writing. The diagnostic worsens the backlog it is diagnosing.
Before fetching each comparison, state what it would look like under each hypothesis. Two adjacent hours off by equal and opposite amounts is occurred_at versus ingested_at bucketing. A whole day offset by exactly N hours is a timezone applied at the wrong layer. A gap confined to one SKU in one workspace is an environment filter. The same (tenant_id, idempotency_key) present in two ingested_day partitions is the dedup horizon losing a retry that crossed midnight.
Make the next bisection cheap by storing the aggregate you keep recomputing. A per-(tenant_id, ingested_day) count and quantity checksum turns step two from thirty probes into one read, and it is the same number the reconciliation job already produces.
Whatever you find, the sealed period does not change value. The correction is an adjustment line pointing at the line it reverses, carrying its own source_rollup_watermark, because the original invoice is the evidence of what the customer was charged.

Worked solution (35 min)

Reproduce the shape locally: generate 2 million events for one synthetic tenant over 5 days, fold them, then inject three defects, namely 0.2% of events bucketed by ingested_at, a handful of duplicate idempotency_key values whose retries cross midnight, and a block of events ingested after the seal.
Write the watermark-bounded query first and record how much of the gap it explains on its own.
Bisect by day, then SKU, then hour, recording the probe count and the rows each probe touches.
For each located cell, write down the predicted signature before querying it, then check whether the data matches the prediction.
Time a full re-fold of the 2 million rows and extrapolate to 90 million.

Expected result: The watermark-bounded query accounts for the post-seal block entirely and removes it from the investigation. The ingested_at bucketing appears as adjacent hours off by equal and opposite amounts. The midnight-crossing duplicates appear as one (tenant_id, idempotency_key) present in two ingested_day partitions. Bisection reaches each cell in fewer than 70 probes.

Verify it yourself

The three injected defects sum to the original gap with no unexplained remainder.
Each probe's row count is bounded by one day-partition for one tenant, not by the period.
Removing the source_max_ingested_at bound from the first query makes a gap appear, and grow, while ingest runs, even after every defect is removed.

Follow-up

The gap is 0.4% in one direction on one day and 0.4% the other way the next day. What does that shape rule in, and what does it rule out?
How do you distinguish a duplicate from a restatement, given revision and recomputed_at on the rollup?
Ingest is still running while you investigate. What makes your two numbers comparable at all?

Migrate a live partitioned event table without blocking ingest

Hard · online migration · ddl locks · expand and contract · partitioning

usage_event is range-partitioned daily on ingested_at, holds roughly 250M rows per day across 400 live partitions, and is written at 10-40k rows/second. Two changes are required: quantity must move from double precision to numeric(20,6), and a new environment column must become NOT NULL with a default of 'production'. Ingest cannot stop. Give the ordered plan, naming for each step the lock it takes, what that lock blocks, and roughly how long it is held. Identify the one step that cannot be rolled back cleanly once traffic depends on it.

Approach

Classify the two changes before planning anything. Adding a column with a non-volatile default has been metadata-only since PostgreSQL 11, so it is cheap. Changing double precision to numeric is not binary-coercible, so alter column ... type rewrites every partition under ACCESS EXCLUSIVE and rebuilds its indexes; on this volume that is hours of blocked ingest and is simply not an option, which is why the plan is expand-and-contract rather than one statement.
Expand: add quantity_numeric numeric(20,6) and environment with its default on the parent. Both are catalogue-only but both take a brief ACCESS EXCLUSIVE that cascades to partitions, so run each with lock_timeout set to a second or two and retry on failure. A queued ACCESS EXCLUSIVE request blocks every reader behind it, which is how a metadata-only change turns into an outage.
Dual-write: deploy producer code that populates both columns on every insert, and leave it running before anything reads the new column. This is the step that cannot be reverted cleanly. Once readers depend on quantity_numeric, reverting the writer leaves rows with a null there, and the gap is only discoverable by re-reading the old column, which the readers have stopped doing.
Backfill older partitions in batches keyed on the primary key, oldest first, committing every few thousand rows with a pause between batches, and skipping the partition still receiving writes until it rotates. Each batch is an ordinary UPDATE taking row locks only. The cost is bloat and WAL rather than blocking, so watch dead tuples and let autovacuum keep pace instead of wrapping 400 partitions in one transaction.
Make NOT NULL cheap with the three-step form: add constraint ... check (environment is not null) not valid (brief ACCESS EXCLUSIVE, no scan), then validate constraint (SHARE UPDATE EXCLUSIVE, scans while reads and writes continue), then set not null, which from PostgreSQL 12 uses the validated check and skips its own full scan. Do this per partition, then on the parent.
Switch and contract: move reads to the new column behind a flag, verify over a full period that both columns agree on freshly written rows, drop the old column (metadata-only), and only then remove the dual-write. Any index on the new column goes on with CREATE INDEX CONCURRENTLY per partition, since CIC is not supported on a partitioned parent: create the parent index with ONLY, build each child concurrently, then ALTER INDEX ... ATTACH PARTITION until the parent index becomes valid.

Worked solution (45 min)

On a scratch cluster, build 10 partitions of 2M rows each and run a writer at a few thousand inserts/second.
Run the naive type change and measure how long writes stall and how far ingest lag grows before killing it.
Run the expand step with lock_timeout = '2s' while the writer runs, and observe a clean lock timeout and retry instead of a pile-up of blocked readers.
Backfill in 5k-row batches and chart dead tuples and WAL generated per batch.
Run the not-valid, validate, set-not-null sequence and confirm from pg_stat_activity and timings that nothing held an exclusive lock through a full scan.
Add an index with CIC per partition plus ATTACH PARTITION and confirm the parent index reports valid only after the last attach.

Expected result: An ordered expand, dual-write, backfill, switch, contract plan with the lock named per step: metadata-only ADD COLUMN under a brief ACCESS EXCLUSIVE taken with lock_timeout; batched UPDATE backfill under row locks; CHECK NOT VALID, then VALIDATE under SHARE UPDATE EXCLUSIVE, then SET NOT NULL; CREATE INDEX CONCURRENTLY per partition with ATTACH PARTITION; and the dual-write-to-read switch identified as the step that cannot be reverted cleanly.

Verify it yourself

No step in the plan holds ACCESS EXCLUSIVE for longer than the configured lock_timeout.
The writer's error rate stays at zero through the rehearsal apart from the deliberate lock timeout.
After VALIDATE, SET NOT NULL completes in milliseconds on a table where the naive form scanned every row.
Freshly written rows agree on both columns for a full verification period before the old column is dropped.

Follow-up

A CREATE INDEX CONCURRENTLY fails halfway through the partition list. What state is the table in, how do you detect it, and what do you run?
The producer computes quantity itself. What happens to a request already in flight when the dual-write deploy lands, and does it matter?
Give two queries that prove the backfill is complete: one cheap enough to run every minute, one authoritative.

Decide which facts an invoice line copies instead of joining

Medium · denormalisation · immutable records · foreign keys · rounding

invoice_line_item already denormalises tenant_id, which is reachable through invoice_id, and stores amount_minor even though quantity times unit_price_micros would recompute it. A reviewer asks you to normalise both away, and separately asks whether the tenant's legal name and billing address should be copied onto the invoice header. Decide each case. For every field you keep denormalised, name the read pattern or the invariant that justifies it, the anomaly the copy can develop, and the mechanism that prevents that anomaly here.

Approach

Split the question into two kinds of copy, because they fail differently. A copy of a currently mutable fact is a cache: it drifts and needs invalidation. A copy of a fact frozen at write time is not a cache at all, it is the record of what happened, and normalising it away destroys information the source no longer holds.
Keep tenant_id on the line. It costs 8 bytes, it leads every index on the table so no read is ever accidentally cross-tenant, and it turns a wrong join into an empty result rather than another tenant's money. Prevent the drift structurally: a unique constraint on invoice (invoice_id, tenant_id) plus a composite foreign key from the line on (invoice_id, tenant_id) makes a mismatched pair impossible, so the database enforces agreement instead of a code review.
Keep amount_minor. Rounding must happen exactly once, at a named site, with a stated mode (half-even here). If readers recompute from quantity and unit_price_micros, every reader owns a rounding decision, and half-up and half-even diverge systematically across thousands of lines rather than cancelling out. A check constraint can bound the stored value but deliberately cannot re-derive it.
Copy the legal name and billing address onto the invoice header, written once and never updated. The statement must show what was true when it was sealed, and the tenant record will change afterwards. This is a snapshot for the same reason source_rollup_watermark is stored per line: without it, nobody can reconstruct what the customer was told.
Name the read pattern that pays for all of it. Rendering, dispute response and export are per-tenant, per-period reads over thousands of lines that would otherwise join back to slowly changing dimensions that no longer hold the historical value. The write side is a once-per-period batch, so the extra columns cost nothing that matters.
Concede the case where the reviewer is right: a mutable operational attribute such as the tenant's current plan name has no business on a line. If a report wants it, join. If a statement needs the plan as of the period, that is another snapshot and it belongs on the header with the rest.

Follow-up

Write the composite foreign key and the unique constraint it requires on the parent. What does it cost on every line insert, and what does it do to a bulk load?
A tenant is renamed after being invoiced. Which rows change, and what does the customer see on last quarter's PDF?
Where does currency live, and what breaks if a tenant's billing currency changes between two periods?

Explain how you would architect a resilient distributed storage cluster to ensure zero data loss during node failure.

Medium · system design and architecture

Approach

Name the failure you are designing for, then the recovery path.
Choose a partition key and say what query it makes expensive.
Fix the scope first: who calls this, how often, and what they do when it fails.

Follow-up

What breaks first when traffic grows ten times?
How does this behave when that dependency is down for an hour?

Walk through the high-level and low-level design for a multi-threaded task scheduler.

Medium · system design and architecture

Approach

Name the failure you are designing for, then the recovery path.
State the consistency you need, and where you are willing to be stale.
Choose a partition key and say what query it makes expensive.

Follow-up

How does this behave when that dependency is down for an hour?
What would you drop to keep the system up under load?

What are the differences between shared and local comparators in Built-In Self-Test (BIST) architecture?

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer contains.
Work from the requirement backwards to the design.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Metering ingest that survives a six-hour producer replay

Hard · idempotency · dedup horizon · partitioning · watermarks

metering-ingest consumes usage events at-least-once - 250M/day, 10-40k/second at peak - and folds them into usage_rollup_hourly keyed (tenant_id, workspace_id, sku, hour_start). usage_event is partitioned daily on ingested_at with unique (ingested_day, tenant_id, idempotency_key). A producer outage ends in a six-hour replay that re-sends events already ingested, some of whose originals crossed midnight. Design the consumer: partitioning, where the acknowledgement sits relative to the commit, the deduplication horizon and its storage cost, and how the rollup watermark advances. Nothing may be double-counted and nothing may be silently dropped.

Approach

Choose the acknowledgement position deliberately and name what each choice costs. Acknowledging after the fold commits makes the consumer at-least-once: a crash between the two replays the batch and produces duplicates, which are ordinary and absorbable. Acknowledging first makes it at-most-once: a crash between the two drops revenue with no error raised anywhere and no way to detect it later. Take at-least-once and design everything downstream to absorb duplicates.
Put the dedup and the fold in one transaction so there is no window between them. Insert the batch into usage_event with ON CONFLICT DO NOTHING, take the rows actually inserted, and fold only those into usage_rollup_hourly with an upsert on (tenant_id, workspace_id, sku, hour_start) bucketed by occurred_at, not ingested_at. A batch of about 2,000 rows is one round trip and one index probe per event.
Attack the partition-key flaw head on: the unique index includes ingested_day because a unique index on a partitioned table must contain the partition key, so the same (tenant_id, idempotency_key) re-sent after midnight is a different index entry and passes. Deduplicate instead against a store keyed (tenant_id, idempotency_key) with no date component, whose horizon exceeds the producer's maximum retry window plus the longest replay you intend to support. At 14 days that is 250M x 14 = 3.5 billion keys, which is a dedicated key-value store, not a larger index on the same table. The alternative - partitioning usage_event on (tenant_id, occurred_day) so the natural key is stable - fixes dedup but loses pruning on ingest time and makes retention by dropping partitions awkward.
Partition the consumer by hash of tenant_id so one tenant's replay stalls only its own partitions, and give replay traffic a separate lower-priority lane so live ingest keeps its latency. The cost is explicit: that tenant's watermark lags while the replay drains, and everything gated on the watermark waits for it.
Define the watermark as a property of committed work, not of wall-clock time: per partition it is the largest occurred_at such that every event with a smaller occurred_at has committed, and the sealing decision uses the minimum across partitions. Record source_max_ingested_at on every rollup row so any number can prove what it did and did not include, and keep restatement legal only while status = 'open' - after sealed_at the value is frozen and a late event becomes an invoice adjustment instead.

Worked solution (40 min)

Write the consumer loop in pseudocode with the acknowledgement after the commit, then annotate each line with what is lost or duplicated if the process dies exactly there.
Size the dedup store: events/day x horizon_days keys, bytes per key including the tenant prefix, and the resulting memory or disk. Compare that cost against simply extending retention on the partitioned table and say why the latter does not fix the problem.
Take one event ingested at 23:59:58 and replayed at 00:00:04 and work out its fate under (a) the partitioned unique index alone and (b) the separate dedup store.
Write the per-partition watermark formula, then what the seal uses, then what a single stalled partition does to sealing.

Expected result: An at-least-once consumer acknowledging after the commit; deduplication on (tenant_id, idempotency_key) in a store whose horizon of at least 14 days exceeds max retry window plus supported replay length, sized at roughly 3.5 billion keys; tenant-hashed partitions with replay on a separate lane; a per-partition watermark with sealing gated on the minimum across partitions; source_max_ingested_at written on every rollup row.

Verify it yourself

The midnight-crossing retry must be rejected by whatever you chose. If the table's unique index is your only dedup, it is accepted and the invoice is wrong by exactly that event.
You should be able to name a distinct failure for acknowledge-before-commit and for acknowledge-after-commit. If both sound the same, the position was not actually chosen.
quantity_sum for any (tenant, workspace, sku, hour) must be reproducible by re-aggregating usage_event filtered to ingested_at <= source_max_ingested_at. If it is not reproducible, the rollup is an opinion rather than a projection.

Follow-up

The dedup store is lost entirely. What can you still guarantee, and how do you rebuild it from what remains?
A replay delivers events for an hour that is already sealed. Trace exactly what happens to them, row by row.
One partition is stuck on a poison message, so the minimum-across-partitions watermark never advances and no tenant can be sealed. What is your escape hatch and what does it cost in correctness?

Hourly rollups merge one hour and lose another

Easy · timezone · rollups · reconciliation

Reconciliation flags one tenant on one day. Summing usage_event.quantity by hour of occurred_at gives 24 non-empty hours, but usage_rollup_hourly holds 23 rows for that tenant, workspace and SKU, one of which carries roughly the sum of two adjacent hours. Other days reconcile exactly, and the affected date matches a civil-time transition. hour_start is documented as truncated to the hour in UTC. You have both tables, the rollup job source, and its runtime environment. Give an ordered checklist, the mechanism, and the correction path for a day that may already be sealed.

Approach

Bisect by dimension until one cell explains the whole difference: tenant, then day, then SKU, then hour. A defect confined to a single transition date already rules out deduplication and late arrival, both of which are indifferent to which hour an event lands in.
Read the truncation with its precondition stated: date_trunc on a timestamptz value is evaluated in the session TimeZone, not in UTC. If the job connects without pinning that setting, it inherits the server or container default.
Follow that to the collision: in a zone that observes daylight saving, two distinct UTC hours map to the same local wall-clock label at the autumn transition, so both fold into one key under the unique constraint on (tenant_id, workspace_id, sku, hour_start) and their quantities sum into one row. At the spring transition a label never occurs and the row is simply absent.
Confirm from data rather than from reading code: run the same aggregate twice, once with the session pinned to UTC and once with the job host zone, and check that the second reproduces the stored rollup exactly.
Fix at the source by pinning the connection to UTC explicitly, or by truncating on occurred_at AT TIME ZONE 'UTC', rather than relying on a default that differs between a developer machine, CI and production.
Correct according to status, not convenience: an open hour is recomputed with revision incremented, a sealed hour is frozen and the difference becomes an adjustment line on the next invoice with voided_by_line_id pointing at the line it reverses.

Follow-up

The same job also emits a daily figure for a dashboard. Why can a correct hourly rollup still produce a wrong day, and what does the tenant's billing timezone have to do with it?
How would you detect this class automatically rather than waiting for reconciliation, given that it only manifests twice a year per zone?

Suggested practice checklist

Roughly ninety minutes on weeknights with one longer weekend block. The plan cuts scope rather than compressing everything, on the assumption that one thing finished per night beats four half-started.

Day 1 — Fix the scope and take a cold baseline

Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list and keep it visible all week.
Take one twenty-five-minute coding problem and one fifteen-minute design prompt cold, and write the single sentence naming what blocked each, because those two sentences decide where the remaining evenings go.
Set the week's rule: one thing finished every night, including the night you only have forty minutes.

Deliverable: A one-page scope with a not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One pattern, written three times from blank

Choose the single pattern most likely to appear in your loop and write it three times from an empty file rather than editing the previous attempt.
On the third pass, write the invariant as a comment before the loop body and the complexity before the first line of code.
Stop at ninety minutes even if the third version is imperfect, and write the one thing you would fix given another hour.

Deliverable: Three independent implementations of the same pattern plus a note on what changed between them.

Day 3 — One design, only to the depth you can defend

Take one system shape and go only as far as requirements, interface and data model, refusing to draw a box you could not survive a follow-up about.
Attach one number to each non-functional requirement, deriving it rather than asserting it, and write the assumption the number rests on.
Write the one tradeoff you are choosing against and the observation that would make you reverse it.

Deliverable: One design at interface-and-schema depth with derived numbers and one written reversible tradeoff.

Day 4 — Only the fundamentals you will have to defend

Write, in under two hundred words each, the answers to the two questions that follow almost any implementation: why this structure and not the obvious alternative, and what happens to this code at a hundred times the input.
Write what an index actually costs: faster lookups on the indexed columns against a write that now maintains a second structure, plus the cases where the planner declines to use it anyway, low selectivity, or a predicate wrapping the column in a function.
Delete any answer you cannot deliver aloud in under a minute, since an answer that needs reading is not an answer you have.

Deliverable: Three written answers, each under two hundred words and each timed aloud.

Day 5 — Your own work, timed

Write a ninety-second and a four-minute version of your main project and time both aloud rather than reading them.
Prepare the two follow-ups that always come: what you would do differently, and how you knew it worked.
Put one number in the first sentence and be ready to say exactly where it came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in the weekend block

Run a sixty-minute mock covering a coding round and a design round in one sitting with no break, because sustained attention is the thing evenings have not trained.
Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

Write the twenty-minute warm-up you will actually do on the morning: one problem you can already solve from a blank file, one design you can narrate, and nothing you have never seen.
Re-read only your own notes from this week and open no new material.
Write the logistics down: the editor or shared document you will be working in, whether execution and lookups are permitted, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the design structure, the project numbers, and the logistics.

Review and mentoring: what you let through

Counting review comments or mentees proves nothing. The useful version is a specific change you approved with a reservation you stated, or one you blocked and the delay that cost. Say which standard you were holding and why it was worth the friction. A mentoring story needs the thing the other person can now do without you.

Resolve a review disagreement over a quota check

Easy · code review · isolation levels · write skew · disagreement

A colleague's pull request enforces a per-tenant quota by selecting the current count and then inserting when it is under the limit. You flag it as a race. They reply that the transaction already runs at repeatable read, so the snapshot makes it safe, and the tests pass. Walk through taking that disagreement to a resolution: what you write in the review, what you demonstrate rather than assert, which fix you propose and why, and what you do if they still disagree after all of it.

Approach

Answer the claim precisely instead of restating your objection, because they have made a specific technical argument. In PostgreSQL, repeatable read is snapshot isolation; this is write skew, which snapshot isolation permits by design. Both transactions read a count that is stable within their own snapshot, insert disjoint rows that the other cannot see, and both commit, so the limit is exceeded by exactly the concurrency.
Demonstrate rather than cite. Two psql sessions, both BEGIN ISOLATION LEVEL REPEATABLE READ, both select the count, both insert, both commit: it succeeds. Repeat at SERIALIZABLE and the second commit fails with serialization_failure, SQLSTATE 40001. That takes two minutes, ends the argument without anyone conceding a position, and leaves an artefact for the next reviewer.
Offer the options with their costs rather than a verdict. Serialisable plus a retry loop on 40001 is correct but obliges every caller to retry and degrades under contention. An increment-and-compare on a counter row — update tenant_quota set used = used + 1 where tenant_id = $1 and used < limit returning used — is safe even at read committed, because a blocked updater re-evaluates the WHERE clause against the row version it finally locks, and zero rows returned means full. A unique or exclusion constraint that makes the surplus write fail is the third.
Name the plausible non-fix explicitly, since it is what usually gets merged instead: folding the count into the insert as insert ... select ... where (select count(*) ...) < limit is still racy under read committed, because the subquery cannot see the other transaction's uncommitted rows. It looks atomic and is not.
Say what you do if they still disagree: escalate the decision rather than the disagreement. Attach the reproduction, hand it to the service owner or a third reviewer, and state that you will not block the merge if the owner accepts the risk knowingly — and that you want that acceptance written down.
Close with the general lesson worth leaving in the review thread: a passing suite is weak evidence for a concurrency claim because it runs one request at a time. Ask for a test that runs two.

Follow-up

Write the counter-row version. Does your answer change if the quota counts child rows rather than a column?
Under serialisable, who performs the retry, and what does the API client see if the retry also fails?
This is the third disagreement with the same reviewer this month. What changes in how you review?

Unblock an engineer on a job run that finished twice

Easy · mentoring · fencing tokens · lease expiry · debugging method

An engineer two weeks into the team brings you a job_run row showing status succeeded with an exit_code written by a worker declared dead ten minutes earlier; the retry attempt also shows succeeded. They have spent a day adding logging and are no closer. You have twenty minutes and you do not want to take the keyboard. Describe how you unblock someone: the question you ask first, what you let them find themselves, the concept you name and when, and how you check the next day that they own the fix rather than having watched you produce it.

Approach

Ask what they expect rather than what they see: which statement set status to succeeded, and what did it check before writing? That question points directly at the update's WHERE clause, which is where the answer lives, and it costs them nothing to answer, so it does not read as a test.
Let them build the timeline themselves from the row: queued_at, started_at, leased_until, finished_at and worker_id, on both the original run and the retry. Two different worker_ids with a lease expiry between them tells the whole story, and they will see it before you say it.
Name the concept once the evidence has earned it. A lease bounds time; it does not prevent a write. The store has to reject a stale writer, which means the update carries a fencing token the row compares — update job_run set status = 'succeeded' where run_id = $1 and lease_token = $2 and status = 'running' — and a long garbage-collection pause or a brief partition is enough to produce what they are looking at.
Point at the second, less obvious half and let them decide it: 'lost' exists in the status enum precisely so a run whose worker vanished is not recorded as failed, because failed asserts an outcome nobody observed and the system then bills and retries on that assertion. Ask them what these two rows should have said.
Leave them with the next step rather than the patch — a test that kills the first worker after the sandbox exits and before the row is written — and say when you are available again, so the offer is real rather than polite.
Check ownership the next day by what they produced, not by asking if it went well: a test that reproduces the window proves they understood it; a test that only asserts the new WHERE clause proves they copied it. Ask them to explain it to a third person and listen for whether the explanation is theirs.

Follow-up

They propose a longer lease instead of a token. What do you say, and what breaks when legitimate runs last thirty minutes?
How can you tell whether your explanation landed or they simply deferred to you?
The same engineer hits a variant of this next month. What did you fail to teach the first time?

Argue against failing open when the control plane is unreachable

Hard · revocation · cache staleness · fail-open vs fail-closed · influence

The gateway caches credential-to-context decisions with a sixty-second TTL. A design proposal says that when control-plane reads fail, pods should keep serving from expired entries indefinitely so a control-plane outage never becomes a product outage. You believe that converts every revocation into an unbounded one. Describe a design you argued against while it was still a live proposal: what you measured or modelled to make the case, what you conceded, who decided, and what happened afterwards. Say what would have changed your mind before the decision, not after it.

Approach

Reframe it from a values argument into a bounded-staleness argument. Both sides already accept the cache; the disagreement is only about the ceiling on how long a revoked credential keeps authorising. Put a number on the table — serve stale for up to fifteen minutes, then fail closed — and make the other side argue against a number rather than against a principle.
Bring arithmetic rather than adjectives: the rate of revocations with revoked_reason in ('suspected_leak','auth_version_bump'), the observed distribution of control-plane unavailability, and the product of the two, which is expected requests served by revoked credentials per outage-hour. At 30k requests/second the unbounded version is not a subtle exposure and the number says so.
Concede the strong half of the opposing case first, because that is what buys you the room: failing closed turns one service's outage into a total outage across three regions, and a control plane doing tens of writes per second is not engineered to the gateway's availability target. A proposal you have not steelmanned reads as reflex.
Propose the asymmetry that usually resolves this: stale entitlements cost bounded money (a quota fifteen minutes out of date over-serves by a computable amount), while a stale revocation costs unbounded access. Split the cached decision by what it authorises, give the two halves different staleness ceilings, and let the entitlement half fail open while the revocation half fails closed.
State the propagation dependency plainly, since it is the part that is missed: validity is also derived from the principal's auth_version, so password reset and sign-out-everywhere flow through this same cache. A design that bounds staleness for explicit revocation and not for auth_version bumps has only fixed half of it.
Say who decided, and what you did afterwards in either outcome: write the decision down with its number and a review date, and instrument the exposure you were worried about so the next round of the argument is settled by data instead of by seniority.

Follow-up

Publish-subscribe invalidation is lossy under a partition, and a TTL is the only hard bound. What TTL do you pick, and what does it cost you at 30k requests/second?
The key was revoked because it was found in a public repository. Does your answer change, and where does that urgency live in the design?
You lost the argument and six weeks later the failure you predicted happens. What do you say in the review, and what do you not say?