Interview guides · Backblaze
Backblaze Software Engineer Interview Questions & Guide 2026
Testing practice for Backblaze software engineer candidates: the boundaries worth covering, and why flakiness is a real defect.
The 60-second brief
As a Software Engineer at Backblaze, you are joining a company defined by its commitment to reliable, scalable, and cost-effective cloud storage. Your work directly impacts how millions of users and businesses protect their most critical data. You aren't just writing code; you are contributing to a infrastructure-heavy environment where performance, efficiency, and data integrity are paramount.
The loop does not sample the job evenly, and arguing about that in the room costs you. Daily work is mostly incremental change inside code someone else wrote, while the loop samples narrow slices of it; prepare for the slices and save the realism argument for your questions at the end.
Focus areas
- Bound blast radius with per-tenant concurrency limits
- Keep money in integer minor units
- Evolve APIs without breaking pinned SDK clients
Overview
The role involves navigating complex technical challenges, ranging from high-throughput storage systems to API development and SDK maintenance. You will work within teams that value practical, hands-on engineering. Whether you are optimizing low-level memory operations or building robust backend services, the work is characterized by its scale and the real-world utility of the Backblaze B2 Cloud Storage platform.
Successful engineers at Backblaze demonstrate a balance between deep technical curiosity and a pragmatic approach to problem-solving. Expect to work in an environment where your ability to communicate your thought process is as valued as your ability to ship clean, efficient code.
The Backblaze interview process
Backblaze candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Recruiter Screen
Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.
What to demonstrate
- Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
- Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
- Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not
How to prepare
- Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
- Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
- If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window
Technical Assessment
Most of the time lost in this format is not lost to thinking. It goes to a standard-library call you half-remember, an off-by-one in a loop bound, and a debugging loop that mutates code at random until something passes. When output is wrong, stop re-reading the whole function: take the smallest input that reproduces it and walk the state through by hand, printing intermediates if the environment allows. Guessing at a fix without a failing case you understand is how a five-minute bug becomes twenty, and the clock does not pause while you do it.
What to demonstrate
- Whether you reach the right structure without a detour, and can write it from memory rather than only recall that one exists
- Whether overflow is considered where the language has fixed-width integers, since a signed 32-bit value stops at 2,147,483,647 and then wraps in Java, is undefined behaviour in C++, and does not arise in Python, whose integers grow instead
- Whether recursion depth is treated as a constraint on large inputs, given that CPython's default limit is 1000 frames and a deep recursion can exhaust the stack in any language where an iterative version would not
- Whether a failing case is isolated and explained before any edit is made to the code
How to prepare
- From an empty file and with no references open, implement the pieces you lean on most: a heap push and pop, an iterative DFS with an explicit stack, and a binary search whose midpoint is written lo + (hi - lo) / 2, which avoids the overflow that (lo + hi) / 2 can hit in a fixed-width integer type
- Time yourself on the ten library calls you look up most, such as sorting with a custom comparator, splitting and joining strings, and finding the next key at or above a value in an ordered map, until the lookup is gone
- Take a solution you know is broken and, before touching it, write one sentence naming the input, the expected value and the actual value. Repeat until you do it without deciding to.
1:1 Technical Interviews
Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.
What to demonstrate
- Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
- Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
- Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
- Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply
How to prepare
- For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
- For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
- Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last
Behavioral Interviews
This round is deciding whether a change you make without supervision can be allowed to reach production. It is scored on what you knew at the moment you decided, not on how it turned out, so a story that opens with the result and works backwards reads as luck retold as judgement. Say what the options were, what you did not know, what you did to shrink the unknown before committing, and what you accepted as the worst plausible case. The detail that separates answers is a bound: how many users, how much data, and for how long, if you had been wrong.
What to demonstrate
- Whether the reasoning you give was available at the time you decided rather than after the result came in, since a story whose deciding evidence arrived later describes an outcome and not a judgement
- Whether you can put units on the exposure (users, rows, minutes of degraded service) and whether the containment you chose actually bounded it: a canary bounds the request path it fronts, while a background job writing to a shared table reaches every user regardless of which version served their requests
- Whether the reversal path existed before you shipped or was improvised during the incident, and whether it restores state or only stops further damage
How to prepare
- For your three largest changes, write down the one thing you would have had to be wrong about for it to fail, and what your best estimate of it was on the day you shipped. If you never held an estimate, that is the gap the follow-up questions will find
- Write the undo procedure for one of those changes as it existed at the time, then mark which steps restore data and which only stop new damage. Turning a flag off or reverting a deploy ends the new writes; rows already written come back only from a copy you kept, and a dropped column comes back empty unless something outside the schema holds the values
- Rehearse one story from the decision point forward and stop before the outcome, then have someone ask what you would do next. If the story only works with the ending attached, it is an anecdote rather than a decision you can defend
Where candidates lose points
Serialising a tenant's writes through select ... for update on a single counter row
It is the first change that makes a counter correct, and it caps that tenant's write throughput at roughly one divided by the lock hold time. A transaction that takes the lock, makes a network call and then commits holds it for the entire round trip: at 2 ms that is about 500 writes per second for the whole tenant, and the largest tenants are exactly the ones that exceed it. The damage then spreads, because every waiter holds a database connection while it queues, so one hot tenant drains the shared pool and the symptom presents as a site-wide latency incident rather than as a lock problem. The repairs are to shrink the critical section to a single statement, to shard the counter into per-(tenant, hour) or per-(tenant, bucket) rows and sum on read, or to batch in memory and flush periodically while accepting the bounded loss that batching implies.
Paginating a growing table with limit and offset
Two unrelated defects share the idiom. Correctness: rows inserted or deleted between page requests shift the window, so a consumer walking an export skips rows and sees others twice, which for a customer-facing sync is silent data loss rather than an error anyone notices. Cost: the database still produces and discards the skipped rows, so page N costs time proportional to N times the page size and a deep page on a large table degrades from milliseconds to seconds. Keyset pagination over a stable, unique, indexed ordering -- where (created_at, id) < ($1, $2) order by created_at desc, id desc limit $3 -- is constant-cost per page and immune to shifting, on the precondition that the cursor columns never change value for a row, which disqualifies updated_at as a cursor.
Tests that assert on the implementation rather than the behaviour
Assert on what a caller can observe, not on the number of internal calls or the shape of a private field. A test that breaks on every refactor but still passes when the answer is wrong costs more than it protects.
Comparing floating-point values for equality, or holding money in them
Binary floating point cannot represent 0.1 exactly, so repeated addition drifts and an equality check fails on values that are mathematically equal. Store currency as integer minor units or a decimal type, and compare floats against a tolerance you chose for a stated reason.
Questions & practice
Create a program to count the frequency of a specific character (e.g., 'a') within a file.
Medium · data structures and algorithms
Approach
- State the target complexity and say which constraint rules the naive version out.
- Restate the input: its shape, its size, and what is guaranteed about it.
- Walk one small example through your approach before writing the whole thing.
Follow-up
- What is the worst case, and how likely is it on real data?
- How does this change if the input no longer fits in memory?
Implement binary search from scratch in Java.
Medium · data structures and algorithms
Approach
- Choose the data structure from the access pattern, not from familiarity.
- State the target complexity and say which constraint rules the naive version out.
- Name the brute-force solution and its complexity before improving on it.
Follow-up
- How does this change if the input no longer fits in memory?
- What is the worst case, and how likely is it on real data?
How do you approach debugging or optimizing existing codebases?
Medium · data structures and algorithms
Approach
- Restate the input: its shape, its size, and what is guaranteed about it.
- Choose the data structure from the access pattern, not from familiarity.
- Walk one small example through your approach before writing the whole thing.
Follow-up
- How does this change if the input no longer fits in memory?
- Which test case would catch an off-by-one here?
Fold a deduplicated usage stream into hourly rollups
Easy · aggregation · deduplication · watermarks · exact-arithmetic
You are given one day of usage_event rows, up to 250 million, each carrying event_id, tenant_id, workspace_id, environment, sku, quantity numeric(20,6), idempotency_key, occurred_at and ingested_at. Produce usage_rollup_hourly cells keyed (tenant_id, workspace_id, sku, hour_start) with quantity_sum, event_count and source_max_ingested_at. An event counts once per (tenant_id, idempotency_key). The rollup grain has no environment column, so state your filter. One pass. Give your time and space bounds, and say what the deduplication actually costs in memory.
Approach
- Bucket on
occurred_at, neveringested_at:hour_start = date_trunc('hour', occurred_at at time zone 'UTC'). The two columns answer different questions.occurred_atsays which hour the customer is billed for;ingested_atsays how current the fold is. Using the second for the first makes late data invisible instead of correctable. - The fold is trivial and the deduplication is the entire cost, so price it before designing anything clever. An exact set over
(tenant_id, idempotency_key)at 250M entries, stored as a 16-byte 128-bit hash in an open-addressed table at 0.7 load factor, needs about 357M slots at 16 bytes each, roughly 5.7 GB. The fix is partitioning byhash(tenant_id) % Pso each shard holds 1/P of the set and no tenant's keys straddle shards. - Rule out a Bloom filter as a replacement, in the right direction: a false positive reports 'already seen' for an event never seen, so you drop a real event and lose revenue with no error raised. It is usable only as a negative pre-filter in front of the exact set, where a miss is conclusive and a hit must fall through to the real lookup.
- Accumulate in scaled integers, not binary floating point.
numeric(20,6)admits values below 10^14, so one event scaled to micro-units can reach 10^20, past int64's 9.22 x 10^18; use a 128-bit or arbitrary-precision accumulator unless you first bound the per-event maximum. binary64 represents integers exactly only to 2^53, about 9.01 x 10^15, and cannot represent 0.1 at all, so two runs that sum in different orders disagree. - Carry
source_max_ingested_at = max(ingested_at)over the events folded into each cell, and countevent_countover accepted, post-dedup events. Without that watermark there is no way to prove later what a number did and did not include, which is the first question any reconciliation asks. - State the environment filter explicitly, because the rollup grain cannot record it. A fold that quietly includes
stagingbills non-production traffic; one that quietly excludes it loses a cost signal. Production-only is the billing answer, and either way it belongs in the job name and the output metadata. Complexity: O(n) time, O(distinct dedup keys) space, dominated by the dedup set rather than by the cells.
Worked solution (25 min)
- Write both key tuples down before any code: dedup key
(tenant_id, idempotency_key), cell key(tenant_id, workspace_id, sku, hour_start), withhour_startderived fromoccurred_atin UTC. - Build a 10,000-row fixture containing one event duplicated three times under the same
idempotency_key, two events sharing anidempotency_keyacross differenttenant_idvalues, one event whoseoccurred_atis two hours before itsingested_at, and onestagingevent inside an otherwise production cell. - Fold it and assert each of those four expectations separately rather than eyeballing a grand total.
- Re-run with the input shuffled and diff the output files.
- Size the dedup set for 250M keys using the load-factor arithmetic and write the number down next to the fixture.
Expected result: The triplicate contributes one event and its quantity once. The two same-key, different-tenant events both count, because the dedup key is the pair. The late event lands in the hour of its occurred_at while that cell's source_max_ingested_at advances to the later timestamp. The staging event is included or excluded per the stated filter and never silently.
Verify it yourself
- Shuffled input produces byte-identical cells, including
event_countandsource_max_ingested_at. - The sum of
quantity_sumacross all cells equals the sum of quantity over the distinct dedup keys, computed independently. - Swapping the accumulator to binary64 makes a fixture of 0.1-valued quantities disagree with the exact sum. That is a failing test, not a rounding note.
Follow-up
- A producer retries at 23:59:59 and the retry lands at 00:00:01. The unique index on the daily-partitioned table must include the partition key. What gets double-counted, and what is the smallest change that fixes it?
- The consumer acknowledges its batch before committing the fold. Which failure loses revenue now, and which arrangement duplicates instead?
- What makes a re-run over the same day produce byte-identical rollups?
Find the join that inflates every invoice total
Medium · join fan-out · aggregation grain · sql correctness
invoice_line_item holds line_id, invoice_id, tenant_id, sku, rate_tier, quantity, unit_price_micros, amount_minor (bigint), currency, kind, voided_at. invoice_payment_attempt holds attempt_id, invoice_id, tenant_id, amount_minor, status (succeeded, failed, pending), created_at, and an invoice has many attempts. A finance report runs select i.invoice_id, sum(l.amount_minor), count(p.attempt_id) from invoice i join invoice_line_item l using (invoice_id) join invoice_payment_attempt p using (invoice_id) group by 1 and the totals are wrong. Say precisely what the sum now equals, and write a version that is also correct for invoices with zero attempts.
Approach
- Compute what the query actually returns before fixing it. The two joins form a Cartesian product per invoice, so each line row repeats once per attempt row:
sum(l.amount_minor)is the true total multiplied by the attempt count, andcount(p.attempt_id)is attempts times lines. Three lines and two attempts report double the money and six attempts. - Reject the reflex repair.
count(distinct p.attempt_id)does fix the count, because attempt_id is unique.sum(distinct l.amount_minor)does not fix the sum, because two legitimate lines with equal amounts collapse into one. DISTINCT inside an aggregate deduplicates values, not rows, and the difference stays invisible until two lines happen to match. - Aggregate each branch to invoice grain before joining: one CTE summing lines by invoice_id, one counting attempts by invoice_id, then join the two results. A LATERAL subquery per invoice is equivalent and sometimes plans better when the outer set is small. Either way every aggregate stays at the grain it was defined at.
- Keep invoices with no attempts by making the attempt branch a LEFT JOIN with
coalesce(attempt_count, 0). An inner join here silently drops every unpaid invoice, which is usually the exact population finance is asking about. - Push each filter to its own grain:
where l.voided_at is nullbelongs inside the line CTE, not the outer query, or it would also filter the attempt branch through the join. Put the tenant predicate on both branches, since the denormalised tenant_id is what stops a wrong join crossing tenants. - Leave yourself a standing check: an invoice total is a function of its non-voided lines and of nothing about payments, so if changing the payment filter moves the money figure, the fan-out is back.
Worked solution (25 min)
- Create one invoice with three lines of 1000, 1000 and 500 minor units and two payment attempts, then run the original query.
- Confirm it reports 5000 and 6 rather than 2500 and 2.
- Apply
sum(distinct l.amount_minor)and confirm the total becomes 1500, which is worse rather than better. - Write the two-CTE version with a LEFT JOIN and coalesce, and confirm 2500 and 2.
- Add a second invoice with lines and no attempts and confirm it still appears.
Expected result: The original sum equals the true line total multiplied by the number of payment attempts and the count equals attempts times lines; the corrected query pre-aggregates each branch to invoice grain in separate CTEs and LEFT JOINs them with coalesce, with voided lines excluded inside the line CTE and the tenant predicate applied to both branches.
Verify it yourself
- The money total is unchanged by adding or removing payment attempts.
- An invoice with zero attempts appears with its total and a zero count.
- Two lines with identical amounts both contribute to the total.
- Voiding a line changes the money figure and leaves the attempt count alone.
Follow-up
- Add a third branch for credit notes applied to the invoice. Does the CTE shape still hold, and when would a single pass with
filter (where ...)be better? - Over 500k invoices this report takes minutes. Which grain would you materialise, and how do you keep it correct when a line is voided?
- The same report is needed per tenant per month. What index makes the line CTE cheap?
Model credential revocation so history survives the delete
Easy · soft delete · partial indexes · check constraints · audit history
tenant_api_key stores key_id, tenant_id, workspace_id, name, key_prefix, secret_hash, scopes text[], status (active, revoked, expired, compromised), auth_version, created_at, expires_at, last_used_at, revoked_at, revoked_reason. Rotation inserts a new row and revocation never deletes, because an incident review asks which credential served a request last quarter. Write the constraints that enforce: a label is unique only among a tenant's live keys, revoked_at and status can never disagree, and scopes is never empty. Then write the authentication lookup predicate, and name one column in this table that must stay out of it.
Approach
- Reach for a partial unique index rather than a plain UNIQUE:
create unique index on tenant_api_key (tenant_id, name) where revoked_at is null. Any number of revoked rows may share a label, the live namespace stays unique per tenant, and the revoked majority is not in the index at all, so it stays small on a table that only grows. - Tie the nullable timestamp to the enum so the two cannot drift:
check ((revoked_at is not null) = (status in ('revoked','compromised')))andcheck ((revoked_at is null) = (revoked_reason is null)). A revocation that records no reason is the one an incident review cannot use. - Write the emptiness check as
check (cardinality(scopes) > 0), notarray_length(scopes, 1) > 0. array_length returns NULL for an empty array, a CHECK constraint passes when its expression is NULL, so the array_length version accepts exactly the value it was written to reject. - Make the lookup a single index probe with every liveness condition inside it:
where secret_hash = $1 and revoked_at is null and (expires_at is null or expires_at > now()) and auth_version = $2, backed by a unique index on secret_hash. Nothing is filtered in application code, so there is no path that forgets a clause. - Keep last_used_at out of that predicate. It is written asynchronously and is allowed to lag by a minute, so it is a usage signal; feeding it into an authorisation decision makes the decision depend on a write that may be late, batched away or lost.
- Flag the modelling smell while you are here:
expiredis derivable fromexpires_at < now(), so storing it as a status obliges a job to keep it true and guarantees the column is wrong between the expiry instant and that job's next run. Derive it in the predicate; keep the stored status for states that are decisions rather than clock readings.
Follow-up
- Rotation issues a replacement while the old key stays live for a 30-day overlap. What does the uniqueness rule become, and what does the UI show to tell two same-named keys apart?
- A password reset bumps the principal's auth_version. No row in this table changed. How does the next request fail, and what query counts how many keys that bump just killed?
- A key turns up in a public repository. Which columns let you find it, and what do you write to the row?
Describe your understanding of L2 networking and basic security domains.
Medium
Approach
- Say what you would check first and why it is the highest-information step.
- State your assumptions explicitly before working the problem.
- Work from the requirement backwards to the design.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Metering ingest that survives a six-hour producer replay
Hard · idempotency · dedup horizon · partitioning · watermarks
metering-ingest consumes usage events at-least-once - 250M/day, 10-40k/second at peak - and folds them into usage_rollup_hourly keyed (tenant_id, workspace_id, sku, hour_start). usage_event is partitioned daily on ingested_at with unique (ingested_day, tenant_id, idempotency_key). A producer outage ends in a six-hour replay that re-sends events already ingested, some of whose originals crossed midnight. Design the consumer: partitioning, where the acknowledgement sits relative to the commit, the deduplication horizon and its storage cost, and how the rollup watermark advances. Nothing may be double-counted and nothing may be silently dropped.
Approach
- Choose the acknowledgement position deliberately and name what each choice costs. Acknowledging after the fold commits makes the consumer at-least-once: a crash between the two replays the batch and produces duplicates, which are ordinary and absorbable. Acknowledging first makes it at-most-once: a crash between the two drops revenue with no error raised anywhere and no way to detect it later. Take at-least-once and design everything downstream to absorb duplicates.
- Put the dedup and the fold in one transaction so there is no window between them. Insert the batch into usage_event with ON CONFLICT DO NOTHING, take the rows actually inserted, and fold only those into usage_rollup_hourly with an upsert on (tenant_id, workspace_id, sku, hour_start) bucketed by occurred_at, not ingested_at. A batch of about 2,000 rows is one round trip and one index probe per event.
- Attack the partition-key flaw head on: the unique index includes ingested_day because a unique index on a partitioned table must contain the partition key, so the same (tenant_id, idempotency_key) re-sent after midnight is a different index entry and passes. Deduplicate instead against a store keyed (tenant_id, idempotency_key) with no date component, whose horizon exceeds the producer's maximum retry window plus the longest replay you intend to support. At 14 days that is 250M x 14 = 3.5 billion keys, which is a dedicated key-value store, not a larger index on the same table. The alternative - partitioning usage_event on (tenant_id, occurred_day) so the natural key is stable - fixes dedup but loses pruning on ingest time and makes retention by dropping partitions awkward.
- Partition the consumer by hash of tenant_id so one tenant's replay stalls only its own partitions, and give replay traffic a separate lower-priority lane so live ingest keeps its latency. The cost is explicit: that tenant's watermark lags while the replay drains, and everything gated on the watermark waits for it.
- Define the watermark as a property of committed work, not of wall-clock time: per partition it is the largest occurred_at such that every event with a smaller occurred_at has committed, and the sealing decision uses the minimum across partitions. Record source_max_ingested_at on every rollup row so any number can prove what it did and did not include, and keep restatement legal only while status = 'open' - after sealed_at the value is frozen and a late event becomes an invoice adjustment instead.
Worked solution (40 min)
- Write the consumer loop in pseudocode with the acknowledgement after the commit, then annotate each line with what is lost or duplicated if the process dies exactly there.
- Size the dedup store: events/day x horizon_days keys, bytes per key including the tenant prefix, and the resulting memory or disk. Compare that cost against simply extending retention on the partitioned table and say why the latter does not fix the problem.
- Take one event ingested at 23:59:58 and replayed at 00:00:04 and work out its fate under (a) the partitioned unique index alone and (b) the separate dedup store.
- Write the per-partition watermark formula, then what the seal uses, then what a single stalled partition does to sealing.
Expected result: An at-least-once consumer acknowledging after the commit; deduplication on (tenant_id, idempotency_key) in a store whose horizon of at least 14 days exceeds max retry window plus supported replay length, sized at roughly 3.5 billion keys; tenant-hashed partitions with replay on a separate lane; a per-partition watermark with sealing gated on the minimum across partitions; source_max_ingested_at written on every rollup row.
Verify it yourself
- The midnight-crossing retry must be rejected by whatever you chose. If the table's unique index is your only dedup, it is accepted and the invoice is wrong by exactly that event.
- You should be able to name a distinct failure for acknowledge-before-commit and for acknowledge-after-commit. If both sound the same, the position was not actually chosen.
- quantity_sum for any (tenant, workspace, sku, hour) must be reproducible by re-aggregating usage_event filtered to ingested_at <= source_max_ingested_at. If it is not reproducible, the rollup is an opinion rather than a projection.
Follow-up
- The dedup store is lost entirely. What can you still guarantee, and how do you rebuild it from what remains?
- A replay delivers events for an hour that is already sealed. Trace exactly what happens to them, row by row.
- One partition is stuck on a poison message, so the minimum-across-partitions watermark never advances and no tenant can be sealed. What is your escape hatch and what does it cost in correctness?
Hourly rollups merge one hour and lose another
Easy · timezone · rollups · reconciliation
Reconciliation flags one tenant on one day. Summing usage_event.quantity by hour of occurred_at gives 24 non-empty hours, but usage_rollup_hourly holds 23 rows for that tenant, workspace and SKU, one of which carries roughly the sum of two adjacent hours. Other days reconcile exactly, and the affected date matches a civil-time transition. hour_start is documented as truncated to the hour in UTC. You have both tables, the rollup job source, and its runtime environment. Give an ordered checklist, the mechanism, and the correction path for a day that may already be sealed.
Approach
- Bisect by dimension until one cell explains the whole difference: tenant, then day, then SKU, then hour. A defect confined to a single transition date already rules out deduplication and late arrival, both of which are indifferent to which hour an event lands in.
- Read the truncation with its precondition stated: date_trunc on a timestamptz value is evaluated in the session TimeZone, not in UTC. If the job connects without pinning that setting, it inherits the server or container default.
- Follow that to the collision: in a zone that observes daylight saving, two distinct UTC hours map to the same local wall-clock label at the autumn transition, so both fold into one key under the unique constraint on (tenant_id, workspace_id, sku, hour_start) and their quantities sum into one row. At the spring transition a label never occurs and the row is simply absent.
- Confirm from data rather than from reading code: run the same aggregate twice, once with the session pinned to UTC and once with the job host zone, and check that the second reproduces the stored rollup exactly.
- Fix at the source by pinning the connection to UTC explicitly, or by truncating on occurred_at AT TIME ZONE 'UTC', rather than relying on a default that differs between a developer machine, CI and production.
- Correct according to status, not convenience: an open hour is recomputed with revision incremented, a sealed hour is frozen and the difference becomes an adjustment line on the next invoice with voided_by_line_id pointing at the line it reverses.
Follow-up
- The same job also emits a daily figure for a dashboard. Why can a correct hourly rollup still produce a wrong day, and what does the tenant's billing timezone have to do with it?
- How would you detect this class automatically rather than waiting for reconciliation, given that it only manifests twice a year per zone?
Suggested practice checklist
For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.
Day 1 — Rebuild the primitives by implementing them
- Implement a dynamic array with doubling growth and an operation counter, then change the growth rule to add a fixed sixteen slots instead, and time both for n of ten thousand, a hundred thousand and a million. The fixed-increment version resizes n/16 times at O(n) each, so its total work is quadratic; doubling is what makes append amortised constant.
- Implement a hash map with separate chaining and a load-factor resize, then insert ten thousand keys engineered to land in one bucket and record what happens to lookup time, so that average-case O(1) becomes a claim with a stated precondition rather than a reflex.
- For dynamic-array append and hash-map insert, write down which cost is amortised rather than worst-case, which single operation pays the whole bill, and what a system with a hard per-operation deadline would have to do instead.
Deliverable: Two working implementations plus a timing table showing the input at which each structure's advertised complexity stops holding.
Day 2 — Arrays under an invariant: two pointers, sliding window, binary search
- Solve longest-subarray-with-sum-at-most-K using a sliding window, then run it on an input containing negative numbers and watch it return the wrong answer: extending the window only moves the sum monotonically when every element is non-negative, and that precondition is the whole reason the technique works.
- Write the binary search that finds the first index satisfying a predicate rather than an exact value, put the loop invariant above the loop in a comment, and verify termination on the two inputs that break careless versions: the empty range, and a range where every element satisfies the predicate.
- Compute the midpoint as lo + (hi - lo) / 2 and write one line on why the obvious (lo + hi) / 2 is a genuine defect in a fixed-width integer type and a non-issue in a language with arbitrary-precision integers.
Deliverable: Three solved problems, each with its invariant written above the loop, plus one recorded input on which the sliding window is provably wrong.
Day 3 — Sorting, heaps, and the greedy argument that has to be proved
- Solve one top-k problem three ways, by full sort, by a size-k heap, and by quickselect, then write the values of n and k at which each becomes the right choice, along with quickselect's quadratic worst case and why a randomised pivot makes that unlikely rather than impossible.
- Implement bottom-up heapify and count sift-down steps to confirm it does linear work rather than n log n, because most nodes sit near the bottom of the tree and therefore move only a short distance.
- Take interval scheduling by earliest finishing time and write the exchange argument out in full: given any optimal schedule, swapping in the earliest-finishing interval keeps it feasible and no smaller. Then construct the weighted variant where that same greedy fails and name what has to replace it.
Deliverable: A three-way top-k comparison with measured crossover points, one written exchange argument, and one counterexample to a greedy rule that looks almost identical.
Day 4 — Recursion, memoisation, and the step to a table
- Take one problem with overlapping subproblems, such as edit distance or coin change, instrument the plain recursion with a call counter to show the blow-up, then add memoisation and re-count.
- Convert the memoised version to a bottom-up table and state the two properties you relied on: each subproblem's result depends only on its arguments, and the dependencies form a DAG you can enumerate in order.
- Rewrite one deep recursion with an explicit stack, then find the input length at which the original hits the interpreter's frame limit, which defaults to about a thousand frames in CPython, so you know when the rewrite is required rather than decorative.
Deliverable: One problem in three forms, naive, memoised and tabulated, with call counts for each and the input length at which recursion depth becomes the binding constraint.
Day 5 — Graphs, where most of the work is choosing the traversal
- Implement BFS and DFS over one adjacency list, then answer for each which finds a shortest path in an unweighted graph and which you would use to detect a cycle in a directed graph, including why the in-progress versus finished distinction matters for the second.
- Implement topological sort by in-degree, feed it a graph containing a cycle, and confirm the failure signature is that fewer than V nodes come out rather than an exception, then note that the order it produces is one of several valid ones.
- Run a shortest-path search on a graph with a single negative edge weight and show the wrong answer, then write the precondition Dijkstra actually needs, non-negative weights, because it finalises a node's distance the first time that node is popped, and name the algorithm you would switch to and its own limit.
Deliverable: A small graph library with BFS, DFS and topological sort, plus two inputs that produce documented wrong answers under the wrong algorithm choice.
Day 6 — One day for everything that is not an algorithm
- Sketch one system only to the depth a coding-heavy loop tends to reach: the endpoints, what the service stores, and the single query pattern that decides the schema. Stop at twenty-five minutes.
- Prepare the project answer for an interviewer who codes, which means rehearsing the two levels they push to: the specific thing you built, and why you chose that approach over the alternative they will name. Open with a number and be ready to say what it excludes.
- Prepare the answer to what you would do differently, choosing a real technical mistake with a specific fix rather than a complaint about process or staffing.
Deliverable: One design sketch at endpoint-and-schema depth, plus a project answer rehearsed to two levels of follow-up.
Day 7 — Solve out loud, under time
- Do three timed problems at twenty-five minutes each in a plain editor with no autocomplete and no execution until the end, then tally separately the failures that were syntax and the ones that were approach, because those two numbers call for different fixes.
- Narrate one solution from the first sentence, stating the approach and its complexity before writing any code, and rehearse the sentence you will use when you realise mid-solution that the approach is wrong.
- Re-solve from blank the two problems you were slowest on this week and compare the times against the day they first appeared.
Deliverable: A recording of one fully narrated solution and a tally that separates syntax failures from approach failures.
The bug you decided not to fix
Engineers over-index on what they repaired. A stronger answer covers something you knowingly left broken: the alert you tuned down, the data inconsistency you documented instead of chasing, the cleanup you deferred past two quarters. Give the reasoning and the condition that would have reopened it, so it reads as a decision and not as neglect.
What is your process for integrating feedback from senior engineers on your code?
Medium · behavioural and engineering judgement
Approach
- Give the blast radius: what could have broken, and what you measured.
- Name the disagreement and how you resolved it with evidence.
- State the situation in two sentences and spend the rest on the reasoning.
Follow-up
- What did you decide not to do, and why?
- How did you know your change caused the improvement?
Describe your approach to cross-functional collaboration.
Medium · behavioural and engineering judgement
Approach
- Pick a story where you made the decision, not one where you watched it.
- Name the disagreement and how you resolved it with evidence.
- Give the blast radius: what could have broken, and what you measured.
Follow-up
- How did you know your change caused the improvement?
- What would you do differently if you ran that again?
Explain your experience with multi-threading and locking mechanisms.
Medium · behavioural and engineering judgement
Approach
- Give the blast radius: what could have broken, and what you measured.
- State the situation in two sentences and spend the rest on the reasoning.
- Pick a story where you made the decision, not one where you watched it.
Follow-up
- What would you do differently if you ran that again?
- How did you know your change caused the improvement?