Interview guides · Amazon Advertising
Amazon Advertising Software Engineer Interview Questions & Guide 2026
Caching practice for Amazon Advertising software engineer prep: settle the staleness budget before choosing an invalidation strategy.
The 60-second brief
As a Software Engineer at Amazon Advertising, you will build high-performance, low-latency, and massively scalable systems that power Amazon's global ad-tech platform. Amazon Advertising operates at an extraordinary computing scale, processing billions of ad requests, real-time auctions, and telemetry events daily. In this role, you directly influence how millions of global brands, agencies, and sellers reach customer audiences across Amazon properties and partner media channels.
If the loop includes an asynchronous take-home, treat it as a code review of you rather than as a puzzle: structure, what you chose to test, and what you wrote down about the constraint you were working under. Hold the stated time box and say what you would have done with more of it.
Focus areas
- Restate attribution results without double-counting credit
- Make impression billing idempotent under at-least-once delivery
- Join late-arriving conversions to windowed touchpoints
Overview
Your work spans complex technical domains, including real-time auction engines, high-throughput analytics pipelines, predictive ad-targeting algorithms, and self-service campaign management APIs. Engineers in Amazon Advertising tackle challenges that push the boundaries of distributed computing, requiring sub-millisecond response times, absolute data consistency for financial transactions, and fault-tolerant architecture capable of handling extreme traffic spikes during major events like Prime Day.
This position demands strong foundational engineering skills in algorithms, object-oriented design, and system architecture, paired with deep alignment with. Joining offers you the opportunity to write code that directly drives multi-billion-dollar business outcomes while designing infrastructure that operates at true hyper-scale.
The Amazon Advertising interview process
Amazon Advertising candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.
Recruiter Screening
Before anything technical happens, someone has to decide which rung of the ladder your loop is calibrated to, and that decision sets the bar for every round after it. It comes from how you describe scope, not from your title, because titles do not convert cleanly between companies. The weak version of the answer is team size and years. The strong version names the largest change you shipped where nobody reviewed the design, what would have broken if you had been wrong, and what you were paged for. Get the level said out loud on this call, because the range and the loop both follow from it.
What to demonstrate
- Whether the scope in your own account maps onto a level the team actually has an opening at, so a mismatch ends the process cheaply rather than after four interviewers have spent a day
- Whether your title needs re-mapping: the same word describes very different amounts of independent decision-making at a twenty-person company and a ten-thousand-person one
- Whether your compensation expectation can be filled at that level in the structure the role pays in, which is why the number gets asked for before any engineer is scheduled
How to prepare
- Write down two changes from the last two years: the largest one you designed with nobody reviewing the design, and the largest one where someone more senior did. Lead with the first when scope comes up, and be ready to say which parts of the second were yours
- Ask which level the loop is calibrated to and what changes at the level above it, then plan your weeks from that answer rather than from the posting
- Settle a total-compensation range beforehand with the split named, base against bonus against equity and its vesting period, so a question about numbers gets a number instead of the word market
Online Assessment
Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.
What to demonstrate
- Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
- Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
- Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
- Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply
How to prepare
- For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
- For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
- Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last
Technical Interviews
What this round decides is narrow: whether you can produce code that runs and is correct on inputs nobody showed you. An elegant solution that does not compile scores below a plain one that does, so write a correct brute force first, say out loud that you know its cost, and improve it with the working version still on screen. What separates strong answers is who finds the broken case. Trace your own code against an empty input, a single element, and duplicate keys before you say you are finished, because being told is far more expensive than noticing.
What to demonstrate
- Whether degenerate inputs get checked without being asked for: an empty collection, one element, every element equal, and the extreme value the input type allows
- Whether the complexity you state matches the code you actually wrote, including a sort or a copy sitting inside a loop
- Whether the finished answer is verified against the worked examples before you call it done, rather than assumed correct because the code reads correctly
How to prepare
- Take five problems you have already solved and, without running anything, write down what each returns for empty input, a single element, and all-duplicates. Then run them and count how many you predicted wrong.
- Drill the brute force as its own skill: on ten problems, write only the obviously-correct slow version and time how long it takes to get it passing. If that is more than a few minutes, that is what to practise, not the optimal version.
- Add a fixed last step before you submit anything, reading only the loop bounds and the initial value of each accumulator, which is where most off-by-one errors live
Behavioral Questions
Your first answer is not really what is scored. It buys the follow-up questions, and those decide the round. An interviewer with fifteen minutes takes one thread and pushes on it four or five times, so a story you can only tell at a single level of detail collapses under the third why. That is an argument for fewer stories known deeply rather than one prepared per prompt. Four or five pieces of work you can still explain down to the code you changed and the argument you had about it will cover nearly anything asked in this round.
What to demonstrate
- Whether a story holds as the questioning moves from what you did to why that instead of the alternative, and then to what you would change knowing what you know now
- Whether you can re-cut a project to answer the question actually asked rather than delivering a rehearsed block that answers an adjacent one
- Whether your level of detail is chosen rather than habitual: going down to the schema when the question is about the data model, staying out of it when the question is about the person who disagreed with you
How to prepare
- Pick four projects and write the chain out four levels deep for each: what you did, why that, why not the alternative, and what would have to be true for the alternative to have won. Where you cannot reach the fourth level, you have a placeholder rather than a story
- Have someone ask why three times in a row on a single thread with nothing else added, and mark the point where you start repeating a sentence you already said. That point is where the interviewer stops learning anything
- Build a one-page index instead of an answer bank: the common prompts in this round (disagreement, a failure that was yours, thin requirements, a deadline you missed, work you inherited) mapped to which of your four projects you would use for each, so the choosing is done now rather than while an interviewer waits
Bar Raiser Interview
You cannot drill a format you do not know, so put the preparation into material that travels. Three pieces of your own work, each rehearsed until you can take a follow-up you did not anticipate, will carry a conversation or a code walkthrough equally well. Specificity is what separates that from filler. A number needs its definition before it means anything: a p99 is over some window and measured at some hop, and a server-side figure excludes the queueing and network time a client would see. The number you cannot qualify is the one to leave out.
What to demonstrate
- Whether your examples carry detail only someone who did the work would hold, such as what the binding constraint actually was, which alternative you rejected and why it was worse, and what you measured on each side of the change
- Whether a number survives one follow-up, meaning you can say what it was measured over and whether it moved because of your change or merely alongside it
- Whether a failure is described with the specific change that followed it, rather than a lesson stated in general terms
- Whether your part in a team effort is stated accurately, including what other people did
How to prepare
- Write a page on each of three projects covering the constraint, the option you rejected, the measurement before and after, and what went wrong. Cut any line you cannot take a follow-up on, since you are writing the parts you will be pressed on rather than a summary.
- Recover the real figures while you still have access: request volume, data size, latency with its percentile and window, team size, timeline. Note where each came from, whether a dashboard, a design document or memory, and mark the estimates so you can say which they are out loud.
- Take your weakest project story to someone who works in a different area and have them ask why four times in succession. The point where you run out of answer is the part to go and re-read before the round.
Where candidates lose points
Recomputing attribution by inserting the new result.
Late touchpoints and late conversions make recomputation routine rather than exceptional, so the same conversion accumulates credit rows across runs and the fractions for one conversion sum above one. Nothing fails, no constraint fires, and every downstream aggregate inflates quietly in proportion to how often the job has run, which makes the error look like genuine growth. Correctness requires supersession with exactly one live revision per conversion and model, and the check worth writing is the assertion that live credit fractions sum to one, because it catches the class rather than the instance.
Treating the win notice as the billing signal.
Winning an auction and rendering an ad are different events separated by a page that may never finish loading, which is exactly why the protocol carries a separate billing notice alongside the win notice. Charging on the win inflates spend against impressions that were never served, and because the inflation is proportional to render failure rates, it varies by supply source and shows up as an unexplained discrepancy against the exchange's own numbers rather than as an obvious bug. The related error is assuming the clearing price equals the bid, which stops being true the moment a supply source runs anything other than a straight first-price auction, and is never knowable at bid time because the price arrives substituted into the notice.
Reading the constraints as preamble rather than as part of the problem
The bounds are usually there to eliminate the obvious approach: n up to 10^5 makes an O(n^2) scan roughly 10^10 operations, far outside any per-test time budget, and an input larger than memory rules out loading it at all. When a bound is not given, ask for it, then say out loud which approach it kills.
Issuing one query per row of a result set
Fetch related rows in a single batched query keyed by the ids you already hold, or join them into the original query. A per-row round trip multiplies network latency by the row count, and it looks perfectly fine against the ten rows in your development database.
Questions & practice
Given a circular network of delivery hubs, calculate the minimum total travel time required to visit a specific sequence of hubs using clockwise or counter-clockwise movements.
Medium · data structures and algorithms
Approach
- Choose the data structure from the access pattern, not from familiarity.
- Name the brute-force solution and its complexity before improving on it.
- Restate the input: its shape, its size, and what is guaranteed about it.
Follow-up
- What is the worst case, and how likely is it on real data?
- Which test case would catch an off-by-one here?
Implement an LRU Cache supporting get and put operations in strict O(1) time complexity while addressing thread-safety considerations.
Medium · data structures and algorithms
Approach
- Walk one small example through your approach before writing the whole thing.
- Name the brute-force solution and its complexity before improving on it.
- Restate the input: its shape, its size, and what is guaranteed about it.
Follow-up
- What is the worst case, and how likely is it on real data?
- How does this change if the input no longer fits in memory?
Find the longest contiguous subarray containing an equal number of zeros and ones, optimizing for both time complexity and memory overhead.
Medium · data structures and algorithms
Approach
- State the target complexity and say which constraint rules the naive version out.
- Choose the data structure from the access pattern, not from familiarity.
- Walk one small example through your approach before writing the whole thing.
Follow-up
- Which test case would catch an off-by-one here?
- How does this change if the input no longer fits in memory?
Given a 2D terrain grid containing radio towers, mountains, and empty spaces, identify all pairs of radio towers that can communicate along unobstructed straight lines.
Medium · data structures and algorithms
Approach
- Walk one small example through your approach before writing the whole thing.
- Restate the input: its shape, its size, and what is guaranteed about it.
- State the target complexity and say which constraint rules the naive version out.
Follow-up
- How does this change if the input no longer fits in memory?
- What is the worst case, and how likely is it on real data?
Design an Amazon Locker delivery and pickup management system, defining classes, spot allocation strategies, and interface abstractions.
Medium · languages, concurrency and fundamentals
Approach
- Distinguish a value from a reference to it, and say which one you handed out.
- Reach for the cheapest primitive that closes the race, not the broadest lock.
- Identify the window where an invariant is briefly untrue.
Follow-up
- What happens if two callers reach this at the same time?
- How would you prove the race exists rather than suspect it?
Implement a Unix-like command-line interface file system walker capable of listing and filtering directories and files efficiently.
Medium · languages, concurrency and fundamentals
Approach
- Say what the runtime actually does before reasoning about the code.
- Identify the window where an invariant is briefly untrue.
- Name what is shared across threads and what owns each piece of state.
Follow-up
- How would you prove the race exists rather than suspect it?
- Where could this allocate more than you expect?
Attribute late conversions across windows without double-counting credit
Hard · windowed join · restatement · fixed-point arithmetic
Given touchpoints (identity_key, impression_id, line_item_id, touchpoint_type in {click, view}, event_ts) and conversions (conversion_id, identity_key, occurred_ts, received_ts), credit each conversion under a 30-day click window and a 1-day view window. Write the join and the credit split for the linear and position-based models, then handle a touchpoint that arrives after the conversion was already credited and reported. Credits per (conversion_id, model_id) must sum to exactly 1 in NUMERIC(9,8). State the complexity and the window state size.
Approach
- Partition both inputs by identity_key and sort by time, then merge-join. For each conversion the eligible touchpoints form a contiguous range ending at occurred_ts, so two binary searches bound it in O(log T) and the split is O(k) over the k touchpoints inside. The sort dominates at O((T + C) log (T + C)); the join itself is linear.
- Apply the two windows as separate lower bounds over the same sorted array - clicks from occurred_ts minus 30 days, views from occurred_ts minus 1 day - and partition by touchpoint_type during a single scan of the wider range rather than filtering the array twice.
- Split credit in integer units rather than floating point. NUMERIC(9,8) gives 10^8 units, so allocate floor(10^8 / k) to each of k touchpoints and distribute the remainder one unit at a time in a deterministic order: three touchpoints become 33333334, 33333333, 33333333 and sum to exactly 10^8. Equal decimal division yields 0.99999999 and fails the sum-to-one assertion on every three-touchpoint conversion.
- State position-based as a policy rather than a law, and write the conventions down: one touchpoint takes everything, two split by an agreed rule, three or more give the first and last their fixed shares with the remainder divided among the middle - under the same largest-remainder allocation, so the total still lands on 10^8.
- Restate by supersession, never by insertion, and put liveness on a header row rather than on the credit rows, because one revision of a multi-touch conversion is k rows by construction. Model it as attribution_revision (revision_id, conversion_id, model_id, is_live, superseded_by_revision_id, computed_at) with one row per attempt, and attribution_credit (revision_id, touchpoint_id, units) holding the k rows whose units sum to 10^8. In one transaction: insert the new header and its credits, then clear is_live on the prior header for that (conversion_id, model_id) and set its superseded_by_revision_id.
- Put the single-live-revision rule in the database, on the header: a partial unique index on attribution_revision (conversion_id, model_id) WHERE is_live. Note explicitly that the same index on attribution_credit would be wrong - linear and position-based crediting write one live credit row per touchpoint, so a (conversion_id, model_id) uniqueness constraint at credit granularity rejects every touchpoint after the first. Credit-level uniqueness, if you want it, is (revision_id, touchpoint_id).
- Size the state honestly: the join holds touchpoints for the longest window plus allowed lateness, so it is the active identity keyspace times 31 days of touchpoints. When that exceeds memory the window moves to a keyed store and the job becomes a lookup per conversion, trading the merge-join's sequential IO for random reads - say which you are choosing and why.
Worked solution (45 min)
- Take one conversion with four touchpoints - clicks 5 and 20 days before, a view 6 hours before, a view 3 days before - and list which are eligible under each window.
- Compute linear credit in integer units over the eligible set and show the remainder allocation landing on exactly 10^8.
- Add a click that arrives late but occurred 10 days before the conversion, recompute, and write out both headers - the superseded revision and the new live one - with the new revision's credit rows listed under it.
- Write the partial unique index on the revision header that makes a second live revision impossible, then show what the same index on the credit table would do to a four-touchpoint conversion, and add the assertion that units summed over the live revision's credit rows equal 10^8.
- State the window state size for 30 days plus 7 days of allowed lateness, and what you do when it does not fit in memory.
Expected result: A merge-join over identity-partitioned, time-sorted inputs with per-window lower bounds, integer largest-remainder credit allocation summing to exactly 10^8 units, and restatement by revision supersession: a partial unique index on the attribution_revision header (conversion_id, model_id) WHERE is_live, with per-touchpoint credit rows hanging off revision_id so a multi-touch conversion can hold k live credits, plus the join complexity and the window state size.
Verify it yourself
- Credit rows joined to their live revision header, grouped by conversion and model, sum to exactly 1 for every conversion, including the three- and seven-touchpoint cases where decimal division does not.
- A conversion with four eligible touchpoints stores four live credit rows under one live revision, and no uniqueness constraint rejects the second, third or fourth.
- Rerunning the job over unchanged input produces no new live revisions.
- The view 3 days before the conversion is excluded under the 1-day view window while the click 20 days before is included under the 30-day click window.
Follow-up
- A batch upload delivers conversions for a period you invoiced three weeks ago. What do you recompute, and what does the advertiser see change?
- The same purchase arrives from the browser pixel and from the server-to-server call. Where in this pipeline do you resolve the pair, and what do you record about the decision?
- Your remainder allocation is deterministic. What order does it use, and does that order survive a touchpoint arriving late?
Model attribution restatement so exactly one revision stays live
Hard · supersession · audit history · constraints
attribution(attribution_id, conversion_id, impression_id, line_item_id, model_id, touchpoint_type, credit_fraction, window_days, revision, is_live, superseded_by_attribution_id, computed_ts) is recomputed whenever a late touchpoint or late conversion changes an already-published result. A multi-touch model writes several rows per conversion, so several rows are live at once, but all live rows for one (conversion_id, model_id) must share a single revision and their credit_fraction must sum to 1. Give the PostgreSQL DDL that enforces what is enforceable, name what no constraint can enforce here, and write the transaction that publishes revision r+1.
Approach
- Separate the three claims by shape, because they need three different mechanisms. At most one live row per (conversion_id, impression_id, model_id) is a uniqueness property. All live rows sharing one revision is a property of a group of rows. Fractions summing to 1 is an aggregate predicate over that group.
- Make the revision itself a row: attribution_revision(conversion_id, model_id, revision, computed_ts, window_days) with PRIMARY KEY (conversion_id, model_id, revision). Detail rows carry that triple with a foreign key to it and UNIQUE (conversion_id, model_id, revision, impression_id). is_live and superseded_by_attribution_id then leave the detail row entirely.
- Make liveness one row: attribution_live(conversion_id, model_id, live_revision) with PRIMARY KEY (conversion_id, model_id) and a foreign key on (conversion_id, model_id, live_revision) into attribution_revision. Exactly-one-live is now that primary key, publishing is a single upsert, and no detail row is ever updated.
- Say why the boolean on the detail row is worse than it looks. A partial unique index on (conversion_id, model_id) WHERE is_live forbids the second touchpoint of any multi-touch model, and flipping the flag across two revisions in one statement can transiently violate the index, which cannot be deferred in PostgreSQL because deferrable uniqueness requires a constraint and unique constraints cannot carry a WHERE clause.
- Concede the sum-to-one invariant honestly: PostgreSQL has no CREATE ASSERTION and a CHECK constraint sees a single row, so this one is not enforceable declaratively. Implement it as a constraint trigger deferred to commit, or as a gate query inside the publish transaction, and say which.
- Write the publish transaction in order: insert the attribution_revision row, insert the detail rows for r+1, run the sum gate over exactly those rows, then upsert attribution_live. A failed gate aborts before the pointer moves, leaving revision r live and the rejected rows retrievable as history rather than lost.
Follow-up
- One conversion is restated hourly, so its attribution_live row is a write hot spot. What does that actually cost, and at what restatement rate does it start to matter?
- A report starts before the pointer flips and finishes after. Which revision does its output contain, and is a report that straddles a restatement acceptable or does it need a snapshot?
- You must delete revisions older than 90 days. How do you avoid dropping a revision that an issued invoice cited?
Explain why this billing-notice query ignores the primary key
Medium · indexing · sargability · partitioning
delivery_event is RANGE partitioned by received_ts date, has PRIMARY KEY (impression_id, event_type), and carries line_item_id, creative_id, exchange_id, clearing_price_micros, clearing_currency, event_ts, received_ts and ivt_status. A PostgreSQL reporting query filters line_item_id = $1 AND event_type = 'billing_notice' AND received_ts::date BETWEEN $2 AND $3 and sums clearing_price_micros. EXPLAIN shows a sequential scan on every partition. Give three separate reasons the existing index cannot serve it, rewrite the predicate so partition pruning applies, and specify the index you would add including column order.
Approach
- Reason one is coverage: the primary key contains neither line_item_id nor received_ts, and its leading column is a UUID with no predicate supplied, so there is no start point to seek to and nothing in the index that answers the filter. A btree scan begins from a prefix of its own column order or it begins from the start.
- Reason two is the cast on the partition key. received_ts::date applies a function to the column, so the planner cannot map the predicate onto partition boundaries and prunes nothing. Rewrite as a half-open range on the raw column, received_ts >= $2 AND received_ts < $3 + interval '1 day', and note that timestamptz to date conversion depends on the session TimeZone, so the rewrite must pin the zone or the day boundaries move.
- Reason three is parameter typing. If $1 arrives as numeric rather than bigint, the comparison casts the column rather than the constant, and a btree on line_item_id becomes unusable. Read the EXPLAIN for which side of the operator the cast landed on, because a cast on the constant is free and a cast on the column is fatal.
- Specify the index as (line_item_id, received_ts) with a partial predicate WHERE event_type = 'billing_notice', created on each partition. Equality column first so the scan starts at one point, range column second so the matching rows are contiguous, and the partial predicate shrinks the index to the minority event type that carries a price.
- Consider INCLUDE (clearing_price_micros) for an index-only scan, and state the precondition rather than asserting the win: an index-only scan requires the partition's visibility map to be largely all-visible, which holds for a sealed older partition and does not hold for the one currently receiving writes.
Worked solution (25 min)
- Count the partitions named in the EXPLAIN output; if it is all of them, pruning failed and the predicate is the reason.
- Rewrite the date filter as a half-open timestamptz range and re-run EXPLAIN to confirm pruning to only the overlapping partitions.
- Create (line_item_id, received_ts) WHERE event_type = 'billing_notice' on one partition and compare the plan and the buffers read with EXPLAIN (ANALYZE, BUFFERS).
- Inspect the EXPLAIN filter text for a cast applied to the column side of the comparison rather than the parameter side.
Expected result: Pruning to exactly the partitions overlapping the requested range, an index scan on (line_item_id, received_ts) restricted by the partial predicate, and a distinct reason given for each of the three failures: the primary key does not contain the filtered columns, a function on the partition key defeats pruning, and a parameter type forces a column-side cast.
Verify it yourself
- EXPLAIN names only the partitions whose bounds overlap the requested range.
- Buffers read fall by orders of magnitude against the sequential scan, and the row count returned is unchanged.
- The rewritten predicate returns the same rows as the original under the session's TimeZone, confirming the change moved the plan rather than the result.
Follow-up
- The partial predicate is a literal event type. What do you do when the same report also needs click counts, and is a second partial index better than one wider index?
- line_item_id = $1 matches 40% of the rows in a partition. Does the planner still choose the index, and what statistic decides?
- How do you create this index on a live partitioned table without blocking inserts?
Architect a cloud-based media streaming service (similar to Amazon Prime Video) emphasizing video ingestion, content delivery networks (CDNs), and playback state persistence.
Medium · system design and architecture
Approach
- Fix the scope first: who calls this, how often, and what they do when it fails.
- State the consistency you need, and where you are willing to be stale.
- Name the failure you are designing for, then the recovery path.
Follow-up
- What would you drop to keep the system up under load?
- What breaks first when traffic grows ten times?
Architect a distributed delivery service platform (similar to Uber or DoorDash) capable of handling real-time driver tracking, order matching, and route optimization.
Medium · system design and architecture
Approach
- State the consistency you need, and where you are willing to be stale.
- Name the read and write paths separately; they rarely have the same bottleneck.
- Fix the scope first: who calls this, how often, and what they do when it fails.
Follow-up
- What breaks first when traffic grows ten times?
- How does this behave when that dependency is down for an hour?
Paginate an append-only delivery log without skipping rows
Easy · pagination · cursors · event-ordering
An advertiser's reporting job syncs delivery_event rows for its line items. The table is keyed on (impression_id, event_type), partitioned by received_ts date, carries an untrusted client-asserted event_ts, and takes order 10^5 inserts per second. The job polls every five minutes and must not skip a row. The current shape is GET /delivery-events?offset=N&limit=1000 ordered by event_ts. Design the replacement: the ordering key, the cursor, the upper bound on what a page may return, and the guarantee the caller resumes against.
Approach
- Rule out OFFSET on both counts. Cost is linear in the offset because the engine still walks and discards the skipped rows, so page 500 reads half a million rows to return a thousand; and with concurrent inserts the underlying result shifts between requests, so a row can be returned twice or skipped entirely with nothing in the response indicating it happened.
- Order on received_ts rather than event_ts. event_ts is asserted by an untrusted client and can move backwards arbitrarily, so a cursor built on it makes no monotone progress. received_ts is server-assigned, which makes it the only timestamp in this table safe to order or partition on. Add a unique tie-break column for a total order and index on (received_ts, impression_id): the keyset predicate (received_ts, impression_id) > (:ts, :id) is then O(log n) to locate plus O(page) to scan.
- Handle commit visibility, which is where a correct-looking keyset scheme still loses rows: received_ts is stamped when the request is received, but the row becomes visible when its transaction commits, so a row stamped 12:00:03 can appear after a reader has already paged past 12:00:05. Bound every page by an as_of watermark set behind now by more than the worst commit lag, and return it so the next poll resumes from it.
- Return an opaque cursor carrying (received_ts, impression_id, as_of) rather than exposing the tuple as query parameters. That keeps the ordering key an implementation detail you can change, and stops callers hand-crafting a cursor that scans outside their own advertiser scope.
- State the guarantee in the contract in the terms the caller must design against: at-least-once, ordered by received_ts, complete up to as_of, caller deduplicates on (impression_id, event_type). Exactly-once is not available over polling, since the caller can crash between reading a page and committing its own progress.
Worked solution (20 min)
- Write both queries out and state the rows examined by each: ORDER BY event_ts OFFSET 500000 LIMIT 1000 against the keyset predicate with LIMIT 1000.
- Name the index the keyset query needs and confirm its column order matches the ORDER BY exactly, so the ordering comes from the index rather than from a sort over the partition.
- Construct the skip case concretely: a row stamped received_ts 12:00:03 that commits at 12:00:06, against a reader that paged past 12:00:05 at 12:00:05.5 — then show the as_of bound that prevents it.
- Define the cursor payload and the response envelope (items, next_cursor, as_of), then write the caller's resume loop including what it does when a page comes back empty.
Expected result: A keyset endpoint ordered by (received_ts, impression_id) with an opaque cursor carrying the as_of upper bound, a documented at-least-once guarantee with caller-side deduplication on the natural key, and a stated watermark lag chosen to exceed worst-case commit visibility.
Verify it yourself
- With inserts running throughout, reading every page returns each committed row at least once and never advances the cursor past a row it has not returned.
- Page cost is flat — the thousandth page costs what the first one did.
- A caller cannot edit a cursor into scanning rows outside its own advertiser scope.
Follow-up
- ivt_status on an already-synced row is later reclassified from unreviewed to invalid_sophisticated — does that row reappear in this feed, and if so, under what ordering key?
- One advertiser's line item has a ten-minute ingestion backlog — what does your as_of watermark do, and what does the caller see?
Stale index snapshots after a synchronized refresh burst
Hard · cache stampede · partial outage · fail closed
Every five minutes, at the same second, the targeting index service saturates and 10-15% of bidder processes fail to fetch the new generation. Those processes keep serving from the previous one and log nothing about it. An advertiser has reported bids placed forty minutes after they paused a line item, and an exchange has flagged a creative whose creative_approval row moved to revoked. The fleet is about 600 processes and a generation is roughly 400 MB. Give the ordered investigation and the design change, covering both the saturation and the bids that should never have been made.
Approach
- Do the arithmetic before proposing capacity, because it decides the whole shape of the answer. Six hundred processes fetching 400 MB in the same second is 240 GB, roughly 1.9 Tbit/s. No horizontal scaling of the service makes a synchronized burst of that size work, so the refresh schedule is the defect and the saturation is its symptom.
- Confirm synchronization empirically rather than assuming it: the failures should cluster modulo the refresh period. Fetch timestamps landing within a second of a wall-clock boundary mean the fleet is cron-aligned instead of independently timed, which is what turns a steady load into a thundering herd.
- Find the silent failure next, because it is the expensive half. Ask for the distribution of snapshot age across the fleet; if that metric does not exist, that is your first finding, since a process cannot fail closed on a staleness bound it does not measure. Read the fetch failure path and confirm it retains the previous generation and returns without raising.
- Separate the two defects explicitly, because they have different owners and different severities. The burst is an availability problem fixable with jitter. Serving from a forty-minute-old snapshot is a correctness problem: a paused status, an ended flight and a revoked creative_approval row all stop being honoured, and serving a revoked creative is a contractual failure rather than degraded performance. Failing open is what protected revenue during the incident and is precisely what served the revoked creative.
- Change the distribution: full jitter so each process picks a random offset within the cycle, single-flight per host so co-located processes share one fetch, and deltas instead of whole generations, sending only line items whose version or targeting_hash changed since the requester's generation. That turns 400 MB into the size of the change set and makes the steady-state load proportional to control-plane writes rather than to fleet size.
- Change the bidder: carry the generation's build timestamp inside the snapshot, publish a staleness bound such as two refresh cycles, and when the snapshot exceeds it drop candidates rather than bid. A pause is only as fast as fan-out, so the guarantee is a bound, and the bidder has to enforce the bound it advertises instead of quietly outliving it.
Follow-up
- With jitter, a process can be one cycle behind by design. What staleness do you publish to advertisers as the time for a pause to take effect, and how do you verify it?
- Deltas require the requester's current generation to be known. What happens to a process that has been down for an hour, and how do you stop that path from becoming a full-fetch stampede of its own?
- During a real index-service outage, failing closed costs all revenue on that fleet. Who decides, and what does the override look like so it cannot be left on?
Suggested practice checklist
For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.
Day 1 — Numbers before diagrams
- Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
- Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
- Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.
Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.
Day 2 — One system, from requirements to schema
- Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
- Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
- Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.
Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.
Day 3 — The consistency you are actually buying
- Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
- Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
- Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.
Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.
Day 4 — Failure is the design
- For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
- Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
- Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
- Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.
Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.
Day 5 — Scaling the hot path
- Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
- Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
- Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
- Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.
Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.
Day 6 — Keep the coding hand in, at the bar that applies to you
- Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
- Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
- Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.
Deliverable: One problem hardened to review standard, with its test list and one written review comment.
Day 7 — Defend it while being interrupted
- Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
- Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
- Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.
Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.
Bounding a vague ticket before writing code
When the requirements were thin, the interesting part is how you fenced the problem off: the assumption you wrote down, who you got to confirm it, the narrow version you shipped first so the rest stayed cheap to change. Guessing and being right is luck. Guessing in writing, where someone could correct you, is method.
Give an example of a time when you were forced to make a critical technical decision under tight deadlines with incomplete data. What was the outcome?
Medium · behavioural and engineering judgement
Approach
- Give the blast radius: what could have broken, and what you measured.
- State the situation in two sentences and spend the rest on the reasoning.
- Close with what you would do differently, concretely.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that again?
Argue against a synchronous budget reservation on the bid path
Hard · distributed systems · deadlines · availability · design review
A proposal reaches design review: make budget enforcement exact by having every bidder reserve budget from a central service before responding to a bid request. The finance stakeholder supports it because overshoot has been an audit finding twice. The exchange enforces an 80 millisecond deadline, the region runs on the order of 10^5 to 10^6 bid requests per second, and the bidders are stateless. Describe an argument like this that you made against a design. Give the reasoning, the number that carried it, what you proposed instead, and what you gave up.
Approach
- Concede first, on the record: the proposal really does deliver the tight bound, which is what was actually asked for. An argument that treats the other design as obviously wrong reads as reflexive and loses the room.
- Translate the cost into the currency of the deadline. A reservation adds one network round trip plus the reservation service's own tail latency inside an 80 millisecond budget that is already being decremented across hops. State what fraction of the budget that consumes at the 99th percentile, not the median, and say which existing step it displaces.
- Name the availability coupling explicitly: bidder availability becomes the product of its own and the reservation service's, so a dependency that was previously optional becomes mandatory on the highest-volume path in the system.
- Identify the real scaling limit, which is not aggregate throughput: reservations for one line item are a single hot key, so the binding constraint is contention on the most popular line items, and that load is heaviest exactly for the campaigns whose budget matters most.
- Point at the leak the proposal creates. Most bids lose, so most reservations are never spent and must expire, which reintroduces a timeout, a release path and a failure mode where a bidder dies holding reservations and the line item under-delivers until they lapse.
- Offer the alternative as a bound rather than a claim, with a number attached: locally applied probabilistic throttling, a shortened control period, and in-flight spend accounting, producing a stated overshoot that the finance stakeholder can accept or reject. Then say how the disagreement actually resolved and what evidence would have changed your mind.
Follow-up
- Suppose the cap is contractual rather than a target. What would you build, and would you still refuse a synchronous path for the last few percent of the budget?
- What happens to a line item when a bidder dies holding reservations, and how does the controller distinguish that from genuine under-delivery?
- How would you have proven your latency claim before the review rather than asserting it?
Own a budget overspend incident from alert to ledger correction
Medium · incident response · pacing · budget control · ledger
A line item with a 50,000 USD daily cap spent roughly 61,000 USD before anyone noticed. Bidders are stateless and apply a throttle rate the pacing controller republishes every few seconds; the real cost of a bid only appears later, when the exchange fires a billing notice carrying the clearing price. Describe an incident of this shape that you owned. State how you detected it, what you did first, how you bounded the damage while notices were still landing, how the correction reached the budget ledger, and what durable change you made afterwards.
Approach
- Open with detection and blast radius in the units the business counts: money over cap, the line items affected, and the wall-clock gap between the first over-cap charge and the alert firing. A narrative that starts with the root cause hides whether you could see the problem at all.
- Separate stopping the bleeding from fixing the cause, and be honest that pausing at the control plane does not stop spend immediately: a pause propagates to bidders on the index fan-out cycle, and impressions already won are still owed a billing notice, so spend keeps arriving after the line item is paused.
- Give the mechanism as arithmetic, not adjectives. With the throttle held constant across one control period, committed spend is about bid rate times win rate times mean clearing price times (control period plus notice lag), summed over the fleet. Point at which term was actually wrong in your incident: a stale throttle replica, a controller that was behind on billed spend, or in-flight spend the loop was not counting at all.
- Describe the correction as rows, not edits: the over-cap charges stay in budget_ledger untouched, and the make-good is a manual_credit or discrepancy_adjustment entry with a negative amount_micros, carrying reverses_ledger_id when it reverses one identified charge. Explain why the balance query still works and why the original charge must remain visible to the advertiser.
- Close with the change that made the bound stateable rather than hoped for, and the number that proves it: a shorter control period, in-flight spend estimated from won-but-unbilled impression_ids, and an alert on the rate of spend against the intended curve rather than on the cap being crossed.
Follow-up
- What should a bidder do when the pacing controller is unreachable for sixty seconds, and does your answer change between a line item on day one of its flight and one at 99% of its daily cap?
- How would you have detected this inside one control period instead of an hour, and what would that alert have cost you in false positives?
- The exchange later invalidates 4% of those impressions as invalid traffic. Does the advertiser get a refund of the overspend, the invalid traffic credit, or both, and what does the ledger look like afterwards?