Interview guides · Amazon DSP

Amazon DSP Data Scientist Interview Questions & Guide 2026

A two-week prep plan for Amazon DSP data scientist candidates: write your answer before reading the solution, revisit misses three days later.

The 60-second brief

As a Data Scientist at Amazon DSP, you sit at the intersection of large-scale advertising technology, statistical rigor, and high-impact business strategy. The Demand-Side Platform powers complex, real-time programmatic ad campaigns across massive datasets, and your core mission is to extract actionable intelligence from this flood of data to optimize bidding algorithms, maximize advertiser ROI, and drive multi-million-dollar revenue decisions. You will build predictive models, design rigorous experiments, and partner closely with product managers and software engineers to bring state-of-the-art machine learning systems into production.

Seniority shifts the scope more than the words in the title do. Earlier-career loops mostly check that you execute a well-posed analysis correctly; senior loops check that you can decide which question is worth answering and defend what you chose not to do.

Focus areas

Diagnose pacing: underdelivery, overdelivery, budget exhaustion
Reason about bid shading under first-price auctions
Design geo holdouts that survive budget interference

Overview

The scale and complexity of the Amazon DSP ecosystem make this role uniquely demanding and intellectually rewarding. You will tackle sophisticated challenges ranging from attribution modeling and user behavior prediction to automated budget allocation and ad-delivery latency optimization. Because advertising markets move at lightning speed, your insights directly shape how algorithms allocate capital in milliseconds, influencing user experience and global commerce at an unprecedented scale.

Succeeding in this environment requires a rare blend of deep technical mastery, commercial acumen, and operational resilience. You must be comfortable navigating ambiguity, communicating complex statistical findings to non-technical stakeholders, and grounding your technical decisions in customer-obsessed business outcomes. Expect a high-performance culture that values rigorous intellectual debate, rapid iteration, and strict alignment with core corporate leadership principles.

The Amazon DSP interview process

Amazon DSP candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screening Call

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Assessment Screen

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Virtual Onsite Loop

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Comparing conversion rates between users who were exposed to an ad and users who were not.

Exposure is the output of a targeting and bidding system that ranks users by predicted conversion propensity, so the exposed group has a higher baseline conversion rate before the ad renders. The measured difference therefore estimates the quality of the targeting model, not the effect of the advertising, and it is biased upward by an amount that grows as the model gets better. The valid control is the set of users the system would have shown the ad to but deliberately did not, captured through a ghost-bid or public-service-announcement holdout that records auction wins without serving the advertiser's creative. Anyone who proposes matching or covariate adjustment to fix this should be asked what unobserved feature the bidder used that the matching model does not have.

Randomising users into treatment and control while both arms draw from the same campaign budget.

The arms compete in the same auctions and against the same budget, so treatment winning more impressions directly starves control, and the measured gap includes that cannibalisation rather than only the ad effect. The test looks methodologically clean, the randomisation is genuinely valid, and the lift is still partly manufactured. This is an interference violation, not a randomisation failure, so checking balance on covariates will not catch it. The fixes are to randomise at a unit that contains the budget, such as a geographic market, or to give each arm its own budget and its own pacing, and then be explicit that you are now comparing two separately-budgeted campaigns.

Building features from data that postdates the prediction time

Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Questions & practice

Explain the fundamental mathematical differences between Pearson correlation and Spearman rank correlation coefficients.

Medium · statistics and probability

Approach

Translate the result into the decision it informs, in one plain sentence.
Quantify uncertainty explicitly rather than reporting a point estimate alone.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

Design an A/B test for a new algorithmic pricing model in the ad exchange, detailing your approach to sample size calculations and variance reduction.

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Permutation-test a market-level holdout for incremental conversions

Hard · permutation-test · incrementality · geo-experiment · statistics

geo_daily has one row per (geo_dma, date): arm ('treatment' or 'control'), conversions, conversion_value_usd, spend_usd. Forty DMAs assigned in twenty matched pairs on pre-period conversions, with fifty-six pre-period dates and twenty-eight flight dates. Control DMAs received no delivery for this advertiser and each arm carried its own budget. Without using a library hypothesis test, estimate incremental conversions per $1,000 of treatment spend with a p-value and a 95% interval. State the permutation scheme, why it matches the assignment mechanism, and the minimum effect this design can detect.

Approach

Fix the estimator before looking at the outcome. Per DMA take y as flight-period conversions and x as conversions over a matched-length pre-period window, form the adjusted outcome y - theta * x, and estimate theta from control DMAs only so it cannot absorb any part of the treatment effect. The statistic is then the mean of the paired differences of the adjusted outcome.
Permute within matched pairs, not across all forty DMAs. Assignment was pairwise, so the exact null distribution is the 2^20 sign flips of the pair differences; sample 20,000 of them. Permuting freely across DMAs tests a weaker null, reintroduces between-pair variance the matching already removed, and makes a real effect harder to detect.
Compute the p-value in the plus-one form, (1 + count of |T_perm| >= |T_obs|) / (1 + B). With B replicates a raw count of zero would report p = 0, which is not an attainable statement about a sampled reference distribution.
Build the interval by inverting the test: find the set of shifts delta for which the same permutation test applied to y minus delta times the treatment indicator is not rejected at 0.05. That interval is consistent with the p-value you just reported, whereas a bootstrap over twenty pairs is thin and can disagree with its own test.
Convert to the per-$1,000 scale by dividing the incremental-conversion estimate by treatment spend over 1,000. State the precondition that makes this a causal quantity: each arm carried its own budget, so treatment wins did not starve control. Had they shared one budget, the arms would compete in the same auctions and the difference would include cannibalisation rather than the ad effect alone.
Report power before the p-value. Compute the minimum detectable effect from the observed standard deviation of the pair differences at twenty pairs; if the MDE is larger than any effect the advertiser would act on, say the design cannot answer the question rather than presenting a non-significant result as evidence of no effect.

Worked solution (40 min)

Collapse geo_daily to one row per DMA with pre-period conversions x, flight conversions y, flight spend, and the pair id and arm; confirm every pair has exactly one treatment and one control DMA.
Estimate theta by regressing y on x within control DMAs only, form adj = y - theta * x, and compute the twenty pair differences d = adj_treatment - adj_control and the observed statistic T_obs = mean(d).
Draw 20,000 sign-flip vectors in {-1, +1}^20, compute mean(sign * d) for each, and evaluate the plus-one two-sided p-value against |T_obs|.
Invert the test over a grid of delta to get the 95% interval, then divide the point estimate and both endpoints by treatment spend over 1,000.
Compute the MDE at 80% power from sd(d) and twenty pairs, and run a pre-period placebo by splitting the fifty-six pre-dates into two 28-day halves and testing the second against the first.

Expected result: A point estimate of incremental conversions per $1,000 of treatment spend, a 95% interval from test inversion, a plus-one permutation p-value with B stated, and an explicit MDE. With twenty pairs the p-value grid is coarse: the smallest attainable two-sided p is 1/20,001 at B = 20,000, against an exact-enumeration floor of 2/2^20.

Verify it yourself

The permutation null under sign flipping is symmetric about zero, and its standard deviation is close to sd(d)/sqrt(20), the analytic standard error of the paired mean.
Re-running with theta = 0 leaves the point estimate essentially unchanged and only widens the interval. A shifted point estimate means theta was fitted using treatment data and is absorbing part of the effect.
The pre-period placebo returns no significant lift and, across repeated seeds, p-values that look roughly uniform. A significant placebo means the matching or the pre-period window is wrong, not that the ads worked.
The reported interval contains the point estimate and excludes zero exactly when the p-value is below 0.05; if the two disagree, the inversion grid is too coarse or the statistic differs between the two computations.

Follow-up

One treatment DMA holds 30% of all conversions. What does that do to the permutation distribution, and would you weight, trim, or re-pair?
The flight ends on day 28 but the click window is 7 days. What is your analysis window, and what does cutting the measurement on day 28 cost you?
How would you detect spillover between adjacent DMAs, and what would you do to the estimate if you found it?

Write a SQL query using SQL window functions to track customer purchases over consecutive months and identify churn patterns.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Aggregate campaign performance data by publisher and placement using group by and summation functions to surface top-tier inventory.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Extract user conversion funnels from raw event logs using advanced joins and conditional aggregation.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Clicks with no conversion inside the attribution window

Medium · anti-join · null semantics · attribution window

ad_click has click_id, click_ts, line_item_id, click_tracking_id TEXT (nullable), ivt_status and is_billable. conversion_event has conversion_id, conversion_ts, click_tracking_id (nullable, populated only on click-through reports) and is_duplicate. For one week of valid billable clicks, return per line_item_id the number of clicks with no non-duplicate conversion whose conversion_ts falls within 7 days after click_ts, plus that count as a share. A colleague wrote the filter as click_tracking_id NOT IN (SELECT click_tracking_id FROM conversion_event). Say exactly what that returns, why, and rewrite it correctly.

Approach

Name the NOT IN failure precisely. If the subquery yields a single NULL, then x NOT IN (...) evaluates to NULL for every x rather than TRUE, the WHERE clause admits nothing, and the query returns zero rows. It raises no error, so the output reads as 'every click converted'. Here conversion_event.click_tracking_id is nullable by design because view-through conversions carry no click, so the subquery is guaranteed to contain nulls.
Rewrite as NOT EXISTS with the correlated predicate inside: NOT EXISTS (SELECT 1 FROM conversion_event c WHERE c.click_tracking_id = k.click_tracking_id AND c.is_duplicate = FALSE AND c.conversion_ts >= k.click_ts AND c.conversion_ts < k.click_ts + interval '7 days'). NOT EXISTS tests row existence rather than value membership, so nulls on either side simply fail to match and the semantics stay correct.
Keep the window and the is_duplicate predicate inside the subquery. Moving either to an outer WHERE after a LEFT JOIN converts the anti-join back into an inner join and silently drops exactly the unmatched rows you were counting.
If you prefer the LEFT JOIN form, every conversion-side predicate must sit in the ON clause for the same reason, and the surviving filter must be c.conversion_id IS NULL.
Treat clicks with a null click_tracking_id as a third bucket. They can never match, so they are unattributable rather than unconverted. Report their share separately, because a rising null rate is an instrumentation regression, not a performance one, and folding them into the unconverted count hides that.
Note that one click can carry several conversions. The anti-join form avoids the fan-out by construction; if you instead join and subtract, you need COUNT(DISTINCT click_id) or the converted count will exceed the click count.

Worked solution (25 min)

Run the colleague's NOT IN query, observe zero rows, then run SELECT count(*) FROM conversion_event WHERE click_tracking_id IS NULL to demonstrate the cause rather than assert it.
Restrict the click side to ivt_status = 'valid' AND is_billable AND click_ts inside the week.
Classify each click in one pass into three buckets: click_tracking_id IS NULL (unattributable), satisfies the NOT EXISTS predicate (unconverted), otherwise converted.
Write the NOT EXISTS with is_duplicate = FALSE and both window bounds inside the correlated subquery, using a half-open interval so a conversion exactly 7 days later is excluded consistently.
Aggregate per line_item_id and take the unconverted share over clicks that carried a tracking id, not over all clicks.

Expected result: Per line item: valid billable clicks, unattributable clicks, unconverted clicks and the unconverted share. For example 41,200 clicks, 1,850 unattributable, 37,900 unconverted out of the remaining 39,350, a 96.3% unconverted share and an implied 3.7% post-click conversion rate.

Verify it yourself

Unconverted plus converted plus unattributable equals the total click count exactly.
Running the NOT IN version beside the NOT EXISTS version returns zero rows versus the real answer, which demonstrates the null semantics rather than relying on memory of them.
Widening the window from 7 to 28 days can only lower the unconverted count, never raise it.
The converted click count matches COUNT(DISTINCT click_id) over the inner-join form, proving the many-conversions-per-click fan-out did not leak in.

Follow-up

The unconverted share jumped 6 points on one exchange only. What do you check, in order?
Extending the window from 7 to 28 days moves this number. Which direction, and how does the size of that move differ between a considered purchase and an impulse one?
What share of conversions arrive with no click_tracking_id at all, and what does that do to every click-based denominator you report?

Share an example of how you handled a disagreement with a product manager regarding feature prioritization or metric definitions.

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Write a query to calculate rolling 30-day active advertisers and flag accounts with declining engagement trends.

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

What product metrics would you monitor to determine if a shift in ad-auction pacing algorithms harms long-term advertiser retention?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

What are the most common experimentation pitfalls you encounter when testing ad-delivery latency improvements?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

How do you handle network effects and interference between treatment and control groups when running marketplace experiments?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

How would you investigate and diagnose a sudden, unexpected 15% drop in campaign conversion rates week-over-week?

Medium

Approach

Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Measure invalid traffic when only labels are observable

Hard · measurement-quality · proxy-bias · restatement

ad_impression.ivt_status is one of valid, givt, sivt, written by a detection system and frequently rewritten days after delivery (ivt_classified_at_ts). Leadership wants a metric that answers whether invalid traffic is getting worse. True invalid traffic is unobservable; you have labels only. Deliverable: the metric, the direction and cause of its bias in one sentence, the auxiliary measurement that bounds that bias, and the evidence that would make you report that the series is not comparable across a period boundary.

Approach

Separate the estimand from the observable and keep them separate for the rest of the answer. The estimand is the share of delivered impressions that were not genuine human ad opportunities. The observable is the share a classifier labelled as invalid. They differ by the classifier's recall, which is unknown and changes with every detection release.
State the bias direction plainly and put it on the dashboard, not in an appendix: the labelled rate is at most the true rate, so the metric is biased downward, and the bias shrinks whenever detection improves. That means a detection upgrade and a genuine increase in invalid traffic produce the same upward movement in the published number, which is the entire difficulty of the question.
Split the two label families because they behave differently. General invalid traffic is list-based and settles within hours. Sophisticated invalid traffic is behavioural and is reclassified days later, so it restates numbers you already published. Measure the reclassification tail directly — the distribution of ivt_classified_at_ts minus served_ts — and freeze a period only after its 95th percentile has elapsed.
Bound the unknown recall with auxiliary measurement rather than pretending it is known. A seeded canary set of known-invalid traffic injected at a controlled rate yields a recall estimate. A reference slice of supply whose composition is held fixed separates classifier drift from supply-mix drift. An independent second vendor label on a sample yields a disagreement rate. Publish labelled rate, recall estimate and reference-slice rate together; none of the three is interpretable alone.
Define the abstention condition in advance, because the honest answer here is sometimes that you cannot tell. If a detection release landed inside the comparison window and the reference slice moved by more than the change in seeded-canary recall can account for, report the series as not comparable across that boundary instead of publishing a delta.
Prepare the restatement note as part of the metric, not as an incident response: what changed, which periods are affected, and the direction and size of the correction — a published number that moves later damages trust far less than one that moves silently.

Worked solution (45 min)

Compute the labelled rate split by givt and sivt, with the denominator as all delivered impressions including invalid ones, bucketed on served_ts.
Measure the reclassification tail from ivt_classified_at_ts minus served_ts and set the freeze lag at its 95th percentile.
Compute the labelled rate on a reference supply slice held fixed on publisher_id, ad_format and geo_country, so composition cannot move the series.
Estimate recall from the seeded canary set and express the labelled rate as a lower bound on the true rate given that recall.
Write the comparability rule and draft the restatement template.

Expected result: A metric spec: givt and sivt rates reported separately over all delivered impressions, frozen only after the measured 95th-percentile reclassification lag, published alongside a canary-derived recall estimate and a fixed-composition reference-slice rate, with a written statement that the labelled rate is a downward-biased proxy whose bias shrinks as detection improves, and a pre-defined condition under which no period-over-period delta is published at all.

Verify it yourself

Re-running a past month today yields a higher sivt rate than the number published at the time, confirming the reclassification tail is real and the freeze lag is necessary.
The fixed-composition reference slice does not move when the overall supply mix moves, which is the test that it isolates classifier behaviour.
The seeded canary recall estimate changes across a detection release, and the size of that change is compared against the movement in the published rate before any conclusion is drawn.
The denominator includes invalid impressions; excluding them makes the rate a ratio of invalid to valid rather than an invalid share.

Follow-up

The detection vendor ships a detector upgrade mid-quarter. How do you restate the numbers already published?
An advertiser demands a credit citing their own vendor's higher invalid rate. What is your response, and what would change it?
How does this bias interact with the viewability measurement rate on the same impressions?

Bid raised fifteen percent, pooled win rate went down

Medium · win rate · simpsons paradox · auction mechanics

A line item's bid_value_usd was raised 15% at 09:00 UTC. Over the following week its pooled auction win rate — bid_request_log rows with win_flag=TRUE over rows with response_status='bid' — fell from 12.4% to 11.1%, while impressions won and spend both rose. The trading team wants to roll the change back. You have bid_request_log with exchange_id, publisher_id, ad_format, auction_type, floor_price_cpm_usd, no_bid_reason, bid_cpm_usd, clearing_price_cpm_usd. Explain the direction of the pooled rate and recommend keep or roll back.

Approach

Notice the internal contradiction before touching data: more wins and more spend with a lower win rate means the bid denominator grew faster than wins. The question is therefore which bids are new, not whether the bid worked.
Stratify on the variable the bid change directly acts on — floor_price_cpm_usd bucket, crossed with auction_type — and compute per-stratum win rate and bid share before and after. A higher bid converts no_bid_reason='below_floor' rows into bids in exactly the high-floor, low-win-rate strata.
Run a Kitagawa decomposition of the pooled change into a within-stratum component (Σ share_post × Δrate) and a mix component (Σ Δshare × rate_pre), and report both numbers. Pooled direction is not evidence about either one on its own.
Confirm the mechanism in the no-bid ledger: the fall in below_floor no-bids should approximately equal the growth in bid rows in the high-floor strata. If it does not, something else expanded eligibility, such as a pacing_throttle release.
Judge the change on economics, not on the rate. Under first_price clearing the bid increase also raises clearing_price_cpm_usd on inframarginal wins, so compare incremental spend against incremental conversions rather than counting impressions.

Follow-up

Under first-price clearing, how much of the extra spend went to impressions the line item would have won at the old bid?
What would you expect the same decomposition to look like under second_price, and why is the inframarginal cost argument weaker there?
The pooled rate is a dashboard tile. What would you replace it with so this cannot recur?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Getting an analysis acted on

Work that nobody used is a common and unflattering pattern in data careers, and interviewers probe for it. Have a story about an analysis that changed a decision, and be specific about how you got it in front of the person who could act. Also have one about work that went nowhere, with your reading of why.

Give an example of a time you uncovered an unexpected anomaly in production data and drove the resolution from end to end.

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

Claiming impact for three projects with honest counterfactuals

Easy · self-assessment · impact · measurement

You are writing a performance self-review. Over two quarters you shipped a bid-shading recalibration, a rewrite of the click-through-rate query that fixed an invalid-traffic filter applied to only one side of the ratio, and an incrementality test that led to a line item being paused. Deliverable: for each, the number you would claim, the counterfactual you claim it against, and the portion of credit that belongs to someone else. One of the three has no defensible dollar figure. Say which, and what you claim instead.

Approach

The interviewer is probing whether you apply the same causal standard to your own work that you apply to an advertiser's, so write the counterfactual for each claim before the number: what would the metric have been over the same period without this work?
For the shading recalibration, compare clearing price and profit on won impressions at matched supply mix across exchange_id, ad_format, device_type and geo_country, and report the profit change rather than the win-rate change; under first-price clearing a higher win rate can accompany worse profit.
For the query fix, claim the magnitude of the bias removed rather than a revenue number: the gap between the old rate, which filtered invalid traffic on the numerator only, and the corrected symmetric rate, plus the decision that was being made on the wrong figure.
For the incrementality test, claim the decision, not the savings. The paused budget's fate was never measured, so any dollar figure would assume that money vanished rather than moved. This is the item with no defensible number.
Name specific collaborators and what they contributed, and pre-empt the obvious challenge by stating the confound you know about, such as seasonality or supply mix drift, and the bound you put on it.

Follow-up

Your manager points out the paused line item's spend simply moved to another line item. Does your claimed impact survive?
How do you claim credit for work whose value is a bad decision that did not happen?

Disclosing an invalid-traffic reclassification that changes invoiced numbers

Hard · restatement · data-integrity · ethics · communication

A measurement vendor reclassifies sophisticated invalid traffic for a six-week period. On re-read, 4.1% of one advertiser's delivered impressions flip from valid to sivt, and their reported CPA rises about 6% across two already-invoiced months. You did not cause it, the contract's make-good language is ambiguous, and a colleague suggests waiting to see whether the advertiser notices. Deliverable: the note you send the advertiser, what you say to finance and the account team beforehand, and the position you take internally on waiting.

Approach

The interviewer is probing whether you will bound an uncomfortable number before negotiating about it, so recompute the six weeks with the restated ivt_status first, splitting givt from sivt, and produce the impact by campaign and by invoiced month: impressions flipped, billed amount on those exact rows, and the CPA delta.
Sequence disclosure so one number exists before anyone speaks externally: finance first, because they own the credit and the invoices; the account team second, because they own the relationship; the advertiser third, with the figure both internal parties have already approved.
Write the note in the restatement shape rather than as an apology: what was reclassified and by whom, which dates, the direction and size, the billing position, and what the advertiser needs to do, which is nothing.
Take the position on waiting explicitly and in commercial terms, not only ethical ones. The reclassification is visible in the vendor's own dashboard, so waiting converts a correction the advertiser receives from you into a discrepancy they discover themselves, which is the version that costs the renewal.
Close it structurally: treat invalid-traffic rates as restatable on read with a published reclassification lag, and subscribe to the vendor's reclassification feed so the next event is detected rather than discovered.

Follow-up

The account team says this disclosure will cost us the renewal. Does that change what you send?
The contract is silent on reclassification make-goods. How would you price one, and what precedent are you setting?
What do you do if finance instructs you not to send the note?