Interview guides · Integral Ad Science

Integral Ad Science Data Scientist Interview Questions & Guide 2026

SQL practice for Integral Ad Science data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.

The 60-second brief

As a Data Scientist at Integral Ad Science (IAS), you will play a pivotal role in shaping the future of digital ad verification, brand safety, and viewability. IAS is a global leader in digital media quality, processing trillions of data points daily to ensure that digital ads are shown to real people, in safe environments, and in the right contexts. Your work will directly impact how major advertisers and publishers optimize their digital campaigns, protecting billions of dollars in ad spend from fraud and low-quality placements.

Scope your preparation by the data you would actually touch, because the title will not tell you. A seat that lives in event logs and weekly readouts rewards fluency in aggregation and metric definitions; a seat that owns a model in production rewards fluency in train/serve skew, retraining cadence and drift monitoring. The fastest way to find out which one you are interviewing for is to ask what the team shipped last quarter and what it gets paged about.

Focus areas

Reconstruct the funnel from bid request to conversion
Reason about bid shading under first-price auctions
Design geo holdouts that survive budget interference

Overview

You will join a highly sophisticated technical team that sits at the intersection of big data, machine learning, and real-time bidding (RTB). The algorithms and models you build will be deployed at an immense scale, requiring not just theoretical machine learning knowledge, but also a deep understanding of software engineering best practices. Whether you are working on natural language processing (NLP) for advanced contextual targeting or computer vision for video ad verification, your contributions will be highly visible and strategically vital to the company’s core product offerings.

This role is both intellectually challenging and highly rewarding. The sheer volume of data and the adversarial nature of ad fraud mean you will constantly face novel, complex problems. Success in this position requires a blend of rigorous scientific thinking, practical coding skills, and the ability to translate complex technical findings into actionable business strategies for both internal stakeholders and external clients.

The Integral Ad Science interview process

Integral Ad Science candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Phone Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Screen

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Take-Home Assignment

The clock is part of the test. Three to six hours is not enough to do everything the dataset supports, so the submission mostly reveals how you spend a fixed budget against an open question. A reviewer sees which paths you took and, by absence, which you abandoned. Work that runs out of time inside the analysis ships a thin conclusion, while work that cuts scope early protects the last hour for writing. The most reliable way to lose here is to leave the scoping decision implicit, so it reads as something you missed rather than something you chose.

What to demonstrate

Whether the scope you settled on is presented as a decision with a reason, rather than left for the reader to infer from what is missing
Whether the depth of the work is consistent with the stated time budget, instead of several half-finished directions left open
Whether the closing section reads as something written on purpose rather than assembled from whichever cells survived

How to prepare

Run a timed rehearsal on a public dataset with a hard stop, holding the final sixty minutes for writing no matter where the analysis has got to
Before opening the data, list the questions it could plausibly answer, pick one, and keep the discarded ones as a short note on what you did not attempt and why
Commit a one-line finding after each analysis step so the writeup is assembled from recorded results rather than from memory at midnight

Panel Interview

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Technical and Behavioral Interviews

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Randomising users into treatment and control while both arms draw from the same campaign budget.

The arms compete in the same auctions and against the same budget, so treatment winning more impressions directly starves control, and the measured gap includes that cannibalisation rather than only the ad effect. The test looks methodologically clean, the randomisation is genuinely valid, and the lift is still partly manufactured. This is an interference violation, not a randomisation failure, so checking balance on covariates will not catch it. The fixes are to randomise at a unit that contains the budget, such as a geographic market, or to give each arm its own budget and its own pacing, and then be explicit that you are now comparing two separately-budgeted campaigns.

Reallocating budget using last-click attribution.

Last-click assigns full credit to whichever touchpoint sits closest to the conversion in time, which structurally favours channels that harvest existing demand — retargeting an already-interested user, or catching a branded search — over channels that create demand in the first place. Optimising against it therefore moves money toward tactics that would have converted many of those users anyway, and the reported cost per action improves at the exact moment true incremental performance gets worse. The signature of this failure is a portfolio where every channel's attributed conversions sum to well above the advertiser's total conversion count. The counter is to treat attributed numbers as a budget-splitting convention and to source the actual reallocation decision from holdout-based incrementality.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Answering a product-sense question with a list of features

Answer with a decision and the measurement that would settle it: the hypothesis, the primary metric, the guardrails, and the result that would make you not ship. A feature brainstorm cannot be wrong, which is exactly why it earns no points.

Questions & practice

What is the difference between Bayesian and Frequentist statistics, and how does it apply to ad testing?

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Translate the result into the decision it informs, in one plain sentence.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

Explain the concept of p-value to a non-technical product manager.

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Translate the result into the decision it informs, in one plain sentence.
Say what the estimate is of, and over what population it generalises.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

Explain the trade-off between precision and recall, and how you would optimize this for a brand safety classifier.

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

Point-in-time join of impressions to versioned line items

Medium · scd2 · point-in-time-join · numpy · pandas

impressions has 20 million rows: impression_id, served_ts, line_item_id, billed_price_micros_usd, is_billable. line_item is SCD type 2 with 400,000 versions: line_item_id, bid_strategy, bid_value_usd, daily_budget_usd, pacing_mode, status, valid_from_ts, valid_to_ts (null on the current row), is_current. A version covers the half-open interval from valid_from_ts to coalesce(valid_to_ts, infinity). Attach to every impression the attributes of the version in force at its served_ts. You may not use merge_asof and you may not materialise the impression-by-version cross product. Return the joined frame and a count of impressions no version covers.

Approach

Validate the dimension before joining: per line_item_id, sort by valid_from_ts and assert the intervals are non-overlapping, and assert at most one is_current row per key. Overlapping versions make the as-of answer ambiguous and are the usual root cause behind duplicated-revenue reports, so finding them is part of the job.
Encode the composite as-of key into one sortable integer so a single np.searchsorted does the work: factorise line_item_id to codes shared across both frames, dense-rank all timestamps from both frames into a small integer, then form code * (n_ranks + 1) + rank. That keeps the composite inside int64 and makes lexicographic order on (key, time) equal to numeric order.
Sort the dimension by that composite, run np.searchsorted(dim_keys, imp_keys, side='right') - 1, and you have a candidate version index for every impression in one vectorised call rather than 400,000 per-group searches.
Reject candidates that do not actually cover the impression: the candidate must belong to the same line_item_id, and served_ts must be less than coalesce(valid_to_ts, far future). Report the uncovered count instead of dropping those rows; impressions before a key's first version normally indicate a backfill gap in the dimension, not bad events.
Materialise attributes with a positional .take on the matched index rather than a merge. One row in, one row out by construction, so fan-out is impossible regardless of what the dimension contains.

Worked solution (30 min)

Run the dimension validation and collect two lists: keys with overlapping intervals and keys with more than one is_current row.
codes, uniques = pd.factorize(pd.concat([...])) applied consistently to both frames; dense-rank the union of served_ts and valid_from_ts; build the int64 composites.
Sort the dimension by composite, np.searchsorted(side='right') - 1 for the impressions, and mask positions of -1 as uncovered.
Validate coverage vectorised: same code, and served_ts < coalesce(valid_to_ts, Timestamp.max); then .take the attribute columns at the matched positions and assemble the output.

Expected result: A frame with exactly len(impressions) rows carrying one attribute set each, an integer count of uncovered impressions, and the two dimension-integrity lists (overlapping keys, duplicate-current keys).

Verify it yourself

Output row count equals input row count exactly; any increase is fan-out, any decrease is a silently dropped uncovered row.
Asserted vectorised over the whole output: valid_from_ts <= served_ts < coalesce(valid_to_ts, Timestamp.max) on every covered row.
On a hand-built case with three versions and an impression exactly at a boundary timestamp, the impression at served_ts == valid_from_ts attaches to the newer version, matching the half-open interval.
On a 10,000-row sample, results agree row for row with a deliberately slow reference implementation that loops each impression over its key's versions. That brute-force check is what proves the composite-key encoding is sound.

Follow-up

Two versions overlap by one second because the writer used an inclusive end timestamp. What does your join return, and how would you make that fail loudly instead of quietly picking one?
The pacing report needs the budget in force at every hour boundary, not just at impression time. What changes in the construction?
Estimate the memory and time cost of the naive merge-on-key-then-filter at these row counts, and say where it breaks first.

Explain the difference between generators and lists in Python, and when you would use each.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Implement a custom function to calculate the rolling average of a stream of ad impression data.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Write a Python function to find the first non-repeating character in a string and analyze its time complexity.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Sessionise device bid requests with a thirty-minute inactivity gap

Medium · window functions · gaps-and-islands · null handling

bid_request_log holds one row per (auction_id, line_item_id) with auction_id, request_ts TIMESTAMPTZ, device_id TEXT (nullable), device_type and exchange_id. For a single UTC day, assign each distinct auction opportunity a session number per device, starting a new session whenever more than 30 minutes have elapsed since that device's previous request. Return device_id, session_number, session_start_ts, session_end_ts, request count and distinct exchange count. A large share of rows carry no device_id; state how you treat them and why that choice is not arbitrary.

Approach

Collapse to auction grain first with SELECT DISTINCT auction_id, device_id, request_ts. The table repeats each opportunity once per line item considered, so sessionising it raw counts a single request as many as there are eligible line items, and the multiplier varies by targeting breadth.
Exclude device_id IS NULL rather than partitioning on it. The null share is a property of the supply source, not noise, so one null bucket becomes a synthetic device with millions of requests and no gaps. Report the excluded share as its own number in the same output.
Get the previous request per device with LAG(request_ts) OVER (PARTITION BY device_id ORDER BY request_ts), then flag a boundary when the lag is null or the gap exceeds interval '30 minutes'.
Turn the boundary flags into session numbers with a running SUM over the identical window. This is the gaps-and-islands pattern: the flag must be an integer 0 or 1, and the SUM must repeat the same PARTITION BY and ORDER BY, or the islands will not line up with the boundaries.
Specify the frame explicitly as ROWS UNBOUNDED PRECEDING. The default frame for an ordered aggregate is RANGE UNBOUNDED PRECEDING, which gives every row sharing a timestamp the same running total, so simultaneous requests from one device collapse into one session number.
Group by (device_id, session_number) and take MIN and MAX of request_ts for the bounds, COUNT(*) for volume and COUNT(DISTINCT exchange_id) for breadth.

Worked solution (25 min)

Build ev AS (SELECT DISTINCT auction_id, device_id, request_ts FROM bid_request_log WHERE request_ts in the UTC day) and compare its row count with the raw count to see the per-auction multiplier.
Split off device_id IS NULL, count it, and record its share of ev; run everything below on the non-null remainder.
Add prev_ts := LAG(request_ts) OVER (PARTITION BY device_id ORDER BY request_ts) and new_session := CASE WHEN prev_ts IS NULL OR request_ts - prev_ts > interval '30 minutes' THEN 1 ELSE 0 END.
Add session_number := SUM(new_session) OVER (PARTITION BY device_id ORDER BY request_ts ROWS UNBOUNDED PRECEDING).
Group by (device_id, session_number) with MIN(request_ts), MAX(request_ts), COUNT(*) and COUNT(DISTINCT exchange_id).
Pick the device with the highest session count, list its raw timestamps, and read the gaps by hand against the output.

Expected result: One row per (device_id, session_number). A device with requests at 09:00, 09:05, 09:50 and 09:52 yields session 1 spanning 09:00 to 09:05 with 2 requests and session 2 spanning 09:50 to 09:52 with 2 requests, because the 45-minute gap exceeds the threshold.

Verify it yourself

The sum of per-session request counts equals the count of distinct auctions with a non-null device_id for that day.
Within each device, session_number starts at 1 and increases by exactly 1, and session_start_ts of session n+1 is strictly greater than session_end_ts of session n.
Re-running LAG inside each session shows no internal gap above 30 minutes.
Raising the threshold to 60 minutes reduces the session count and never increases it.

Follow-up

A device shows 400 requests in 12 minutes across 30 exchanges. Is that a session or an invalid-traffic signature, and what would you look at next?
The null device_id rate differs sharply between mobile_app and mobile_web supply. What does that asymmetry do to any per-device metric computed on this table?
Where does 30 minutes come from? What in the inter-request gap distribution would justify a different threshold?

Walk us through your approach to the take-home data analysis assignment. Why did you choose that specific modeling strategy?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

If our ad fraud detection rate suddenly dropped by 10% overnight, how would you investigate the root cause?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

How would you define and measure "viewability" for a new, non-standard video ad format?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Measure invalid traffic when only labels are observable

Hard · measurement-quality · proxy-bias · restatement

ad_impression.ivt_status is one of valid, givt, sivt, written by a detection system and frequently rewritten days after delivery (ivt_classified_at_ts). Leadership wants a metric that answers whether invalid traffic is getting worse. True invalid traffic is unobservable; you have labels only. Deliverable: the metric, the direction and cause of its bias in one sentence, the auxiliary measurement that bounds that bias, and the evidence that would make you report that the series is not comparable across a period boundary.

Approach

Separate the estimand from the observable and keep them separate for the rest of the answer. The estimand is the share of delivered impressions that were not genuine human ad opportunities. The observable is the share a classifier labelled as invalid. They differ by the classifier's recall, which is unknown and changes with every detection release.
State the bias direction plainly and put it on the dashboard, not in an appendix: the labelled rate is at most the true rate, so the metric is biased downward, and the bias shrinks whenever detection improves. That means a detection upgrade and a genuine increase in invalid traffic produce the same upward movement in the published number, which is the entire difficulty of the question.
Split the two label families because they behave differently. General invalid traffic is list-based and settles within hours. Sophisticated invalid traffic is behavioural and is reclassified days later, so it restates numbers you already published. Measure the reclassification tail directly — the distribution of ivt_classified_at_ts minus served_ts — and freeze a period only after its 95th percentile has elapsed.
Bound the unknown recall with auxiliary measurement rather than pretending it is known. A seeded canary set of known-invalid traffic injected at a controlled rate yields a recall estimate. A reference slice of supply whose composition is held fixed separates classifier drift from supply-mix drift. An independent second vendor label on a sample yields a disagreement rate. Publish labelled rate, recall estimate and reference-slice rate together; none of the three is interpretable alone.
Define the abstention condition in advance, because the honest answer here is sometimes that you cannot tell. If a detection release landed inside the comparison window and the reference slice moved by more than the change in seeded-canary recall can account for, report the series as not comparable across that boundary instead of publishing a delta.
Prepare the restatement note as part of the metric, not as an incident response: what changed, which periods are affected, and the direction and size of the correction — a published number that moves later damages trust far less than one that moves silently.

Worked solution (45 min)

Compute the labelled rate split by givt and sivt, with the denominator as all delivered impressions including invalid ones, bucketed on served_ts.
Measure the reclassification tail from ivt_classified_at_ts minus served_ts and set the freeze lag at its 95th percentile.
Compute the labelled rate on a reference supply slice held fixed on publisher_id, ad_format and geo_country, so composition cannot move the series.
Estimate recall from the seeded canary set and express the labelled rate as a lower bound on the true rate given that recall.
Write the comparability rule and draft the restatement template.

Expected result: A metric spec: givt and sivt rates reported separately over all delivered impressions, frozen only after the measured 95th-percentile reclassification lag, published alongside a canary-derived recall estimate and a fixed-composition reference-slice rate, with a written statement that the labelled rate is a downward-biased proxy whose bias shrinks as detection improves, and a pre-defined condition under which no period-over-period delta is published at all.

Verify it yourself

Re-running a past month today yields a higher sivt rate than the number published at the time, confirming the reclassification tail is real and the freeze lag is necessary.
The fixed-composition reference slice does not move when the overall supply mix moves, which is the test that it isolates classifier behaviour.
The seeded canary recall estimate changes across a detection release, and the size of that change is compared against the movement in the published rate before any conclusion is drawn.
The denominator includes invalid impressions; excluding them makes the rate a ratio of invalid to valid rather than an invalid share.

Follow-up

The detection vendor ships a detector upgrade mid-quarter. How do you restate the numbers already published?
An advertiser demands a credit citing their own vendor's higher invalid rate. What is your response, and what would change it?
How does this bias interact with the viewability measurement rate on the same impressions?

Weekly eCPM lift that survives no calendar alignment

Medium · seasonality · ecpm · mix shift

Supply leadership reports eCPM up 6.8% week over week and wants to credit a floor-price change shipped ten days ago. eCPM is SUM(billed_price_micros_usd)/1e6 over billable ad_impression rows, divided by their count over 1000, bucketed on served_ts date UTC. The comparison is the trailing 7 days against the prior 7 days; one of those windows contains a public holiday. You also have ad_format and device_type via bid_request_log on auction_id. Decide whether the lift is real and state the size of the real component.

Approach

Align the calendar before comparing anything. Rebuild both windows so they contain the same count of each weekday, and either exclude the holiday or compare it against the same holiday in a prior year rather than against an ordinary weekday.
Plot a 28-day trailing series of daily eCPM and check whether there is a level shift at the ship date at all. A change that is only visible in a two-point comparison usually is not there.
Decompose eCPM the same way as any weighted mean: within-stratum price change versus mix across ad_format and device_type. ctv and instream_video carry much higher prices, so a weekend-heavy window inflates eCPM with no price change anywhere.
Report impression count next to eCPM. A floor raise removes low-priced inventory, so eCPM rises mechanically while revenue can fall; eCPM alone cannot tell you which happened.
Judge the floor change on eligible RPM — viewability-adjusted, invalid-traffic-filtered revenue per thousand eligible bid requests — because that denominator keeps the unsold opportunity the floor created in scope.

Follow-up

The floor change did raise eCPM and cut fill. What number decides whether to keep it, and what would make you reverse it?
How would you build a day-of-week adjustment that survives a moving holiday like Eid or Thanksgiving?
If ctv share is genuinely growing week on week, is the eCPM trend real or not — and does that distinction matter to the decision?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
On the third attempt, write the grain of every CTE as a comment before writing its body.
Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
Re-read only your own notes from this week, and open no new material.
Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

Defending a null incrementality result against a headline ROAS

Hard · influence · incrementality · attribution · conflict

Your geo holdout shows a retargeting line item's incremental conversions per $1,000 of spend are indistinguishable from zero. The same line item reports last-click ROAS of 8.3 and is the headline number in a renewal deck presented in six days. The account team believes your test is wrong; the advertiser has never questioned the 8.3. Deliverable: the internal case you make, the specific evidence you bring, the limits you concede, and the smallest reversible experiment you propose if your reading is rejected.

Approach

The interviewer is probing whether you can hold an unpopular finding without turning it into a credibility fight, so open by granting that both numbers are computed correctly: last-click measures which touchpoint was nearest in time, the holdout measures what changed because of the spend. Framing attribution as 'wrong' converts a methodological point into a turf argument you will lose on relationship grounds.
State the null as a bound rather than as zero. Report the minimum detectable effect the 20-market design could resolve and say 'we can rule out incremental ROAS above this value'. A null without an MDE is not a finding and is trivially dismissed as an underpowered test.
Bring converging evidence from tables you already have, and say what each piece can and cannot show. Credit sensitivity: recompute the same conversion cohort under a second attribution_model, first-touch or even credit across in-window touchpoints, holding cohort and windows fixed, and report what share of the line item's credited volume survives; that bounds how much of the 8.3 is a property of the crediting rule rather than of the spend, and it is a sensitivity result, not a lift estimate. Harvesting signature: the share of retargeting-credited conversions whose user already had an earlier impression, click or site session inside click_window_days, together with the distribution of conversion_ts minus click_ts, where a heavy mass inside a few minutes is consistent with an ad served into a session that was already converting.
Quantify both error directions before recommending anything: the spend at risk if you are right and ignored, and the renewal revenue at risk if you are wrong and act. This is what separates a defensible position from an ideological one.
Propose the reversible version rather than the shutdown: a 20% budget reduction in half the matched markets for six weeks, powered against the MDE you just computed, with the analysis window extended past the flight by the click window plus ingestion lag.

Follow-up

The account lead says telling the advertiser their favourite channel does nothing will cost us the renewal. How do you respond without either caving or escalating?
What specific evidence would change your mind about this line item?
How do you handle it if the budget-down test also comes back underpowered?

Claiming impact for three projects with honest counterfactuals

Easy · self-assessment · impact · measurement

You are writing a performance self-review. Over two quarters you shipped a bid-shading recalibration, a rewrite of the click-through-rate query that fixed an invalid-traffic filter applied to only one side of the ratio, and an incrementality test that led to a line item being paused. Deliverable: for each, the number you would claim, the counterfactual you claim it against, and the portion of credit that belongs to someone else. One of the three has no defensible dollar figure. Say which, and what you claim instead.

Approach

The interviewer is probing whether you apply the same causal standard to your own work that you apply to an advertiser's, so write the counterfactual for each claim before the number: what would the metric have been over the same period without this work?
For the shading recalibration, compare clearing price and profit on won impressions at matched supply mix across exchange_id, ad_format, device_type and geo_country, and report the profit change rather than the win-rate change; under first-price clearing a higher win rate can accompany worse profit.
For the query fix, claim the magnitude of the bias removed rather than a revenue number: the gap between the old rate, which filtered invalid traffic on the numerator only, and the corrected symmetric rate, plus the decision that was being made on the wrong figure.
For the incrementality test, claim the decision, not the savings. The paused budget's fate was never measured, so any dollar figure would assume that money vanished rather than moved. This is the item with no defensible number.
Name specific collaborators and what they contributed, and pre-empt the obvious challenge by stating the confound you know about, such as seasonality or supply mix drift, and the bound you put on it.

Follow-up

Your manager points out the paused line item's spend simply moved to another line item. Does your claimed impact survive?
How do you claim credit for work whose value is a bad decision that did not happen?

Ranking three teams' requests against one sprint of capacity

Medium · prioritisation · stakeholder-management · delivery

You have three requests and capacity for roughly one and a half. The supply team wants eligible-RPM impact sizing for a floor-price change shipping in two weeks. An advertiser team wants a pacing diagnosis for a $400k line item delivering at 60% of budget. The invalid-traffic team wants help validating a new sophisticated-invalid-traffic classifier before it restates last quarter. Deliverable: your ranking with the reason for each, exactly what you say to the two teams you deprioritise, and the trigger that would reorder the list.

Approach

The interviewer is probing whether you rank on decision structure or on social pressure, so attach two attributes to each request before ranking: the date the decision is actually made, and how expensive it is to reverse afterwards. Seniority of the requester is not one of the inputs.
Look for the request that can be collapsed rather than sequenced. The pacing diagnosis is usually a single grouped query over bid_request_log by no_bid_reason and response_status per hour for that line_item_id; time-boxed to two hours it stops competing for the sprint at all.
Size the other two in revenue at risk per week so the ranking is arguable rather than asserted: the floor change's expected eligible-RPM delta against the spend it touches, and the invoiced spend inside the period the classifier would restate.
Rank on irreversibility where deadlines are close together. A classifier that restates already-invoiced periods is the hardest of the three to unwind once shipped, which can outrank a nearer deadline.
Deprioritise with a date and a fallback rather than an apology: name what you will not do, when you could, the smallest thing you can give them now, and who else could take it.

Follow-up

The underdelivering advertiser escalates to your director. What actually changes in your plan, and what do you refuse to change?
How do you stop the pacing question arriving fresh every sprint?