Interview guides · Depop

Depop Data Scientist Interview Questions & Guide 2026

SQL practice for Depop data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.

The 60-second brief

As a Data Scientist at Depop, you are at the intersection of community, fashion, and high-scale marketplace dynamics. Your work directly influences how millions of users discover unique items, engage with sellers, and experience the platform. You are not just crunching numbers; you are a product partner tasked with turning raw behavioral data into actionable insights that shape the future of the Depop ecosystem.

When randomisation is off the table, the skill being checked is naming an identification strategy together with the assumption it rests on: parallel trends for difference-in-differences, relevance and exclusion for an instrument, overlap and conditional ignorability for matching. Say the assumption out loud and say how you would try to break it.

Focus areas

Query request, match and payout grains without fan-out
Diagnose whether a market is supply- or demand-constrained
Price incentives against contribution margin, not gross bookings

Overview

This role is inherently product-focused. You will likely contribute to critical domains such as search and recommendation algorithms, marketing attribution, spam detection, and overall platform health. You will balance the need for rigorous experimentation with the fast-paced, creative environment of a global resale marketplace. Success here requires a blend of technical precision, business intuition, and the ability to articulate complex findings to non-technical stakeholders.

You should expect to operate in a high-impact environment where your analyses determine whether a new feature drives genuine growth or creates unintended friction. While the work is intellectually stimulating and highly visible, it demands a candidate who is comfortable with ambiguity and thrives when translating open-ended business questions into structured, data-driven solutions.

The Depop interview process

Depop candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Talent Partner Screening

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Technical Discussion

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Take-Home Task

A take-home is graded as an argument, not as a notebook. Somebody reads the submission without you in the room, so every choice has to survive on the page: why the question was framed this way, and what was deliberately left out. The gap between a strong and a weak submission is almost never model quality. It is whether the writeup names the specific question it answers and commits to a recommendation, including what evidence would overturn it. A high-accuracy model attached to no conclusion reads as effort that stopped before the decision.

What to demonstrate

Whether the question you answered is stated outright, and whether it is the question the prompt posed rather than an easier neighbour of it
Whether the recommendation is specific enough to act on, with the uncertainty attached to it instead of parked in a caveats section at the end
Whether analytical choices such as the metric definition, the population filter and the time window are justified in the prose, not merely visible in code

How to prepare

Take a dataset you have already worked with, write the one-paragraph conclusion first, then check whether the analysis you were planning actually supports it and cut whatever does not
Practise stating a metric in one sentence that fixes the population, the time window and the denominator, then confirm your query computes exactly that sentence and nothing adjacent to it
Hand a draft to someone outside the problem and ask them to tell you back what you recommended and why; anything they cannot recover is not on the page yet

Live Technical Challenge

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Final Round

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Conditioning the analysis on completed orders

Wait-time distributions, price elasticities and rating models fit only on completed orders are conditioned on an outcome that the intervention itself changes. The requests that never matched, or that the consumer abandoned, are the population a liquidity fix targets, so excluding them biases every estimate toward the status quo and can flip the sign of a price elasticity. Any query starting FROM fct_order is already inside this trap; start from fct_request and left join.

Pooling across markets with a shifting mix

Aggregate take rate, order frequency and fill rate all move when the market mix moves, because a fast-growing newly launched market has lower prices, lower frequency and different supply density. A company-level metric can decline while every individual market improves, which is Simpson's paradox with a launch calendar attached. Report within-market deltas, or fix the mix by weighting to a reference period, and say which one you did.

Comparing periods without accounting for seasonality or day-of-week

Compare whole weeks against whole weeks and check whether the same swing appeared in prior cycles or prior years before attributing it to anything you changed. Weekday and weekend populations often differ enough that a Tuesday-to-Saturday comparison is meaningless.

Analysing at a different unit than the one randomised

Say out loud what was randomised (user, device, account, cluster) and make the analysis unit match, or account for the clustering with cluster-robust standard errors, the delta method, or aggregation up to the randomised unit. Randomising users and then running a test over sessions understates variance and inflates the false-positive rate.

Questions & practice

Implement SLA fill rate from its written definition

Easy · metric definition · ratio of sums · groupby · timezones

You are given requests (request_id, market_id, requested_at_utc, requested_at_local, matched_at_utc which is NaT when the request never matched, terminal_at_utc, request_status) and markets (market_id, timezone, sla_seconds). Implement fill_rate(requests, markets) returning the market-day SLA fill rate. Numerator: requests where matched_at_utc minus requested_at_utc is at most that market's sla_seconds. Denominator: all requests created in the window except those with request_status 'abandoned_pre_match' that terminated within 10 seconds of creation. Compute at market-hour grain first, then roll up to market-day. Return numerator and denominator counts alongside the rate.

Approach

Merge sla_seconds onto requests by market_id first, so the SLA comparison is a vectorised column operation rather than a per-market branch.
Bucket on requested_at_local, not requested_at_utc: a market-day is a local-calendar object, and a pandas datetime64 column can only carry one timezone, which is exactly why the local column exists in the source table.
Build two integer columns on the full frame, eligible (1 minus the 10-second abandon exclusion) and matched_in_sla, instead of filtering rows away; the exclusion then stays auditable and both counts share one denominator.
Aggregate with groupby([market_id, local_hour]).sum() on those two counts, then roll to market-day by summing the counts again and dividing once at the end.
Return eligible and matched_in_sla next to the rate so a downstream consumer physically cannot re-average the rate across markets or hours.

Worked solution (25 min)

Merge markets onto requests (validate='m:1'), then derive time_to_match_seconds = (matched_at_utc - requested_at_utc).dt.total_seconds(), leaving NaN where matched_at_utc is NaT.
Set excluded = (request_status == 'abandoned_pre_match') & ((terminal_at_utc - requested_at_utc).dt.total_seconds() < 10); set eligible = (~excluded).astype(int).
Set matched_in_sla = (time_to_match_seconds.notna() & (time_to_match_seconds <= sla_seconds) & eligible.astype(bool)).astype(int).
Group by market_id and requested_at_local.dt.floor('h'), summing eligible and matched_in_sla; this is the market-hour table.
Roll to day by grouping the market-hour table on market_id and the local date, summing both counts, then dividing to get fill_rate.

Expected result: A frame keyed by (market_id, local_date) with eligible, matched_in_sla and fill_rate = matched_in_sla / eligible computed once at the day grain; day counts equal the exact sum of their hourly counts, and fill_rate is NaN, not 0, where eligible is 0.

Verify it yourself

Sum of hourly eligible per market-day equals the day-level eligible exactly.
Print both the ratio-of-sums day rate and the mean of the hourly rates; they differ whenever hourly volume varies, and you should be able to say which is correct and by how much they diverge.
eligible plus excluded equals the total request count in the window, so nothing was dropped silently.
Bucketing on UTC instead of local shifts day boundaries by the market's offset; run it both ways on one non-UTC market and confirm the totals move.

Follow-up

The 10-second exclusion removes 3% of requests in one market and 11% in another. What would you check before trusting either market's fill rate?
A request can be re-offered after a provider declines. Does matched_at_utc still define your numerator, and what would you add to separate a dispatch problem from a supply problem?

Sessionise a provider heartbeat stream into supply sessions

Hard · sessionisation · event streams · cumsum keys · interval arithmetic

You are given pings: provider_id, market_id, event_at_utc, status in ('idle','en_route','engaged','offline'), one row per app heartbeat, nominally every 30 seconds but with gaps. Reconstruct the supply-session table. A session starts at the first non-offline ping and ends at an explicit offline ping, at a market change, or when the gap to the next ping exceeds 10 minutes. Produce session_id, provider_id, market_id, online_at, offline_at, online_seconds, engaged_seconds, en_route_seconds, idle_seconds and end_reason, with the three state components summing to online_seconds exactly in integer seconds.

Approach

Sort by (provider_id, event_at_utc), then build a new_session boolean: first ping of a provider, previous status equals 'offline', market_id changed, or the gap from the previous ping exceeds 600 seconds. A cumsum over that boolean is the session key, and it removes any need for a per-provider Python loop.
Attribute duration to intervals, not to pings: each ping owns the seconds until the next ping inside the same session, and the final ping owns a capped 30 seconds. Because the state seconds are the intervals themselves, they sum to online_seconds by construction rather than by a correction step.
Encode the three terminations distinctly. Gap timeout ends at last_ping + 30s with end_reason 'app_background_timeout'; an explicit offline ping ends at that ping with 'manual_offline'; a session with no terminating event before the data ends is 'session_still_open' with offline_at NaT.
On a market change, close the old session at its last ping in the old market and open the new one at the first ping in the new market; the seconds in between belong to neither session, and the output should say so rather than quietly padding one side.
Aggregate with a single groupby on the session key, pivoting the per-interval state into the three second columns, then assert the sum identity and that consecutive sessions for one provider never overlap.

Follow-up

A provider is engaged on a 40-minute order and the app backgrounds mid-order. What does your 10-minute rule do to that session, and what does it do to utilisation?
Utilisation divides engaged by online. Which of your three end_reason cases biases it most, and in which direction?

Simulate dispatch cascades and the censoring of wait time

Medium · simulation · censoring · monte carlo error

A request is offered to one provider at a time. Each offer resolves 12 seconds after it is sent, and each provider accepts independently with probability 0.55. After six declines the request is marked no_supply. Independently, the consumer abandons at time A drawn from an Exponential distribution with mean 90 seconds; abandonment before a pending offer resolves ends the request unmatched. Simulate 200,000 requests and report: the share matched, the mean time-to-match over matched requests, and the mean over requests that would have matched with abandonment switched off. Give a Monte Carlo standard error for the share.

Approach

Vectorise the cascade: draw K with np.random.default_rng().geometric(0.55), mark K > 6 as no_supply, and draw A = rng.exponential(90) independently; the match condition is K <= 6 and A > 12*K. Looping request by request is the difference between two seconds and two minutes of runtime.
Compute both means on the same draws so the comparison is paired and the difference is not itself a Monte Carlo artefact.
Recognise the structure driving the answer: abandonment censors long cascades harder than short ones, so conditioning on matched requests is not a neutral filter, it is a filter correlated with the quantity being measured.
Quote the share to three decimals only: the standard error of a proportion is sqrt(p(1-p)/n), roughly 0.0009 at n = 200,000, so further digits are noise.
Check against the closed form P(match) = sum over k of 0.45^(k-1) * 0.55 * exp(-12k/90) for k = 1..6; a simulation with no analytical check is an untested function.

Follow-up

A change ships that makes consumers abandon sooner. What happens to your reported mean time-to-match, and how would you report latency so that this cannot look like a win?
How would you estimate the same quantity from production data, where you never observe the latent match time of an abandoned request?

Explain the performance trade-offs between different types of joins in a large-scale database.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Using window functions, identify the top 3 sellers per category based on sales volume.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Allocate session time to local market-hours for utilisation

Hard · interval clipping · utilisation · timezones

fct_supply_session has provider_id, market_id, online_at_utc, offline_at_utc (NULL while open), online_seconds, engaged_seconds, en_route_seconds, idle_seconds, where engaged plus en_route plus idle equals online. dim_market has market_id and timezone as an IANA name. Produce provider utilisation, engaged seconds over online seconds, per market per local hour for one week. Sessions cross hour boundaries and can run for hours, and open sessions must be clipped to the window end rather than dropped. The engaged total is per session with no timestamps, so state and defend your allocation rule.

Approach

Pick one direction for time and stay in it: either generate the hour grid in market-local time and convert each bucket back to UTC for the overlap test, or convert each session once with (online_at_utc AT TIME ZONE 'UTC') AT TIME ZONE m.timezone. Mixing the two is where daylight-saving errors enter.
Clip with overlap_seconds = EXTRACT(EPOCH FROM (LEAST(session_end, bucket_end) - GREATEST(session_start, bucket_start))), keeping only positive overlaps. COALESCE an open session's end to the window end before this so it contributes to the hours it genuinely covered instead of vanishing on a NULL comparison.
Allocate engaged time by pro-rating: engaged_seconds * overlap_seconds / online_seconds. Say explicitly that this assumes engagement is uniform within the session, which smooths real peaks; an exact allocation needs an engagement event stream this table does not carry.
Aggregate by summing allocated engaged and allocated online per market-hour and dividing the sums. Averaging per-session ratios into the hour weights a 12-minute session the same as a 6-hour one.
Handle daylight saving deliberately. In a market-local series the spring-forward day has 23 local hours and the autumn day has 25, so a grid built by adding 24 rows per day either loses an hour or folds two distinct hours into one label.

Worked solution (40 min)

Take one session that crosses four hour boundaries and compute its four overlaps by hand; they must sum to its clipped duration.
Build the bucket grid for one market and one day and count the rows: 24, except on the two daylight-saving days.
Join, clip, pro-rate and aggregate, then compare summed allocated online seconds against summed clipped session durations for the market.
Divide the sums and read one market-day as a series to check the shape against known demand peaks before trusting the whole week.

Expected result: One row per (market_id, local_hour_start) for the week, with allocated_online_seconds, allocated_engaged_seconds and a ratio in [0,1]. Summed allocated_online_seconds across all buckets in a market equals that market's total clipped session seconds inside the window, to rounding.

Verify it yourself

For a hand-checked sample, per-session overlap seconds sum to the session's clipped duration.
No bucket has allocated_engaged greater than allocated_online.
Sessions with offline_at_utc IS NULL appear in the output; if none do, open sessions were dropped by a NULL comparison.
Each market has 168 buckets for the week, except where a daylight-saving transition falls inside it.

Follow-up

How does uniform pro-rating bias a comparison between a market with long sessions and one with short ones?
If you could add one column to fct_supply_session to make this exact, which would it be, and what does collecting it cost at this event volume?

Propose a set of metrics to evaluate the health of the Depop marketplace.

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

If the "Add to Cart" rate suddenly drops by 10%, how do you investigate the root cause?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you design a metric to capture the quality of a seller's listing?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you measure the success of a new "Trending Items" feed?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

What are the common pitfalls when running A/B tests on a two-sided marketplace?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
Say whether units interfere with each other, and switch design if they do.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

Explain how you would handle network effects or spillover bias in an experiment.

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

Design a switchback for a batched matching algorithm

Hard · switchback · carryover · delta method · power

A matching change batches requests for up to 15 seconds before assigning, instead of assigning greedily. Median engaged time per order is 18 minutes. You have six markets, 14 days, and 12 active hours per day. The primary metric is SLA fill rate from fct_request. Design the switchback: block length, burn-in, assignment scheme, the estimand, and the variance estimator. Then compute the minimum detectable effect at 80% power and two-sided alpha = 0.05, given a block-level fill-rate standard deviation of 6 percentage points.

Approach

Set block length from the carryover horizon, not from convenience. An accepted order occupies a provider for roughly the engaged duration, so assignments made under one arm constrain the pool available in the next block. With an 18-minute median, use 1-hour blocks and discard the first 20 minutes of each block as burn-in, analysing the trailing 40 minutes. Shorter blocks buy more units but leak more of the previous arm into every block, pulling both arms toward the middle and biasing the effect toward zero.
Randomise arm independently per market-hour, stratified so every market gets a balanced count within each hour-of-week. Do not alternate deterministically: a fixed ABAB pattern aliases with the daily demand cycle and with any weekly operational rhythm, and no variance estimator will warn you.
Count the real sample. Six markets times 12 hours times 14 days is 1,008 blocks, about 504 per arm. That is the sample size. The millions of requests inside those blocks share a treatment assignment and a supply pool, so they are not independent units and cannot be counted as such.
Fix the estimand before the variance estimator, because there are two defensible choices and mixing them is the common error. Equal weight per block, the mean of block-level fill rates, matches the randomisation unit and yields a plain two-sample comparison. Demand-weighted, the ratio of summed numerator to summed denominator, matches the reported business metric and needs the delta method: Var(R) = (1/n) * (1/mu_D^2) * [Var(N) - 2RCov(N, D) + R^2*Var(D)] with R = mu_N / mu_D. State which one you are powering and reporting.
Compute the MDE on the equal-weight estimand: (z_0.975 + z_0.80) * sigma * sqrt(2 / n_per_arm) = 2.80 * 0.06 * sqrt(2/504) = 2.80 * 0.06 * 0.0630 = 0.0106, about 1.1 percentage points absolute. If the change is expected to move fill rate by less than that, the test as scoped cannot decide, and the honest options are more markets, more days, or variance reduction rather than a longer squint at the result.
Check serial correlation before trusting the interval. Adjacent blocks inside one market are correlated, so run an A/A over historical blocks; if residuals are autocorrelated, cluster at market-day rather than block, or use randomisation inference over the actual assignment mechanism, which is exact and assumes nothing about block independence, and recompute power with the resulting design effect.

Worked solution (40 min)

Fix 1-hour blocks against the 18-minute carryover horizon and define a 20-minute burn-in excluded from analysis.
Count blocks: 6 markets * 12 hours * 14 days = 1,008 blocks, 504 per arm.
Compute the standard error of the difference on the equal-weight estimand: 0.06 * sqrt(2/504) = 0.06 * 0.0630 = 0.00378.
Multiply by 2.80, the sum of z at two-sided 5% and z at 80% power, giving an MDE of 0.0106.
Compare 1.1 percentage points against the shipping threshold and decide whether to extend the window, add markets, or apply pre-period covariate adjustment (rho = 0.7 gives sigma 0.0428 and an MDE near 0.76 percentage points).
Run an A/A over historical blocks to confirm the realised standard error and to test for residual autocorrelation before launching.

Expected result: One-hour market-hour blocks with a 20-minute burn-in, stratified by hour-of-week, 504 blocks per arm, an equal-weight-per-block estimand with block-clustered or randomisation-based inference, and a minimum detectable effect of about 1.1 percentage points absolute on fill rate.

Verify it yourself

An A/A switchback replayed on historical blocks should return a non-significant effect and a standard error near 0.0038.
Dropping the burn-in changes the requests inside each block but not the block count; if fill rate in the burn-in window differs from the analysed window within the same arm, carryover is real and the block is too short.
The equal-weight and demand-weighted estimates will differ whenever block sizes vary, so the reported number, the powering calculation and the variance estimator must all refer to the same one.

Follow-up

Median engaged time rises to 45 minutes in one vertical. What changes in the design, and what does it cost in power?
What MDE would you get with covariate adjustment at rho = 0.7 on pre-period market-hour fill rate?
The team wants provider earnings per online hour from the same test. Is the market-hour block the right unit for that metric?

A fill-rate step change that no dispatch change caused

Medium · definition change · metric governance · numerator denominator

SLA fill rate rose 2.3 points on a single day in every market at once and held there. A dispatch radius change shipped that same week, but only in three markets. The metric is requests matched within the market's SLA divided by all requests created, excluding requests abandoned within 10 seconds of creation. Decide whether the step is a real improvement, and give the one plot that distinguishes a definition change from a behaviour change.

Approach

Start from the simultaneity. Marketplace behaviour does not step in every market on the same minute. A change that lands everywhere at once is a code, config or definition change; a real dispatch change appears only in the three markets that received it, and usually with a ramp.
Plot the numerator count and the denominator count on the same axis instead of plotting the rate. A definition change normally moves exactly one of them by a level shift, and the 10-second exclusion moves the denominator down while leaving the numerator flat, because a request abandoned inside 10 seconds was never matched and so was never in the numerator.
Backfill one definition across the whole window; do not difference the two definitions day by day. Old and new differ by the 10-second exclusion, which applies on every day of history, so the two series sit apart throughout the window and their gap dates nothing. The diagnostic is single-definition continuity: recompute the entire window under the new definition, then again under the old one. If each single-definition series runs continuously through the step date while the published series steps, the definition changed that day and history was never restated. If the step survives under a consistently applied definition, the data or the behaviour moved and the definition is not the story.
The gap between the two definitions is still worth plotting, for its level rather than its timing. It should sit near the historical share of requests abandoned within 10 seconds of creation and stay roughly flat across the window. A gap that itself jumps at the step date means the excluded population changed as well, which is a data change wearing a definition change's clothes.
Read the SLA source. The threshold is per market, so an edit to the SLA config, or a new market silently inheriting a default, shifts the numerator with a flat denominator, which is the opposite signature.
Only once the step is explained should you evaluate the dispatch change, and then strictly inside the three treated markets against the untreated ones over the same days, both computed under one definition.

Follow-up

How do you report a metric whose definition changed mid-quarter without restating history as an improvement?
What would you require of any future metric change so this is unambiguous on day one?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
On the third attempt, write the grain of every CTE as a comment before writing its body.
Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
Re-read only your own notes from this week, and open no new material.
Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

How do you handle a project where the data is messy or the requirements are ambiguous?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Defend a finding that a provider bonus is incremental but uneconomic

Medium · incremental spend · holdout · stakeholder conflict

A provider bonus ran in 40 treated market-hours. The program owner's read is +12% completed orders versus the prior week in the treated cells. Your holdout analysis, against untreated cells that shared the same demand shock, puts the effect at 0.06 incremental completed orders per incentive dollar, 95% interval [0.01, 0.11], against contribution margin of $1.40 per completed order. The owner presents the +12% at a review tomorrow and has not seen your number. Deliverable: what you do before the meeting, what you say in it, and the evidence you bring.

Approach

Reproduce the owner's +12% exactly, from their cells and their window, before doing anything else; a disagreement where you cannot reproduce the other number is a credibility fight rather than a measurement one.
Be precise about what you found: the interval [0.01, 0.11] excludes zero, so the bonus does buy orders and you are not claiming otherwise. The argument is about price, not existence, and saying 'it did nothing' hands the owner a refutation built from your own numbers.
Set the bar in the same units as the estimate: breaking even at $1.40 of contribution margin needs 1 / 1.40, about 0.71 incremental orders per dollar. The upper end of your interval, 0.11, is about six and a half times short of that, so every value the data support loses money - which is what makes a wide interval decision-ready without being narrowed.
Invert the ratio, because per-dollar increment is easy to nod at and hard to feel: 0.06 orders per dollar means about $16.70 of bonus per incremental order against $1.40 of margin, the optimistic end 0.11 is about $9.10, and the pessimistic end 0.01 is about $100.
Show the displacement rather than asserting it, and keep the evidence cells out of the comparison set: adjacent untreated hours and adjacent zones where completed orders fell while treated cells rose is the signature of volume moved rather than created. State the precondition - if displacement also reached the difference-in-differences comparison cells, their post-period is depressed by the treatment, the estimate is biased upward, and 0.06 is an upper bound rather than a point estimate.
Separate the measurement question from the decision question and name what would change your mind: a randomised holdout at the same granularity as the incentive, sized in advance, with a date. Offer to run it rather than only to block the program.

Follow-up

The owner argues the bonus buys provider retention rather than orders - how would you test that claim, and over what horizon?
At what contribution margin per completed order would 0.06 orders per dollar break even, and is that margin reachable in this marketplace?
How would you tell displacement across hours apart from a genuine demand shift?

Describe an analysis you got wrong after a decision shipped

Medium · post-mortem · rollup bias · process controls

Describe a number you published that turned out wrong, where someone had already made a decision on it. State the mechanism of the error rather than the feeling; how long it was live; who acted on it and what that cost; how it surfaced and whether you were the one who found it; and the control you put in afterwards. An error caught in review before anyone acted does not qualify for this question. Deliverable: three minutes, ending with the one process change that is still in place today.

Approach

Choose an error with a real mechanism you can draw in one sentence, not a communication miss; the question is probing whether you understand how your own work fails, and a 'they misunderstood my chart' story answers a different question.
State the blast radius honestly and numerically: days live, decisions taken, dollars or headcount moved. Vagueness here reads as an error you never actually measured.
Say how it surfaced, including the unflattering version if someone else found it. Claiming self-detection on an error that a stakeholder caught is the fastest way to lose the room.
Separate the mechanism from the conditions that let it survive: a wrong formula is one bug, but no reconciliation check and no second reader are the reasons it lived for weeks.
End on a structural control, not an intention. 'I will be more careful' is not a control; a test that fails the job when two computations of the same metric disagree is.

Follow-up

How soon after you knew did the decision-maker know, and who told them?
Has the control you added caught anything since, and how would you know if it had silently stopped working?
What class of error would that control still miss?