Interview guides · Deliveroo

Deliveroo Data Scientist Interview Questions & Guide 2026

Causal inference drills for Deliveroo data scientist prep: diff-in-diff, matching, instruments, and the limits of observed data.

The 60-second brief

As a Data Scientist at Deliveroo, you sit at the intersection of complex logistics, hyper-local marketplace dynamics, and consumer behavior. Your work directly influences how millions of customers interact with restaurants, how thousands of riders navigate cities, and how the platform optimizes delivery times and efficiency. You are not just building models; you are solving high-stakes, real-world problems that define the company's competitive advantage in the food-delivery space.

How much statistics you need depends on the flavour of the seat. Experiment-facing work wants you deep enough to notice that repeated looks at accumulating data inflate the false positive rate of a fixed-sample test; modelling-facing work wants estimation and honest uncertainty intervals.

Focus areas

Design switchback tests that survive supply-side interference
Query request, match and payout grains without fan-out
Diagnose whether a market is supply- or demand-constrained

Overview

The role is deeply collaborative and strategic. You will work closely with Product Managers, Engineers, and Operations teams to translate ambiguous business challenges into analytical frameworks. Whether you are improving surge pricing, forecasting demand, or optimizing dispatch algorithms, your impact is immediate and measurable. Deliveroo values candidates who can balance technical rigor with a strong commercial mindset, ensuring that every data-driven decision contributes to a seamless user experience.

The Deliveroo interview process

Deliveroo candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Application Review

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Phone Screen

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Assessment

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Behavioral Interview

Rounds of this kind usually include one question about work that did not go well, and it is the part that carries the most information. Anyone can narrate a shipped win. What the interviewer learns from a project that stalled is how you behave without a result to hide behind: whether you noticed the problem yourself, how long it took, and who you told. Answers that route the failure onto a data pipeline or a reorganisation close the topic without answering it, and the follow-up comes back to your own part.

What to demonstrate

Whether you found the error yourself or someone else found it, and how long it sat before anyone knew
What you changed afterwards, stated as a check you now run rather than a lesson you now believe
Whether the mistake you choose has real cost attached, such as a quarter of misdirected roadmap or a metric that was reported upward, instead of one that flatters you

How to prepare

Choose a failure you caught yourself and be ready to say what tipped you off. A story where someone else caught it is still usable, but you will be asked why you missed it.
Write down the check you added afterwards and where it lives now, so the correction is a concrete artefact rather than a resolution.
Rehearse saying the cost out loud. Candidates shrink the number by instinct once the interviewer is in the room.

Final Loop

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Denominator drift in per-active-user metrics

Orders per active consumer falls when acquisition succeeds, because new cohorts transact less than tenured ones, so the metric penalises the thing the company is trying to do. A team that optimises it will quietly prefer weaker acquisition. Decompose into cohort size times cohort frequency, or hold the cohort fixed and read frequency by tenure bucket, before drawing any conclusion about engagement.

Randomising individual consumers when supply is shared

A feature that makes treatment consumers book faster consumes the same idle providers the control consumers would have used, so the control group is degraded by the treatment and the measured lift overstates the market-level effect. The bias is largest precisely when supply is tight, which is when the feature is supposed to help, so the experiment is most misleading exactly where the decision matters. The fix is randomising the market or the time block (switchback) and clustering the variance at the randomisation unit, accepting far fewer effective units.

Analysing at a different unit than the one randomised

Say out loud what was randomised (user, device, account, cluster) and make the analysis unit match, or account for the clustering with cluster-robust standard errors, the delta method, or aggregation up to the randomised unit. Randomising users and then running a test over sessions understates variance and inflates the false-positive rate.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Questions & practice

Permutation test and block bootstrap for a switchback

Medium · switchback · permutation test · clustered variance · bootstrap

You have mh: market_id, hour_start_utc, block_id, arm ('treat' or 'control'), eligible_requests, matched_in_sla. One row per market-hour, fourteen days in a single market, randomised in two-hour blocks. Without scipy.stats or statsmodels, produce three things: the effect estimate as the difference in ratio-of-sums fill rate between arms; a two-sided permutation p-value that reassigns arm at the block level holding the observed number of treated blocks fixed; and a 95% confidence interval from a bootstrap that resamples whole blocks with replacement. Use 10,000 iterations for each.

Approach

Collapse to the randomisation unit first: sum eligible_requests and matched_in_sla per block_id and keep the arm label. Every later operation runs over 168 blocks, which is what makes the variance estimate honest.
Estimate the effect as ratio of sums within each arm, not as the mean of per-block rates, because blocks carry very different volume and the mean of rates answers a different question.
Permutation: shuffle the arm vector across blocks keeping the treated count fixed, recompute the same statistic, and report p = (1 + count of |stat*| >= |stat_obs|) / (B + 1). The plus-one is the correct finite-sample form and keeps p strictly positive.
Bootstrap: resample block indices with replacement within each arm, recompute the ratio difference, and take the 2.5th and 97.5th percentiles; resampling blocks rather than hours propagates both the rate and the volume variation.
State the effective sample size as 168 blocks rather than the request count, and convert that into the minimum detectable effect the design actually supports.

Worked solution (35 min)

blocks = mh.groupby(['block_id','arm'], as_index=False)[['eligible_requests','matched_in_sla']].sum().
Define stat(arm_vector) = matched[treat].sum()/eligible[treat].sum() - matched[control].sum()/eligible[control].sum(), operating on NumPy arrays.
Permutation loop: for each of 10,000 draws, rng.permutation of the arm vector, recompute stat, accumulate the count of |stat*| >= |stat_obs|.
Bootstrap loop: draw block indices with replacement separately inside each arm, recompute stat, store; take np.percentile at 2.5 and 97.5.
Report the point estimate in percentage points, the p-value, the interval, and the number of blocks.

Expected result: A fill-rate difference in percentage points, a permutation p-value bounded below by 1/10001 and never exactly zero, and a percentile interval that contains the point estimate; all three computed over 168 blocks, not over 336 hours or over individual requests.

Verify it yourself

The permutation null distribution is centred on approximately zero; its mean should sit within one Monte Carlo standard error of zero.
Recompute the p-value permuting hours instead of blocks: it is visibly smaller, and you should be able to say why that is wrong rather than merely different.
Report both the ratio-of-sums estimate and the mean-of-block-rates estimate; they differ when block volumes differ, and you should defend the one you chose.
The bootstrap interval width shrinks roughly as the square root of the block count if you subsample blocks, confirming the clustering is active.

Follow-up

Blocks are two hours and the median order lasts 25 minutes. Where does carryover leak across the boundary, and what would you do with the first minutes of each block?
How would you add covariate adjustment on pre-period market-hour fill rate without breaking the permutation argument?

Decompose completed-order change into requests, fill, completion

Hard · funnel decomposition · exact additivity · lmdi · mix shift

You are given panel: market_id, period ('base' or 'current'), requests, matched, completed, one row per market per period. Completed orders satisfy C = R * f * c where f = matched/requests and c = completed/matched. Write decompose(panel) returning, per market, the contribution of the change in R, f and c to the change in C under two methods: the sequential chain rule with its interaction terms written out, and the log-mean (LMDI) decomposition. Contributions must sum to the change in C to floating-point tolerance under both methods. Handle a market with zero requests or zero matched in one period.

Approach

Write the sequential identity before coding and confirm it telescopes: dC = (R1-R0)f0c0 + R1*(f1-f0)c0 + R1f1*(c1-c0) expands exactly to R1f1c1 - R0f0c0. It is exact but order-dependent, and the interaction terms live wherever the chosen order puts them.
Implement LMDI with the logarithmic mean L(a,b) = (a-b)/(ln a - ln b) and L(a,a) = a: the contribution of R is L(C1,C0)*ln(R1/R0), and likewise for f and c. It is exact and order-independent, which is why it is the one to use when two factors move together.
Handle the degenerate cases in code rather than in a comment: zero requests, zero matched (c undefined), and equal values hitting the log-mean's removable singularity. Return NaN with a reason column instead of a silent zero.
Assert additivity per market with np.allclose on the sum of contributions minus the change in C, and raise if it fails; a decomposition that does not close is not a decomposition.
Roll up by summing contributions across markets, since sums of exact per-market decompositions stay exact, and report the aggregate rates separately because they will not equal the request-weighted per-market rates once the market mix has moved.

Follow-up

Requests are flat, fill fell two points, completed fell six percent. What do you query next to separate fewer online hours from worse dispatch?
Aggregate fill rate fell while every single market's fill rate rose. Write the mix term that reconciles those two facts.
Which of the two decompositions would you put in a recurring report, and what would you have to explain to its readers every month?

Simulate dispatch cascades and the censoring of wait time

Medium · simulation · censoring · monte carlo error

A request is offered to one provider at a time. Each offer resolves 12 seconds after it is sent, and each provider accepts independently with probability 0.55. After six declines the request is marked no_supply. Independently, the consumer abandons at time A drawn from an Exponential distribution with mean 90 seconds; abandonment before a pending offer resolves ends the request unmatched. Simulate 200,000 requests and report: the share matched, the mean time-to-match over matched requests, and the mean over requests that would have matched with abandonment switched off. Give a Monte Carlo standard error for the share.

Approach

Vectorise the cascade: draw K with np.random.default_rng().geometric(0.55), mark K > 6 as no_supply, and draw A = rng.exponential(90) independently; the match condition is K <= 6 and A > 12*K. Looping request by request is the difference between two seconds and two minutes of runtime.
Compute both means on the same draws so the comparison is paired and the difference is not itself a Monte Carlo artefact.
Recognise the structure driving the answer: abandonment censors long cascades harder than short ones, so conditioning on matched requests is not a neutral filter, it is a filter correlated with the quantity being measured.
Quote the share to three decimals only: the standard error of a proportion is sqrt(p(1-p)/n), roughly 0.0009 at n = 200,000, so further digits are noise.
Check against the closed form P(match) = sum over k of 0.45^(k-1) * 0.55 * exp(-12k/90) for k = 1..6; a simulation with no analytical check is an untested function.

Follow-up

A change ships that makes consumers abandon sooner. What happens to your reported mean time-to-match, and how would you report latency so that this cannot look like a win?
How would you estimate the same quantity from production data, where you never observe the latent match time of an abandoned request?

Write a query to identify top-performing restaurants based on order frequency and delivery time.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Given two tables (orders and riders), how would you join them to find the average delivery time per city?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Fill rate by market-day without averaging hourly rates

Easy · fill rate · aggregation · timezones

fct_request holds one row per demand request: request_id, market_id, requested_at_utc, requested_at_local, matched_at_utc (NULL when the request never matched), terminal_at_utc, request_status. dim_market holds market_id, vertical, timezone. Produce SLA fill rate per market per local calendar day for the last 28 days. A request enters the numerator when matched_at_utc is not null and matched_at_utc minus requested_at_utc is at or under 120 seconds for vertical 'ride' and 600 seconds for 'delivery'. Exclude requests with request_status = 'abandoned_pre_match' whose terminal_at_utc is within 10 seconds of requested_at_utc. Return market_id, local_date, numerator, denominator, rate.

Approach

Build the eligible population as a CTE first: cut the 28-day window, then drop the sub-10-second abandons. Applying the exclusion before any aggregation guarantees the numerator and the denominator are computed over the identical row set.
Cut the window and take the bucket off the same clock. local_date is date(requested_at_local), and the window is local_date BETWEEN end_date - 27 AND end_date, where end_date is the last local calendar date that has closed in every market. Cutting on requested_at_utc instead moves the day boundary by up to 14 hours: the same query then returns 29 local dates for any market at a non-zero UTC offset, the first and last of them partial. A partial edge day holds a few hours of one market's overnight or evening traffic, so its rate is computed over a slice whose fill rate is nothing like the day's, and it is then read beside 27 whole days.
Express the SLA as a CASE on dim_market.vertical rather than a hardcoded constant, then compute the numerator as COUNT(*) FILTER (WHERE matched_at_utc IS NOT NULL AND matched_at_utc - requested_at_utc <= sla).
Aggregate once at market by local_date and divide summed numerator by summed denominator. If an hourly view is also wanted, compute the hourly counts and re-sum them to the day; do not average the hourly rate.
Divide through NULLIF(denominator, 0). Under the CTE-first shape a market-day whose rows were all excluded produces no row at all rather than a zero denominator, so a gap in the dates means zero surviving requests, not a zero rate. Keep the NULLIF anyway: it is what stops a divide-by-zero the day someone moves the exclusion inside the aggregate instead of ahead of it.

Worked solution (20 min)

Count rows inside the 28 local-date window before and after the sub-10-second exclusion; the exclusion should remove a low single-digit percentage, not a third of the table.
Write the CASE-based SLA and hand-check five matched requests against their computed time-to-match.
Aggregate to market by local_date with COUNT() for the denominator and COUNT() FILTER (...) for the numerator, in one pass.
Divide the summed counts with NULLIF on the denominator, then count distinct local_date per market before ordering the output by market_id, local_date.

Expected result: One row per (market_id, local_date), covering at most 28 local dates per market, with integer numerator and denominator and a rate in [0,1]. A market-day with no surviving requests produces no row, so a missing date is a zero-volume day rather than a zero rate. Because the rate is SUM(numerator)/SUM(denominator), re-aggregating the output to a market total by re-summing the two count columns reproduces that market's 28-day rate exactly.

Verify it yourself

SUM(denominator) over the whole output equals a plain count of surviving in-window requests computed with no joins, where both sides cut the window on requested_at_local.
COUNT(DISTINCT local_date) per market is at most 28 and never 29. A 29th date, or a first and last date carrying a fraction of their neighbours' volume, means the window was cut on requested_at_utc while the bucket was taken from requested_at_local.
No row has numerator greater than denominator, and no rate exceeds 1.
Re-summing the day rows to a market total gives a different number from AVG of the daily rates; if the two agree to four decimals, your daily volumes are implausibly uniform and something is wrong.

Follow-up

Fill rate is flat at the market-day level but down four points in the 17:00 to 19:00 local hours. Which cut do you run next, and what would confirm a supply cause rather than a demand spike?
The 10-second exclusion removes six percent of requests in one market and half a percent in another. What do you check before trusting either market's rate?

How would you measure the impact of adding more restaurants to a specific delivery zone on order growth?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

You notice a drop in conversion rates on the checkout page; how would you investigate the root cause?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you define a "healthy" marketplace for a new city launch?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

If we change the delivery fee structure, how would you estimate the impact on total revenue and order volume?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Explain the concept of SUTVA (Stable Unit Treatment Value Assumption) and why it is critical for our experiments.

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

How do you design an A/B test for a new feature in a marketplace environment?

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
Decide the analysis before seeing data, including how long it runs and when you look.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

What are the common pitfalls in causal inference when analyzing observational data?

Medium

Approach

Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Value a bonus at an order-count threshold without randomisation

Hard · regression discontinuity · manipulation test · local average treatment effect

A weekly bonus pays providers who complete at least 25 orders in a calendar week, and the app shows a live progress counter. Finance asks what the bonus does to next-week retention, defined as at least one completed order in week w+1. A randomised holdout was refused. You have 18 months of weekly per-provider completed-order counts from fct_order and bonus payments from fct_money_movement. Propose an identification strategy, name the assumption it rests on, give the test that would falsify that assumption here, and say what you do when the test fails.

Approach

Propose the design the data invites: a sharp regression discontinuity in the running variable, completed orders in week w, at the cutoff of 25, comparing week w+1 retention just above and just below. Estimate with local linear regression on each side, an MSE-optimal bandwidth and bias-corrected robust confidence intervals. Avoid a high-order global polynomial, which gives weight to observations far from the cutoff and manufactures artefacts at the boundary.
State the identifying assumption plainly: continuity of potential outcomes at the cutoff, meaning providers just below 25 are a valid counterfactual for those just above. A live progress counter is precisely the mechanism that destroys it, because providers who can reach 25 will, so the population just above is selected on motivation and availability rather than on chance.
Falsify it before estimating anything. Run a density-continuity test on the running variable, looking for a deficit just below 25 and a spike at 25 and 26. Test continuity at the cutoff of variables fixed before the week began, including tenure, prior-week orders, market and approval date; a jump in any of them is the same warning arriving by a different route.
When the density test rejects, say the design is not identified and stop treating the coefficient as causal. A donut specification that drops a window around the cutoff addresses heaping in a discretised running variable, not sorting driven by the incentive itself, so it does not rescue this case and offering it as a fix is the wrong answer.
Rank the fallbacks by credibility rather than by convenience. First, an instrument the provider cannot observe or manipulate, used in a fuzzy design, where the exclusion restriction requires that the instrument affect retention only through bonus receipt, which disqualifies anything providers are told about. Second, difference-in-differences or synthetic control across markets whose bonus threshold moved on known dates, clustering at market and showing the pre-trend leads. Third, negotiate a staggered rollout, which is usually cheaper to obtain than it looks and turns the question back into an experiment.
Bound the claim even under a valid design. Regression discontinuity identifies a local effect at 25 orders per week. It says nothing about providers at 5 or at 60, while the decision Finance is making, whether to keep funding the bonus, is about the whole distribution.

Worked solution (40 min)

Histogram weekly completed-order counts in unit-width bins from 15 to 35 and look for a deficit below 25 and a spike at 25 to 26.
Run a local-polynomial density-continuity test at the cutoff and record the estimate, standard error and p-value.
Test continuity at the cutoff of pre-determined covariates: tenure in weeks, prior-week completed orders, market and provider_approved_at_utc cohort.
If either test rejects, write the regression discontinuity off as descriptive, document the bunching evidence, and cost the fallback designs including a staggered threshold rollout.
If both pass, estimate with local linear regression, MSE-optimal bandwidth and robust bias-corrected intervals, and report the local effect with its bandwidth sensitivity and its explicit scope limit.

Expected result: Bunching at 25 with a rejected density-continuity test, so the sharp design is not identified; the deliverable is that falsification evidence plus a costed fallback, a staggered threshold rollout or a cross-market difference-in-differences on dated threshold changes, rather than a discontinuity coefficient.

Verify it yourself

The same density test run on weeks before the bonus existed should show no bunching; a placebo that also bunches means the heaping is an artefact of counting, not of the incentive.
A valid discontinuity estimate is stable across bandwidths inside its robust interval, so large movement with bandwidth is further evidence against identification.
Jumps at the cutoff in variables determined before the week started must be zero, because nothing that happened during the week can have caused them.

Follow-up

The counter is hidden for a random subset of providers next quarter. What design does that enable, and what does it identify?
The density test passes in one vertical and fails in another. Would you publish the estimate from the vertical that passed?
What would have to be true for an instrument based on a provider market-level threshold to satisfy the exclusion restriction?

Median time-to-match improved while pre-match abandonment rose

Hard · censoring · competing risks · selection bias

Median time-to-match, computed as matched_at_utc minus requested_at_utc over fct_request rows where matched_at_utc IS NOT NULL, improved from 74 to 61 seconds in one market. Over the same days, request_status = 'abandoned_pre_match' rose from 6% to 11% of requests created. A dispatch change shipped that week. Using fct_request alone, say whether waiting genuinely improved, give an estimator that is not conditioned on matching, and state the assumption that estimator needs.

Approach

Name the conditioning first. The statistic is computed only on requests that matched, and abandonment removes the longest waits from that subset before they can be observed, so anything that makes consumers quit sooner shortens the measured distribution mechanically.
Replace the conditional mean with a fixed-horizon rate at the request grain: the share of all requests created in the window that matched within T seconds, T being the market's SLA. It is unconditional, it moves only when matching actually improves, and it is the SLA fill rate the market already reports.
If the full curve is needed, estimate the cumulative incidence of matching with abandonment as a competing event, not Kaplan-Meier with abandonment as censoring. KM requires the censoring mechanism to be independent of the match hazard, and here people abandon precisely because they are still waiting, so KM would overstate the improvement.
Distinguish the two stories that both fit the data: the match hazard rose (dispatch got faster), or consumers got less patient, for instance because a new screen surfaces a worse quote. Look at the abandonment hazard by elapsed second and at the quoted_eta_seconds distribution before and after. Worse quotes with an unchanged wait means people are quitting on the quote.
Report matched requests per 100 created next to any latency number, every time, and state whether the two movements net to more or fewer matches. That net figure is what the decision depends on.

Follow-up

Give the two-sentence version for a product review, without using the word censoring.
Abandonment is flat but 'no_supply' expiries rose instead. Does that change your estimator?
How would you power a switchback on the fixed-horizon metric given market-hour variance?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Stories where someone disagreed with you

Have two ready. In one, the data was on your side and you had to move someone who outranked you. In the other, the pushback was correct and you changed position. The second is the harder story and it lands better, because it shows you separate being right from being attached to an answer. Name the person's actual objection.

How do you handle date-time formatting and time-zone conversions in PostgreSQL?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Defend a finding that a provider bonus is incremental but uneconomic

Medium · incremental spend · holdout · stakeholder conflict

A provider bonus ran in 40 treated market-hours. The program owner's read is +12% completed orders versus the prior week in the treated cells. Your holdout analysis, against untreated cells that shared the same demand shock, puts the effect at 0.06 incremental completed orders per incentive dollar, 95% interval [0.01, 0.11], against contribution margin of $1.40 per completed order. The owner presents the +12% at a review tomorrow and has not seen your number. Deliverable: what you do before the meeting, what you say in it, and the evidence you bring.

Approach

Reproduce the owner's +12% exactly, from their cells and their window, before doing anything else; a disagreement where you cannot reproduce the other number is a credibility fight rather than a measurement one.
Be precise about what you found: the interval [0.01, 0.11] excludes zero, so the bonus does buy orders and you are not claiming otherwise. The argument is about price, not existence, and saying 'it did nothing' hands the owner a refutation built from your own numbers.
Set the bar in the same units as the estimate: breaking even at $1.40 of contribution margin needs 1 / 1.40, about 0.71 incremental orders per dollar. The upper end of your interval, 0.11, is about six and a half times short of that, so every value the data support loses money - which is what makes a wide interval decision-ready without being narrowed.
Invert the ratio, because per-dollar increment is easy to nod at and hard to feel: 0.06 orders per dollar means about $16.70 of bonus per incremental order against $1.40 of margin, the optimistic end 0.11 is about $9.10, and the pessimistic end 0.01 is about $100.
Show the displacement rather than asserting it, and keep the evidence cells out of the comparison set: adjacent untreated hours and adjacent zones where completed orders fell while treated cells rose is the signature of volume moved rather than created. State the precondition - if displacement also reached the difference-in-differences comparison cells, their post-period is depressed by the treatment, the estimate is biased upward, and 0.06 is an upper bound rather than a point estimate.
Separate the measurement question from the decision question and name what would change your mind: a randomised holdout at the same granularity as the incentive, sized in advance, with a date. Offer to run it rather than only to block the program.

Follow-up

The owner argues the bonus buys provider retention rather than orders - how would you test that claim, and over what horizon?
At what contribution margin per completed order would 0.06 orders per dollar break even, and is that margin reachable in this marketplace?
How would you tell displacement across hours apart from a genuine demand shift?

Disagree with a product lead about the randomisation unit

Medium · interference · randomisation unit · switchback

A product lead wants to ship a faster-dispatch change on the strength of a consumer-level A/B: 400,000 consumers, +6.1% completed orders per consumer, p below 0.001. You believe the design measures the wrong quantity, because treated consumers take the idle providers control consumers would otherwise have matched with. The lead's counter is that the sample is enormous and the p-value tiny. Deliverable: the argument you make including the direction of the bias, the evidence you can produce from existing data in two days, and the design you propose instead with its calendar cost.

Approach

Concede the internal comparison and dispute the estimand: the test cleanly estimates a between-consumer contrast under a shared supply pool, which is not the market-level effect of shipping to everyone. Sample size does not touch this, because the bias does not shrink with n.
State the direction and the mechanism: control consumers are degraded by treatment, so the contrast is inflated. The inflation is largest when idle supply is scarce, which is precisely the condition the feature is meant to help, so the test is most wrong where the decision matters most.
Produce the fingerprint from data already in hand: split the measured lift by market-hour provider utilisation decile. Interference predicts the lift rises with utilisation; a genuine effect that does not steal supply does not have to. Pair it with control-arm fill rate against a pre-period baseline in high-utilisation hours, stating the precondition that this comparison is only informative if the pre-period is seasonally comparable or an untested market is available.
Propose the switchback concretely: block length longer than a typical order duration so carryover does not leak across the boundary, a burn-in discarded after each switch, randomisation at the market-hour, and variance clustered at the block.
Price the design honestly in calendar time so the lead can trade it off, and offer an interim: ship to one market with the rest held out, which is slower to read but not biased in the same direction.

Follow-up

If the lift does not rise with utilisation, what does that tell you, and would you then ship?
How do you choose block length when order durations have a long right tail?
What variance reduction still works under interference, and what does it assume?