Interview guides · Blinkit

Blinkit Data Scientist Interview Questions & Guide 2026

Blinkit data scientist prep on data pitfalls: selection bias, target leakage, and checking an outlier's provenance before you drop it.

The 60-second brief

As a Data Scientist at Blinkit, you are at the heart of one of the world’s fastest-moving quick-commerce ecosystems. Your role is to transform massive, real-time datasets into actionable intelligence that powers everything from hyper-local demand forecasting and ETA predictions to sophisticated search ranking and personalized product recommendations. Because Blinkit operates on a model where every minute counts, the work you do has an immediate, tangible impact on customer satisfaction, delivery efficiency, and operational profitability.

Scope your preparation by the data you would actually touch, because the title will not tell you. A seat that lives in event logs and weekly readouts rewards fluency in aggregation and metric definitions; a seat that owns a model in production rewards fluency in train/serve skew, retraining cadence and drift monitoring. The fastest way to find out which one you are interviewing for is to ask what the team shipped last quarter and what it gets paged about.

Focus areas

Estimate cross-side elasticities from cohort and holdout data
Query request, match and payout grains without fan-out
Price incentives against contribution margin, not gross bookings

Overview

You will function as a strategic partner to product and engineering teams, translating ambiguous business problems into rigorous analytical frameworks. Whether you are diagnosing a sudden drop in a key conversion metric or designing an A/B test to validate a new feature rollout, your contributions directly influence how the platform scales. This is a high-stakes, high-velocity environment where your ability to synthesize complex statistical findings into clear product recommendations will define your success.

The Blinkit interview process

Blinkit candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Technical Screening

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

System Design

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Behavioral Alignment

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Final Assessment

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Treating abandoned requests as missing rather than censored

Consumers who give up before matching are censored observations, so the mean time-to-match computed over matched requests understates true waiting and improves mechanically whenever abandonment rises. A change that makes people quit sooner will look like a latency win. Use survival methods with abandonment as the censoring event, or never report time-to-match without reporting abandonment next to it.

Conditioning the analysis on completed orders

Wait-time distributions, price elasticities and rating models fit only on completed orders are conditioned on an outcome that the intervention itself changes. The requests that never matched, or that the consumer abandoned, are the population a liquidity fix targets, so excluding them biases every estimate toward the status quo and can flip the sign of a price elasticity. Any query starting FROM fct_order is already inside this trap; start from fct_request and left join.

Averaging per-user rates to produce a population rate

Decide which quantity you want: the mean of per-user ratios and the ratio of summed numerator to summed denominator are different estimands, and heavy users dominate one but not the other. For a ratio metric, aggregate numerator and denominator separately and use the delta method for its variance.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Questions & practice

Sessionise a provider heartbeat stream into supply sessions

Hard · sessionisation · event streams · cumsum keys · interval arithmetic

You are given pings: provider_id, market_id, event_at_utc, status in ('idle','en_route','engaged','offline'), one row per app heartbeat, nominally every 30 seconds but with gaps. Reconstruct the supply-session table. A session starts at the first non-offline ping and ends at an explicit offline ping, at a market change, or when the gap to the next ping exceeds 10 minutes. Produce session_id, provider_id, market_id, online_at, offline_at, online_seconds, engaged_seconds, en_route_seconds, idle_seconds and end_reason, with the three state components summing to online_seconds exactly in integer seconds.

Approach

Sort by (provider_id, event_at_utc), then build a new_session boolean: first ping of a provider, previous status equals 'offline', market_id changed, or the gap from the previous ping exceeds 600 seconds. A cumsum over that boolean is the session key, and it removes any need for a per-provider Python loop.
Attribute duration to intervals, not to pings: each ping owns the seconds until the next ping inside the same session, and the final ping owns a capped 30 seconds. Because the state seconds are the intervals themselves, they sum to online_seconds by construction rather than by a correction step.
Encode the three terminations distinctly. Gap timeout ends at last_ping + 30s with end_reason 'app_background_timeout'; an explicit offline ping ends at that ping with 'manual_offline'; a session with no terminating event before the data ends is 'session_still_open' with offline_at NaT.
On a market change, close the old session at its last ping in the old market and open the new one at the first ping in the new market; the seconds in between belong to neither session, and the output should say so rather than quietly padding one side.
Aggregate with a single groupby on the session key, pivoting the per-interval state into the three second columns, then assert the sum identity and that consecutive sessions for one provider never overlap.

Worked solution (40 min)

Sort, then compute gap = event_at_utc.diff() per provider and prev_status / prev_market via shift(1) within provider.
new_session = provider changed | prev_status == 'offline' | market changed | gap > 600s; session_key = new_session.cumsum().
interval_seconds = next_event_at - event_at within session, with the last row of each session clipped to 30 seconds; drop rows whose own status is 'offline' from the state attribution.
Group by session_key: online_at = first event_at, offline_at from the termination rule, and sum interval_seconds overall and per status via a pivot.
Assert engaged + en_route + idle == online_seconds with integer equality, and assert non-overlap per provider before returning.

Expected result: One row per reconstructed session in which engaged_seconds + en_route_seconds + idle_seconds equals online_seconds exactly as integers, no two sessions for the same provider overlap in time, and every non-offline ping maps to exactly one session.

Verify it yourself

Assert the sum identity with integer == rather than np.isclose; a tolerance here hides the exact bug this exercise is about.
Session count equals the number of True values in new_session.
Per provider, the session intervals are ordered and disjoint under a strict comparison.
A hand-built 12-ping fixture containing one long gap, one explicit offline and one market change produces three sessions with the three distinct end_reason values.

Follow-up

A provider is engaged on a 40-minute order and the app backgrounds mid-order. What does your 10-minute rule do to that session, and what does it do to utilisation?
Utilisation divides engaged by online. Which of your three end_reason cases biases it most, and in which direction?

Net take and margin per market-month without ledger fan-out

Medium · join fan-out · unit economics · attribution window

You have orders (order_id, market_id, completed_at_utc, order_status, gross_booking_cents, provider_payout_cents, consumer_incentive_cents, provider_incentive_cents, tip_cents) and ledger (ledger_id, order_id, entry_type in charge, refund, chargeback, incentive, payout, adjustment, processing_fee; amount_cents signed positive into the platform; posted_at_utc; settlement_status). For completed orders, produce a market-month frame with net take, margin before support cost, and the order count. Attribute refunds, chargebacks and processing fees to the order's completion month, not the posting month, and drop ledger rows whose settlement_status is 'pending' or 'failed'.

Approach

Aggregate the ledger to the order grain before touching orders: filter settlement_status, then pivot entry_type into one summed column each. Merging first and summing after multiplies gross_booking_cents by the number of ledger rows on that order.
Merge the pivoted ledger onto orders with validate='m:1' so an unexpected duplicate raises instead of silently inflating every total.
Net take = gross_booking - provider_payout - consumer_incentive - provider_incentive over completed orders only; tips are excluded on both sides because they pass through to the provider and never enter platform revenue.
Add the signed ledger columns rather than subtracting absolute values: refunds, chargebacks, payouts and processing fees are negative under this convention, so margin = net_take + refund + chargeback + processing_fee. Verify the sign on one known order before trusting the aggregate.
Group by market_id and completed_at_utc month, never posting month; that is the whole point of the attribution rule, and it is why a month's margin is not final until the chargeback window closes.
Carry completed_orders in the output so per-order margin can be recomputed downstream without averaging an average.

Follow-up

A chargeback posts two months after completion and changes an already-reported month. How do you publish a metric that is not final, and what lag would you quote?
Should the denominator be matched orders, completed orders, or completed-and-settled orders? Argue for one and name what it hides.

Implement SLA fill rate from its written definition

Easy · metric definition · ratio of sums · groupby · timezones

You are given requests (request_id, market_id, requested_at_utc, requested_at_local, matched_at_utc which is NaT when the request never matched, terminal_at_utc, request_status) and markets (market_id, timezone, sla_seconds). Implement fill_rate(requests, markets) returning the market-day SLA fill rate. Numerator: requests where matched_at_utc minus requested_at_utc is at most that market's sla_seconds. Denominator: all requests created in the window except those with request_status 'abandoned_pre_match' that terminated within 10 seconds of creation. Compute at market-hour grain first, then roll up to market-day. Return numerator and denominator counts alongside the rate.

Approach

Merge sla_seconds onto requests by market_id first, so the SLA comparison is a vectorised column operation rather than a per-market branch.
Bucket on requested_at_local, not requested_at_utc: a market-day is a local-calendar object, and a pandas datetime64 column can only carry one timezone, which is exactly why the local column exists in the source table.
Build two integer columns on the full frame, eligible (1 minus the 10-second abandon exclusion) and matched_in_sla, instead of filtering rows away; the exclusion then stays auditable and both counts share one denominator.
Aggregate with groupby([market_id, local_hour]).sum() on those two counts, then roll to market-day by summing the counts again and dividing once at the end.
Return eligible and matched_in_sla next to the rate so a downstream consumer physically cannot re-average the rate across markets or hours.

Follow-up

The 10-second exclusion removes 3% of requests in one market and 11% in another. What would you check before trusting either market's fill rate?
A request can be re-offered after a provider declines. Does matched_at_utc still define your numerator, and what would you add to separate a dispatch problem from a supply problem?

How would you structure a SQL query to identify users who churned after their third order?

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Write a query to identify the top 5 products purchased by users in their first week of registration.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Approved providers who never matched and never went online

Medium · anti-join · null semantics · supply onboarding

dim_user has user_id, side, provider_status, provider_approved_at_utc. fct_request has request_id, provider_id and matched_at_utc, where provider_id is NULL on every request that never matched. fct_supply_session has session_id, provider_id, online_at_utc. List providers whose provider_status is 'approved' and whose provider_approved_at_utc falls in a given calendar month, who have never appeared as fct_request.provider_id and who have no row in fct_supply_session. Return user_id, provider_approved_at_utc and days since approval, oldest first.

Approach

Use NOT EXISTS for both exclusions, correlated on provider_id. NOT EXISTS is unaffected by NULLs in the subquery and lets the planner stop at the first matching row.
If you reach for NOT IN, provider_id is nullable, so the subquery result contains at least one NULL, every comparison evaluates to UNKNOWN instead of TRUE, and the query returns zero rows. There is no error and no warning; an empty result set reads as good news.
The correct alternatives are a LEFT JOIN with WHERE right_key IS NULL, or NOT IN with an explicit WHERE provider_id IS NOT NULL inside the subquery. Both work; NOT EXISTS is the one that survives someone later making a second column nullable.
Filter the cohort on provider_approved_at_utc within the month and provider_status = 'approved', so suspended and deactivated accounts do not inflate what will be read as an onboarding failure.
Compute days since approval as a date difference on the approval timestamp, and sanity check the result size against the cohort size before drawing any conclusion.

Worked solution (20 min)

Run SELECT COUNT(*) FROM fct_request WHERE provider_id IS NULL. A non-zero count is the proof that NOT IN is unusable against this column.
Write the cohort CTE and count it, so the denominator is known before any filtering.
Add the two NOT EXISTS clauses one at a time and record the row count after each; which one does most of the work is itself the finding.
Order by provider_approved_at_utc ascending and add the days-since column.

Expected result: One row per never-active approved provider from the month, with user_id, provider_approved_at_utc and an integer days-since-approval, ordered oldest first. The row count is strictly below the approval cohort size and at most the count of cohort providers with zero supply sessions, since the match condition only removes further rows.

Verify it yourself

Swapping NOT EXISTS for NOT IN on provider_id returns zero rows. If it returns the same rows, provider_id has no NULLs and the table does not match the stated schema.
Every returned user_id is absent from SELECT DISTINCT provider_id FROM fct_supply_session.
The result count is at most the cohort count of providers with no supply session at all.

Follow-up

The list is 40 percent of the approval cohort. Is that an onboarding failure or a data problem, and which single query settles it?
How would you separate providers who never went online from providers who went online and were never offered anything, given that fct_request only records the provider who actually matched?

If we notice a 5% drop in conversion rate on the checkout page, how would you go about diagnosing the root cause?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you balance the trade-off between delivery speed and delivery cost in a surge-demand scenario?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize your work when you have multiple competing requests from different teams?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you design a metric to measure the success of a new "instant delivery" feature?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

What are the most common pitfalls when running experiments in a two-sided marketplace?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

How do you interpret results when your A/B test shows statistical significance but the practical impact is negligible?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

Decide a price cut when guardrail and north star diverge

Hard · trade-offs · guardrails · supply elasticity · unit economics

A market proposes cutting base_price_cents by 10%, holding the take rate fixed so provider_payout_cents per completed order falls 10% too. The forecast is requests +18%, completed orders +12%, and online hours roughly flat for the first four weeks. Completed orders per active consumer rises; provider earnings per online hour falls unless volume per hour compensates. State the exact breakeven for the guardrail, give the decision rule you would apply, and name the evidence from the first two weeks that would reverse it. Supply responds over weeks, not days.

Approach

Write the guardrail as an identity before touching the forecast: provider earnings per online hour = (completed orders per online hour) x (earnings per completed order). Any argument about this trade-off is an argument about which factor moves more, so make the identity explicit.
Solve the breakeven: earnings per order falls to 0.90 of baseline, so completed orders per online hour must reach 1 / 0.90 = 1.111, a rise of 11.1%, to hold hourly earnings flat.
Apply the forecast: orders +12% with flat online hours gives 1.12 x 0.90 = 1.008, so hourly earnings rise about 0.8%. The forecast clears the breakeven by roughly one point, which is inside forecast error, so the correct statement is that the guardrail is neutral rather than positive.
Run the sensitivity that decides the question: if online hours rise 5% because providers respond to higher volume, orders per online hour becomes 1.12 / 1.05 = 1.0667, and 1.0667 x 0.90 = 0.96, a 4% fall in hourly earnings. The sign of the guardrail is set by the supply response, not by the demand response.
State the decision rule in a form someone can act on: proceed only with a pre-registered supply floor (for example, halt if earnings per online hour falls more than 3% against the matched control for two consecutive weeks) and only if the cut is tested in a subset of markets rather than launched, because a four-week read on a supply intervention measures the demand response and almost none of the supply response.
Close the escape hatch explicitly: defending hourly earnings with provider incentives converts a pricing decision into a spend decision, and that spend lands in contribution margin per completed order, which is the guardrail on the other metric. Say this before someone proposes it.

Worked solution (35 min)

Write the earnings-per-hour identity and confirm both factors are computable from fct_supply_session and fct_order.
Compute the breakeven: 1 / 0.90 = 1.111, so +11.1% orders per online hour.
Evaluate the central forecast: 1.12 x 0.90 = 1.008, hourly earnings +0.8%.
Re-run with online hours +5%: 1.12 / 1.05 = 1.0667; 1.0667 x 0.90 = 0.96, hourly earnings -4.0%.
Write the decision rule with a numeric halt condition and the measurement horizon it requires.
Add the note that an incentive backstop moves the cost into contribution margin per completed order rather than removing it.

Expected result: Breakeven stated as +11.1% completed orders per online hour. Central case +0.8% hourly earnings, which is neutral inside forecast error. With online hours +5% the guardrail turns to -4.0%. Decision: test in a subset of markets with a pre-registered halt at a 3% hourly-earnings shortfall over two consecutive weeks against matched controls, and read the supply response at eight weeks rather than four.

Verify it yourself

1 / 0.90 = 1.1111 and 1.12 x 0.90 = 1.008 both reproduce exactly.
1.12 / 1.05 x 0.90 = 0.96 reproduces exactly, and you can state in one sentence why more online hours make the guardrail worse rather than better.
The decision rule contains a number and a horizon, not a direction.
The incentive backstop is priced against contribution margin rather than treated as free.

Follow-up

Providers respond over weeks. What would you measure at week eight that is uninformative at week two?
The market wants to hold hourly earnings flat with a temporary bonus. What does that do to the payback calculation, and how would you report it?
How would you randomise this given that price is market-visible and providers talk to each other?

Requests fell four percent on one local day only

Easy · seasonality · timezone · day-of-week

Daily requests in one market fell 4% against the previous day and recovered the day after. The dashboard groups fct_request by DATE(requested_at_local), and dim_market.timezone holds the market's IANA zone. Before anyone opens an incident, rule out calendar and clock causes. Deliverable: the ordered checks you run, the arithmetic for the largest mechanical cause you find, and the residual threshold at which you would still escalate.

Approach

Compare like weekday to like weekday. Marketplace demand has a strong weekly cycle, so a day-over-day comparison that crosses a weekend boundary is not a signal at all. Use the same weekday for the trailing four to eight weeks as the baseline.
Read the day's length from the zone, not from the row labels. requested_at_local is a naive wall-clock timestamp, so on an autumn fall-back day the repeated hour lands on the same label twice: COUNT(DISTINCT date_trunc('hour', requested_at_local)) returns 24 for a 25-hour day, never 25. That count can only ever catch spring-forward, and even then only in a market busy enough that every surviving hour has at least one request. Compute the length from dim_market.timezone instead - ((d + 1)::timestamp AT TIME ZONE m.timezone) - (d::timestamp AT TIME ZONE m.timezone) returns 23, 24 or 25 hours for local date d, does not depend on volume, and still returns 23 or 25 in the zones whose transition falls at local midnight, where the non-existent instant resolves forward.
Size the clock effect properly. The shortfall is not one hour in 24, which would be 4.2%, but the share of daily requests that normally falls in the hour that disappeared, read off the trailing same-weekday hourly profile. In most zones the transition lands overnight, so the true mechanical effect is usually well under one percent. A fall-back day runs the other way, and its data-side fingerprint is the duplicated label carrying roughly twice its usual share - which is the only way that case shows up at all if you are counting distinct local hours.
Check the local calendar for the date: public holidays, school terms, large scheduled events, severe weather. These move the whole day rather than a single hour, which is how you tell them apart from a clock change.
Escalate only the residual. Subtract the weekday effect and the clock effect, then express what is left in units of the same-weekday standard deviation over the trailing eight occurrences, and give the threshold you are using.

Follow-up

The same 4% drop lands on the same local date in five markets across three timezones. What do you check first now?
The product dashboard groups by local date and the finance report by UTC date. What reconciles the two?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Rigour against a deadline

Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.

How do you handle network effects when testing a feature that impacts both buyers and delivery partners?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Choose between three teams' requests with one analyst week

Medium · prioritisation · stakeholder management · unit economics

Three requests arrive the same morning and you have one analyst-week. Pricing wants a fee elasticity refresh for a change scheduled in ten days. Supply wants a churn model for approved providers who never opened a session. Finance wants the monthly contribution-margin restatement that attributes refunds and chargebacks in fct_money_movement to the order's completion month rather than the posting month. Deliverable: your ranking, the criterion behind it, the smallest useful version of each, and what you say to the two teams you rank below first.

Approach

Rank on decision coupling rather than requester seniority or intrinsic interest: what decision hangs on this, on what date does it become useless, and how expensive is it to reverse if the answer is wrong.
Notice the dependency before the ranking: the margin restatement changes the denominator of every unit-economics answer, including the elasticity work, so doing it second means redoing part of the first task.
Unbundle each request into its smallest decision-bearing piece. The restatement is a change to the attribution date in one query, about half a day. The elasticity refresh needs a range and its preconditions, about two days. The churn model is the only item with no date and the longest build.
Replace the churn model with the descriptive cut that may make it unnecessary: approved-provider time-to-first-session conversion by approval cohort and acquisition_channel, from dim_user provider_approved_at_utc against the first fct_supply_session.online_at_utc, one day. If conversion collapses in one channel or one market, the fix is operational and no model is needed.
Communicate the ranking in writing where all three can see it, with the reason and the trigger that would reorder it, and give each deprioritised team a smaller concrete deliverable rather than a place in a queue.

Follow-up

The supply lead escalates to your manager - what do you say, and what do you not say?
What if the fee change's elasticity cannot be estimated observationally over the range they need?
Which of the three would you drop entirely if you lost two days to an incident?

Explain a switchback confidence interval to a non-technical executive

Easy · communicating uncertainty · switchback · decision framing

A switchback test of a dispatch-radius change ran 1,152 market-hour blocks across six markets. SLA fill rate moved +1.8 percentage points, 95% interval [-0.4, +4.0], variance clustered at the block. Those markets serve about 250,000 eligible requests a week at 88% fill and 93% completion. An executive with no statistics background wants a ship-or-wait answer inside a five-minute update. Deliverable: the two-minute spoken explanation, your recommendation, and the single condition that would change it. You may not use the words significant, p-value, or confidence interval.

Approach

Open with the decision and the recommendation, then justify; an executive who hears the caveat first stops listening before the ask arrives.
Translate both interval bounds into the unit the executive already manages: eligible requests times percentage points times completion rate gives weekly completed orders, so the range becomes 'between about 1,000 fewer and about 9,300 more completed orders a week, best single guess about 4,200 more'.
Say plainly what the range does and does not rule out: it does not rule out a small loss, and it is wide because the test has 1,152 effective units, not 250,000 consumers. Block-level randomisation is the reason the sample is small, and it is the reason the number is trustworthy at market level.
Price the two errors against each other: a reversible dispatch parameter with a bounded downside is cheap to ship and cheap to revert, so the decision rule is not 'is the effect proven' but 'is the worst case affordable and detectable'.
End with the one condition that flips you: name the monitoring metric (provider utilisation and idle time, since a wider radius can raise fill by burning provider hours) and the threshold at which you revert.

Follow-up

How many more weeks of blocks would it take to halve the width of that range, and is that worth the delay?
The executive asks 'so is it real or not' - what do you say without reaching for statistical vocabulary?
What would you monitor post-ship that the experiment itself could not measure?