Interview guides · Bhatia Brothers

Bhatia Brothers Data Scientist Interview Questions & Guide 2026

SQL practice for Bhatia Brothers data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.

The 60-second brief

The role of a Data Scientist at Bhatia Brothers is pivotal in driving data-informed decision-making across the organization. As a Data Scientist, you are expected to leverage advanced analytics, statistical modeling, and machine learning techniques to extract meaningful insights from large datasets. Your work directly influences product development, enhances user experiences, and optimizes operational efficiencies, making your contributions invaluable to the company's growth and strategic direction.

Allocate prep to your weakest link rather than your favourite topic. Of the three things that usually gate the outcome (SQL that is correct under messy joins, sound reasoning about experiments, and structured framing of an open-ended problem), candidates tend to over-invest in modelling theory and under-invest in framing.

Focus areas

Separate novelty effects from durable behaviour change
Size an experiment before anyone launches it
Turn a vague request into a measurable question

Overview

At Bhatia Brothers, Data Scientists are integral to cross-functional teams that address complex business challenges, ranging from customer behavior analysis to predictive modeling for inventory management. You will be involved in projects that not only showcase your technical expertise but also allow you to collaborate with various stakeholders, ensuring that data-driven insights align with business objectives. This role is critical as it combines technical acumen with strategic thinking, providing you with the opportunity to impact products and services that resonate with users.

The Bhatia Brothers interview process

Bhatia Brothers candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Application Screening

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Preliminary Call

An extra round usually exists because something is still open after the standard loop: a skill the earlier interviews did not sample, a level decision, or two interviewers who disagreed. It is rarely a rerun of what you already did well. Ask the recruiter who you are meeting, what function they sit in, and how long the session runs. That is an ordinary scheduling question, and the answer changes what you should prepare. What separates a strong candidate here is treating the round as a fresh evaluation with its own bar, rather than assuming earlier performance carries you through or sinks you.

What to demonstrate

Whether you can answer well on ground the earlier rounds did not cover, without leaning on what you already said to someone else
Consistency of the facts in your stories: the same sample size, timeframe, team size and scope of your own role as in earlier conversations
How you handle an unfamiliar format live, including whether you ask what kind of answer is wanted before producing one

How to prepare

Ask the recruiter for the interviewer's function, the length, and whether to expect a coding surface, a discussion, or a presentation. Preparing for a 30 minute conversation with a partner team is not the same work as preparing for a 60 minute technical block.
Write out what each earlier round actually covered, then list the two or three areas nobody probed. That gap is the most likely subject of the extra round.
Re-read the numbers in the project stories you have already told, so a second telling does not quietly contradict the first.

Technical Assessment

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Interviews with Team

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Final Interviews

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Reading a pooled rate that moved because the mix moved, not because any behaviour changed

A pooled conversion rate is a weighted average, and a shift in the weights can move it in the opposite direction to every one of its parts. A paid campaign that brings low-converting traffic drops overall signup conversion even if desktop, mobile web and app conversion each rose that week, which is Simpson's paradox and it is the single most common cause of an inexplicable dashboard move. The discipline is to decompose before explaining: recompute the rate holding last period's segment weights fixed, and compare that counterfactual to the actual, so the mix effect and the rate effect are separated numerically rather than argued about. Segment on the dimensions that actually reweight, which in this domain are almost always device_type, referrer_channel, country and new versus returning.

Slicing a flat experiment until a segment reaches significance

Testing one metric across twenty segments at a nominal 5% level produces a significant result about two thirds of the time when nothing is happening anywhere, and the segment that surfaces is by construction the one with the most favourable noise. The reported effect in that slice is then badly overstated, because selection on significance conditions the estimate on being large. What makes it dangerous rather than merely wrong is that a post-hoc segment always has a plausible story attached, so it survives the meeting. The controls are declaring the small number of segments of interest before launch, correcting across the ones tested, and treating anything discovered afterwards as a hypothesis that needs its own adequately-powered test rather than a finding.

Naming a model class before naming the deployment constraints

Set out the latency budget, the label delay, the retraining cadence, the interpretability requirement and the number of labelled examples, then pick the model that fits them. A boosted-tree answer to a problem where each decision must be explained to the affected user is a well-executed answer to the wrong question.

Dropping rows with missing values without naming the mechanism

Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.

Questions & practice

Discuss how you would balance model accuracy with interpretability.

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Sessionise an event stream with gap and midnight rules

Hard · sessionisation · vectorisation · event streams

Sessionise a raw event stream. Input: a DataFrame with visitor_id, user_id (often NULL), occurred_at_utc and event_name, unsorted, up to 5 million rows. A session breaks when the gap from that visitor's previous event exceeds 30 minutes, and is force-closed at UTC midnight so no session spans two calendar dates. A gap of exactly 30 minutes does not break. Emit one row per session with session_id, visitor_id, the user_id as of the last event in the session, started_at_utc, ended_at_utc, session_date, duration_seconds and event_count. Vectorise; do not loop per visitor.

Approach

Sort by ['visitor_id', 'occurred_at_utc', 'event_id'] once, then express the whole problem as one boolean vector: a row starts a new session when the visitor changed, or the gap exceeds 30 minutes, or the UTC date differs from the previous row's UTC date. Cumsum that vector and you have the session key.
Get the comparison direction right on the gap: the rule is strictly greater than 1800 seconds, so an event at exactly 1800 seconds continues the session. Write it as gap > pd.Timedelta(minutes=30), and make the tie a test case rather than an assumption.
Derive the midnight break from the date change, not from inserting synthetic boundary rows. A date change implies a break even when the gap is two seconds, which is precisely the force-close rule and is why the two conditions are ORed rather than one subsuming the other.
Aggregate with a single groupby on the session key: min and max of occurred_at_utc, size for event_count, and last for user_id, which is correct because the frame is already sorted so 'last' is the final event in the session. That is the identity-as-of-session-end rule.
Compute duration_seconds as (max - min).dt.total_seconds(), which makes a single-event session 0 seconds. Say so explicitly, because a downstream mean session duration is sensitive to whether single-event sessions are 0 or excluded.

Follow-up

Sessions are used as the denominator of a conversion rate. How does moving the inactivity gap from 30 to 45 minutes move that rate, and in which direction?
A visitor's clock is 40 minutes ahead, so their events arrive with future occurred_at values. What does your sessioniser do, and what would you rather it did?
The same person signs up mid-session on mobile and continues on desktop. How many sessions and how many users does your output show, and is that the right answer?

Permutation test for a difference in conversion rates

Medium · permutation test · resampling · inference

Write a two-sided permutation test from scratch for a difference in conversion rates, using no scipy hypothesis function. Input: a DataFrame with unit_id, variant in {control, treatment} and converted in {0,1}, one row per randomisation unit. Compute the observed difference in proportions, then build the null distribution by reshuffling the variant labels while holding each arm's size fixed. Report the p-value as (1 + the count of permuted statistics at least as extreme in absolute value) / (B + 1) with B at least 10,000, and return the permutation distribution.

Approach

Name the null being tested: the sharp null that each unit's outcome is the same under either label. That is what licenses permuting labels, and it is stronger than the null of equal means, which matters when someone asks whether the test is valid under unequal variances.
Extract converted to a single numpy array of 0s and 1s and record n_treatment. Every permutation is then just a reshuffle of one array, and the treatment mean is the mean of the first n_treatment entries of the shuffled array.
Vectorise the B permutations with rng.permuted on a tiled 2-D array, or with argsort of a (B, n) random matrix. A Python loop calling np.random.shuffle B times is correct but roughly an order of magnitude slower and often runs past the time limit.
Use the +1 correction in both numerator and denominator. Without it a p-value of exactly 0 is reportable, which is false: the observed labelling is itself one of the permutations, so the smallest attainable p-value is 1/(B+1).
Compare the resulting p-value against a two-proportion z-test as a sanity check. At these sample sizes they should agree closely; a large divergence means the statistic or the shuffle is wrong, not that the permutation test found something subtle.

Worked solution (25 min)

y = df['converted'].to_numpy(); n_t = (df['variant'] == 'treatment').sum(); obs = y[treat_mask].mean() - y[~treat_mask].mean().
Build the null: for B draws, shuffle y and take the mean of the first n_t entries minus the mean of the rest.
p = (1 + (np.abs(null) >= abs(obs) - 1e-12).sum()) / (B + 1), with the small tolerance so exact ties count as at least as extreme.
Return obs, p and the null array; plot or describe the null to confirm it is centred at 0.

Expected result: The observed difference, a p-value in [1/(B+1), 1], and a null distribution whose mean is approximately 0 and whose standard deviation approximately matches the analytic pooled standard error sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2)).

Verify it yourself

The null distribution's mean is within a few Monte Carlo standard errors of 0; if it is centred on the observed difference, the code bootstrapped instead of permuted.
The null's standard deviation agrees with the analytic pooled standard error to within a couple of percent.
The p-value agrees with a hand-computed two-proportion z-test p-value to within about 0.01 at these sample sizes.
Setting converted identical in both arms by construction yields a p-value near 1, and a planted large effect yields the floor of 1/(B+1).

Follow-up

The arms are 200 and 20,000 units. Does the permutation test stay valid, and what happens to its resolution at B = 10,000?
Give a 95 percent confidence interval for the difference. Can you get it from this permutation distribution, and if not, what would you run instead?
The randomisation unit is user_id but the outcome is per session. What breaks, and what is the fix?

Write a function to calculate the mean and median of a list in Python.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Can you demonstrate how to optimize a given SQL query?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Paying accounts with no active seat in 28 days

Medium · anti-join · null-semantics · retention-risk

dim_account holds account_id, account_type, lifecycle_status, seats_licensed. fct_event holds account_id, user_id, occurred_at_utc, is_core_action, and its account_id is NULL for every signed-out and pre-signup event. Find accounts with lifecycle_status = 'active' and account_type <> 'internal' that had no distinct user complete a core action in the trailing 28 days. Return account_id, seats_licensed and days since that account's most recent core action, with NULL where the account has never emitted one. Order by seats_licensed descending.

Approach

Build the recent-activity set first: fct_event rows with is_core_action = TRUE, occurred_at_utc >= now() - interval '28 days', and an explicit account_id IS NOT NULL. Making the NULL exclusion explicit in the CTE is what lets you reason about the anti-join afterwards.
Express the exclusion with NOT EXISTS (correlated on account_id) or a LEFT JOIN with an IS NULL guard. Do not use NOT IN against this column: it is nullable, and SQL's three-valued logic turns the whole predicate UNKNOWN, returning zero rows.
Compute last-seen separately as MAX(occurred_at_utc) per account over all history, LEFT JOINed on, so an account that has never emitted a core action (NULL) is distinguishable from one that went quiet six weeks ago. Those two cases have different causes and different owners.
Rank by seats_licensed, or better by the account's current mrr_cents_constant_fx if you are allowed the subscription table, because a silent fifty-seat account is a renewal conversation and a silent one-seat account is noise.
Before shipping, check whether the never-seen group is a cluster by signup date or surface. A block of accounts with no events at all is usually an instrumentation gap, not a set of customers who stopped using the product.

Worked solution (25 min)

Count NULL account_id rows in fct_event over the window so you know the trap is live in this data rather than hypothetical.
Write the active-account spine and the 28-day activity CTE.
Write the anti-join with NOT EXISTS, then deliberately run the NOT IN version and record that it returns zero rows.
Add the all-time MAX(occurred_at_utc) LEFT JOIN and derive days_since as a date difference.
Split the output into 'quiet' and 'never seen' and eyeball the never-seen group for a shared signup window or surface.

Expected result: One row per silent active account with seats_licensed and days_since_last_core_action, ordered by seats_licensed descending. Accounts that never emitted a core action show NULL and are reported as a separate group rather than as the worst churn risks.

Verify it yourself

Silent accounts plus accounts with at least one active user equals the total count of active non-internal accounts.
The NOT IN variant returns zero rows, which is the demonstration that a NULL is present.
No account_id appears twice in the output.
Every days_since_last_core_action is either NULL or greater than 28.

Follow-up

How would you distinguish a genuinely idle account from one whose events lost their account_id after an instrumentation change?
Would you count on fct_event.account_id or resolve user_id through dim_user instead, and what does each choice miss?
Licensed-seat utilisation is the continuous version of this. How would you turn this boolean into that ratio?

If you found inconsistencies in a dataset, how would you address them?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you approach building a recommendation system for our products?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you prioritize your tasks when working on multiple projects?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

You are given a dataset with sales information. What steps would you take to analyze it?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

Describe how you would design an A/B test for a new feature on our platform.

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Halve the runtime of a flat test with pre-period data

Medium · cuped · variance reduction · stratification

A test on 28-day core actions per user is powered for six weeks and the team will not wait. You have fct_event (user_id, occurred_at_utc, is_core_action) covering the period before each unit's first_exposed_at_utc in fct_experiment_exposure, and the pre-period count correlates with the in-experiment count at r = 0.6. Apply CUPED, state precisely how much runtime it buys, and handle the units with no pre-period because they signed up after the experiment started.

Approach

Define the adjusted outcome: Y_cuped = Y - theta (X - Xbar), with theta = Cov(Y, X) / Var(X) estimated on the pooled sample so theta itself carries no treatment effect. It is unbiased because X is pre-exposure and therefore has equal expectation in both arms.
Quantify the gain exactly: Var(Y_cuped) = Var(Y)(1 - r^2) = Var(Y) x 0.64, so 36% of the variance is removed. Required sample scales with variance, so six weeks becomes 3.8 weeks, and at the original sample the MDE shrinks by sqrt(0.64) = 0.8.
State the precondition that actually bites: X must be measured strictly before first_exposed_at_utc, not before assigned_at_utc and not before the calendar start of the experiment. Any post-exposure information in the covariate lets the treatment effect leak in and biases the estimate rather than merely failing to help.
Handle missing pre-periods as a stratum, never by dropping rows. Dropping changes the population; setting X to the pooled mean leaves those units' adjustment at zero, which is unbiased but buys them no reduction, so report the covered share and expect the blended reduction to fall short of 36%.
Note that stratification is the same mechanism with a categorical covariate. Post-stratifying on device_type and referrer_channel removes the variance those stratum means explain, and it composes with CUPED provided the strata are also fixed before exposure.

Worked solution (30 min)

Build the covariate: for each unit_id in fct_experiment_exposure, count fct_event rows with is_core_action = TRUE and occurred_at_utc in [first_exposed_at_utc - 28 days, first_exposed_at_utc).
Estimate theta on the pooled sample as Cov(Y, X) / Var(X), and confirm Xbar is statistically indistinguishable between arms before using it.
Form Y_cuped = Y - theta (X - Xbar) and run exactly the test you would have run on Y.
For units with no pre-period, set X to the pooled mean so their adjustment is zero, and report what share of the sample that is.
Report unadjusted and adjusted estimates side by side, with the interval narrower on the adjusted one.

Expected result: 1 - 0.6^2 = 0.36 of the outcome variance is removed, so the required sample falls to 64% of the original and the six-week plan becomes about 3.8 weeks. At the original sample the MDE shrinks by a factor of 0.8. The blended reduction is smaller in proportion to the share of units with no pre-period, and that share is stated alongside the result.

Verify it yourself

Adjusted and unadjusted point estimates should be close. CUPED reduces variance, it does not move the effect, so a large shift means X was imbalanced at baseline and needs investigating rather than celebrating.
The realised standard error ratio should be near sqrt(1 - r^2) using the r observed in this experiment, not the historical r.
Var(Y_cuped) must be no larger than Var(Y) for the fitted theta. A larger value means theta has the wrong sign or was estimated on the wrong window.

Follow-up

Here the covariate is the same metric as the outcome. When would you deliberately pick a different one, and what is the risk if that covariate is itself affected by treatment?
The point estimate moves by 0.4 standard errors when CUPED is applied. Is that reassuring or alarming, and what would you check?
How does CUPED combine with cluster randomisation on account_id, and at which grain is the covariate built?

Tell a pipeline outage from a collapse in usage

Medium · pipeline-failure · instrumentation · event-volume

Weekly active accounts completing a core action fell 9%, and almost the entire fall sits in accounts whose events carry surface = 'ios'. App crash rates and store reviews are unchanged. You have fct_event with occurred_at_utc, received_at_utc, event_name, is_core_action, app_version and surface, plus fct_session and the ingestion job run log. Establish within the hour whether iOS engagement fell or iOS events stopped arriving, name the evidence that distinguishes them, and say what you would publish on the dashboard in the meantime.

Approach

Compare the event-name composition inside surface = 'ios' against the prior four weeks as shares, not counts. A behaviour collapse scales most event names together; a dropped event definition or a broken downstream filter hits specific event_name values while page_view and session-opening events hold steady. That shape difference is the fastest discriminator available.
Profile the received_at_utc minus occurred_at_utc distribution per day for surface = 'ios'. A stalled-then-backfilling pipeline shows a fat upper tail and a recovering p99; a silently dropped stream shows an unchanged lag distribution over a smaller volume. The two failure modes have different remedies and different histories.
Cut by app_version. A logging SDK change arrives with one build and ramps with its adoption curve; an infrastructure fault arrives across every build within the same hour. Checking this costs one group-by and rules out half the hypothesis space.
Cross-check against a signal that does not travel the suspect path: server-emitted events with session_id NULL, and subscription or billing activity for the same accounts. If those accounts are still transacting, the users did not leave.
Publish an ex-iOS total with an explicit annotated break rather than a blended total. A blended number during a known ingestion gap is wrong in a direction you can already name, and republishing it daily spreads the artefact into every downstream report.

Follow-up

Suppose the events do eventually backfill. What is your policy for restating the published weekly numbers, and who needs to be told?
What monitor would have caught this before a human noticed the weekly metric, and what would it alert on?
If is_core_action is maintained in the tracking plan, what governance would stop a change to that list from silently moving a north-star metric?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

Describe a situation where you had to persuade a team to adopt your data-driven recommendation.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that project again?
What did you decide not to do, and why?

Turn an ambiguous onboarding question into a measurable metric

Easy · scoping · metric definition · stakeholder

Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.

Approach

Recognise what is being probed: whether you convert a goal into a computable predicate without stalling for requirements or guessing in silence. Listing clarifying questions is the generic answer; shipping a defensible default alongside them is the strong one, because the review is in two days and it will happen with or without you.
Infer the decision behind the request. A question about whether onboarding works, arriving before a planning cycle, usually means whether to staff it next quarter. That points at a rate with visible headroom over several cohorts, not at a descriptive dashboard.
Write the three questions so that each one changes the SQL. Which population, all signups or only self-serve from dim_user.signup_surface. What counts as working, reaching a core action or completing the onboarding flow_id. Against what bar, last quarter's cohorts or a stated target.
Propose the default explicitly: seven-day activation on weekly signup cohorts. Numerator, users with is_core_action = TRUE events on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). Denominator, the signup cohort with is_internal = FALSE. Publish with an eight-day lag, and state that the two-distinct-days threshold is a frozen choice rather than a discovery.
Name the exclusion in the same breath as the number. The series shows whether users activate; it does not establish that onboarding caused the level, which needs a staged rollout or an experiment.

Follow-up

The director replies that they meant the onboarding flow specifically, not activation. What changes in the query and in the caveats?
Your cohort metric needs an eight-day lag and the review is in two days. What do you present, and how do you label it?
Two of your three questions come back unanswered. Which one do you refuse to proceed without?

Explain a wide interval to a non-technical executive

Medium · communication · uncertainty · pricing

A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.

Approach

Recognise what is being probed: whether you can carry uncertainty into a decision instead of either hiding it or hiding behind it. The generic answer promises to explain the interval in plain English; the strong one replaces the question 'what is the number' with 'across this range, where does the decision change'.
Find the threshold before you draft anything. Ask what the pricing case assumes, then compute the conversion drop at which the higher price stops adding revenue: price uplift on the conversions kept against the revenue lost from conversions forgone. That single figure is what makes the range legible.
Restate the estimate and both bounds in the unit the audience already reasons in. Convert percentage points into monthly first-paid conversions at current trial volume, then into mrr_cents_constant_fx, so the slide reads as money per month rather than as statistics.
Place the range against the break-even and say which part of it sits on each side. If most of the range clears the threshold, that is a recommendation to proceed with a monitoring plan; if the range straddles it, that is a recommendation to narrow the range first.
Name what would narrow it and what that costs in weeks, then give one recommendation with an explicit condition for revisiting it. Uncertainty stated without a next step is read as indecision and the midpoint gets used anyway.

Follow-up

The executive says to give the midpoint and they will manage the risk. What do you do?
How does the slide change if the interval were a 4.6pp to 0.2pp drop, with no positive outcomes in range?
Why is a geo holdout the credible read here rather than the attributed channel numbers you already have?