cswork / libraryOpen practice workspace

Interview guides · Fetch

Fetch Data Scientist Interview Questions & Guide 2026

Data scientist prep for Fetch candidates: drill A/B design, power and minimum detectable effect on original exercises.

The 60-second brief

As a Data Scientist at Fetch, you play a pivotal role in transforming raw data into strategic insights that directly influence business decisions and product development. This role is crucial for driving data-driven solutions that enhance user experience, optimize operations, and contribute to Fetch's overall mission of improving customer engagement through actionable analytics. You will work with vast datasets to uncover trends, build predictive models, and support various teams in implementing data-driven strategies.

In modelling rounds the live question is usually why this model class for this problem, and how you would know six weeks after launch that it is still working. Deriving gradients by hand is rarely what is being probed.

Focus areas

  • Separate novelty effects from durable behaviour change
  • Pick a randomisation unit that respects interference
  • Turn a vague request into a measurable question

Overview

The impact of your work as a Data Scientist extends beyond just number crunching; it touches upon every aspect of Fetch's offerings. You will collaborate with cross-functional teams, including product management, engineering, and marketing, to shape the direction of new features and initiatives. You'll also be involved in solving complex problems that require innovative thinking and a deep understanding of both statistical techniques and the business landscape. This blend of technical skill and strategic influence makes the Data Scientist role at Fetch both challenging and rewarding.

The Fetch interview process

Fetch candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

  • Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
  • Whether you name what you have not done instead of stretching to cover every line of the posting
  • Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

  • Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
  • Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
  • Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Evaluations

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

  • Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
  • Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
  • Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

  • Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
  • Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
  • If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Team Discussions

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

  • Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
  • Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
  • How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

  • Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
  • For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
  • Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Where candidates lose points

Reading a pooled rate that moved because the mix moved, not because any behaviour changed

A pooled conversion rate is a weighted average, and a shift in the weights can move it in the opposite direction to every one of its parts. A paid campaign that brings low-converting traffic drops overall signup conversion even if desktop, mobile web and app conversion each rose that week, which is Simpson's paradox and it is the single most common cause of an inexplicable dashboard move. The discipline is to decompose before explaining: recompute the rate holding last period's segment weights fixed, and compare that counterfactual to the actual, so the mix effect and the rate effect are separated numerically rather than argued about. Segment on the dimensions that actually reweight, which in this domain are almost always device_type, referrer_channel, country and new versus returning.

Comparing cohort retention curves of different maturities, or building the curve from users who are still present

A cohort four weeks old has no week-8 value, so an average taken across cohorts silently drops young cohorts from the later columns and keeps them in the earlier ones. The curve then bends upward at the tail, and the reading that 'retention is improving over time' is an artefact of which cohorts survived to be measured. The same error appears in the denominator when retention is computed over users active in the current period rather than over the full original cohort, which conditions on survival and guarantees a flattering number. The fix is a triangle: fix the cohort at signup, bound every window on both sides, and only compare cells where every cohort has had the full elapsed time, publishing the rest as blank rather than as a partial average.

Over-explaining the method and under-explaining the implication

Lead with the answer and what you would do about it, then give the approach when asked. Roughly one sentence of method per three of implication is the right ratio for a stakeholder-facing answer; the interviewer already knows what a regression is.

Explaining an aggregate move without decomposing the mix shift

Split the change in the aggregate into within-segment movement and movement in segment weights before you explain it. Every segment's rate can fall while the overall rate rises, purely because volume shifted toward segments that already had higher rates.

Questions & practice

Explain the time complexity of sorting algorithms you are familiar with.

Medium · machine learning and modelling

Approach

  1. Set a baseline first, so any model has something honest to beat.
  2. Say how the offline result would be validated online before it is trusted.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • How would you choose the decision threshold, and who owns that choice?
  • What would you monitor after launch to know the model is still valid?

Simulate the false positive cost of repeated peeking

Medium · simulation · peeking · type i error

Quantify the cost of peeking. Simulate a two-arm experiment with no true effect: each arm accumulates Bernoulli conversions at a base rate of 0.10 up to 40,000 units per arm. Run a two-sided two-proportion z-test at alpha 0.05 at ten equally spaced interim points, and record whether the test ever crossed. Report the false positive rate over at least 10,000 replications, alongside the rate for a single look at the final sample only. Use a fixed seed and report a Monte Carlo standard error on both figures.

Approach

  1. Generate each replication as two cumulative sums of Bernoulli draws, then read the interim points off the cumulative arrays. Regenerating data at each look would make the looks independent, which destroys exactly the dependence the exercise is about: later looks share data with earlier ones.
  2. Use the pooled-variance two-proportion z: p_pool = (x1+x2)/(n1+n2), z = (p1-p2) / sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2)), reject when |z| > 1.96. State that the normal approximation is fine here because the smallest look has roughly 400 expected conversions per arm.
  3. Vectorise across replications rather than looping: draw a (reps, n) array of uniforms, threshold at 0.10, cumsum along axis 1 and slice the ten look indices. A per-replication loop at 10,000 by 40,000 is unnecessarily slow.
  4. Record the any-cross indicator per replication, take the mean, and compute the Monte Carlo standard error as sqrt(p*(1-p)/reps) so the reported figure comes with its own precision.
  5. Report the single-look rate in the same run as a control. If it does not land near 0.05, the bug is in the test statistic and not in the peeking argument.

Follow-up

  • Re-run with 40 looks instead of 10. Why does the curve flatten rather than continue rising linearly?
  • Among the replications that crossed, what is the mean observed lift, and why is it not zero?
  • What does an O'Brien-Fleming boundary or an always-valid confidence sequence change about this simulation, and what does each cost in power?

Cluster bootstrap for a per-session rate randomised on users

Hard · cluster bootstrap · randomisation unit · variance

An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.

Approach

  1. Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
  2. Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
  3. Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
  4. Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
  5. Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.

Worked solution (35 min)

  1. per_user = df.groupby(['variant','user_id'])['converted'].agg(['sum','size']); split into two arrays per arm.
  2. point = (t_sum.sum() / t_n.sum()) - (c_sum.sum() / c_n.sum()).
  3. For b in range(B): idx = rng.integers(0, len(t_sum), len(t_sum)); ratio_t = t_sum[idx].sum() / t_n[idx].sum(); same for control; store the difference. Vectorise by drawing a (B, n) index matrix if memory allows.
  4. ci = np.percentile(diffs, [2.5, 97.5]); naive_se = sqrt(p_t*(1-p_t)/n_sessions_t + p_c*(1-p_c)/n_sessions_c); naive_ci = point +/- 1.96*naive_se.
  5. width_ratio = (ci[1]-ci[0]) / (naive_ci[1]-naive_ci[0]).

Expected result: A point estimate identical to the full-data ratio difference, a percentile interval containing that point estimate, and a width ratio greater than 1 whose value approximates sqrt(1 + (m-1)*rho) for the data's mean cluster size m and intraclass correlation rho. A ratio of approximately 1.0 means sessions were resampled instead of users.

Verify it yourself

  • Construct data where every user has exactly one session: the cluster and naive intervals then agree to within Monte Carlo error, since m = 1 makes the design effect 1.
  • Construct data where converted is constant within each user: rho is 1, and the width ratio should approach sqrt(m).
  • The bootstrap distribution's mean sits close to the point estimate; a large offset means the resampled arms have drifted in size.
  • Doubling B narrows the Monte Carlo noise in the percentile endpoints without shifting the interval's centre.

Follow-up

  • Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
  • Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
  • Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?

Implement a function in Python to calculate the Fibonacci sequence.

Medium · sql and data manipulation

Approach

  1. State the window function and its partition and ordering out loud before writing it.
  2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

  • How does the query change if the join becomes one-to-many?
  • How would you verify this result without re-running the same query?

How would you optimize a slow-running SQL query?

Medium · sql and data manipulation

Approach

  1. Compute rates by summing numerator and denominator separately, never by averaging rates.
  2. Check whether any join is one-to-many before aggregating, or the sums inflate.
  3. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

  • How does the query change if the join becomes one-to-many?
  • How would you verify this result without re-running the same query?

Seven-day activation rate by weekly signup cohort

Easy · cohorts · activation · date-bounds

dim_user holds user_id, account_created_at_utc, is_internal. fct_event holds user_id, occurred_at_utc, is_core_action. A user is activated when core-action events fall on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). Return, for the last twelve complete weekly signup cohorts, the cohort week, cohort size, activated users and the activation rate. Exclude is_internal users. Every signup in the cohort week stays in the denominator, including users who never returned.

Approach

  1. Start from dim_user as the denominator spine with is_internal = FALSE and DATE_TRUNC('week', account_created_at_utc) as the cohort key. Driving the query from the event table instead would silently condition on having events and delete the entire non-activating population.
  2. Join fct_event on user_id with is_core_action = TRUE and a per-user bound, occurred_at_utc >= u.account_created_at_utc AND occurred_at_utc < u.account_created_at_utc + interval '7 days'. The bound is correlated to each user's own signup timestamp, not a single global date range.
  3. Aggregate per user with COUNT(DISTINCT occurred_at_utc::date) >= 2, then LEFT JOIN that back onto the spine and COALESCE the flag to FALSE so non-activators contribute a zero rather than vanishing.
  4. Restrict the published cohorts to those whose week ended at least eight days ago. A cohort younger than that has not finished its seven-day window, so its rate is mechanically low and reads as a decline.
  5. Roll up by summing the numerator and denominator per cohort week, and state the two-distinct-days threshold next to the number since it is a choice that re-bases the whole history if changed.

Worked solution (20 min)

  1. Write the cohort spine and confirm its total equals the count of non-internal signups in the date range.
  2. Write the per-user distinct-active-days CTE with both interval bounds and inspect a handful of users manually.
  3. LEFT JOIN, COALESCE the flag, aggregate to cohort week.
  4. Apply the eight-day publication lag and drop the incomplete cohort.
  5. Re-run with a closed upper bound (<= +7 days) and note how many users change state, to show the boundary is doing work.

Expected result: Twelve rows, one per cohort week, each with cohort_size, activated and rate. Cohort sizes sum to the total non-internal signup count across those twelve weeks. No cohort is younger than eight days.

Verify it yourself

  • activated is never greater than cohort_size in any row.
  • A user with three core actions on a single day is not counted as activated.
  • Removing the LEFT JOIN in favour of an inner join raises every rate, which demonstrates the denominator was being conditioned on survival.
  • Cohort sizes sum to an independent COUNT(*) over dim_user for the same range.

Follow-up

  • Why two distinct days rather than one event? What happens to the published history if someone changes it to three?
  • Invited seats and SSO-provisioned users get an account_created_at_utc at provisioning and may never sign in. Should they be in this denominator?
  • The rate rose 3 points this week. What do you check before believing it?

Given a dataset with sales data, how would you approach predicting future sales?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

What strategies would you employ to ensure data integrity during analysis?

Medium · product sense and metrics

Approach

  1. State what result would change your recommendation, so the answer is falsifiable.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • Which segment would you cut first, and what would that rule out?

How would you analyze user engagement in the Fetch app to identify potential areas for improvement?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

How would you communicate complex data findings to a non-technical audience?

Medium · product sense and metrics

Approach

  1. State what result would change your recommendation, so the answer is falsifiable.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

Describe how you would structure an A/B test for a new feature in the app.

Medium · a/b testing and experimentation

Approach

  1. Decide the analysis before seeing data, including how long it runs and when you look.
  2. State the primary metric and the minimum effect worth shipping, then size the test.
  3. Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

  • How would you handle interference between treated and control units?
  • What would you conclude if the result is positive but the test is underpowered?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

  1. Clarify what is being asked and what a complete answer would contain.
  2. State your assumptions explicitly before working the problem.
  3. Say what you would check first and why it is the highest-information step.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Halve the runtime of a flat test with pre-period data

Medium · cuped · variance reduction · stratification

A test on 28-day core actions per user is powered for six weeks and the team will not wait. You have fct_event (user_id, occurred_at_utc, is_core_action) covering the period before each unit's first_exposed_at_utc in fct_experiment_exposure, and the pre-period count correlates with the in-experiment count at r = 0.6. Apply CUPED, state precisely how much runtime it buys, and handle the units with no pre-period because they signed up after the experiment started.

Approach

  1. Define the adjusted outcome: Y_cuped = Y - theta (X - Xbar), with theta = Cov(Y, X) / Var(X) estimated on the pooled sample so theta itself carries no treatment effect. It is unbiased because X is pre-exposure and therefore has equal expectation in both arms.
  2. Quantify the gain exactly: Var(Y_cuped) = Var(Y)(1 - r^2) = Var(Y) x 0.64, so 36% of the variance is removed. Required sample scales with variance, so six weeks becomes 3.8 weeks, and at the original sample the MDE shrinks by sqrt(0.64) = 0.8.
  3. State the precondition that actually bites: X must be measured strictly before first_exposed_at_utc, not before assigned_at_utc and not before the calendar start of the experiment. Any post-exposure information in the covariate lets the treatment effect leak in and biases the estimate rather than merely failing to help.
  4. Handle missing pre-periods as a stratum, never by dropping rows. Dropping changes the population; setting X to the pooled mean leaves those units' adjustment at zero, which is unbiased but buys them no reduction, so report the covered share and expect the blended reduction to fall short of 36%.
  5. Note that stratification is the same mechanism with a categorical covariate. Post-stratifying on device_type and referrer_channel removes the variance those stratum means explain, and it composes with CUPED provided the strata are also fixed before exposure.

Worked solution (30 min)

  1. Build the covariate: for each unit_id in fct_experiment_exposure, count fct_event rows with is_core_action = TRUE and occurred_at_utc in [first_exposed_at_utc - 28 days, first_exposed_at_utc).
  2. Estimate theta on the pooled sample as Cov(Y, X) / Var(X), and confirm Xbar is statistically indistinguishable between arms before using it.
  3. Form Y_cuped = Y - theta (X - Xbar) and run exactly the test you would have run on Y.
  4. For units with no pre-period, set X to the pooled mean so their adjustment is zero, and report what share of the sample that is.
  5. Report unadjusted and adjusted estimates side by side, with the interval narrower on the adjusted one.

Expected result: 1 - 0.6^2 = 0.36 of the outcome variance is removed, so the required sample falls to 64% of the original and the six-week plan becomes about 3.8 weeks. At the original sample the MDE shrinks by a factor of 0.8. The blended reduction is smaller in proportion to the share of units with no pre-period, and that share is stated alongside the result.

Verify it yourself

  • Adjusted and unadjusted point estimates should be close. CUPED reduces variance, it does not move the effect, so a large shift means X was imbalanced at baseline and needs investigating rather than celebrating.
  • The realised standard error ratio should be near sqrt(1 - r^2) using the r observed in this experiment, not the historical r.
  • Var(Y_cuped) must be no larger than Var(Y) for the fitted theta. A larger value means theta has the wrong sign or was estimated on the wrong window.

Follow-up

  • Here the covariate is the same metric as the outcome. When would you deliberately pick a different one, and what is the risk if that covariate is itself affected by treatment?
  • The point estimate moves by 0.4 standard errors when CUPED is applied. Is that reassuring or alarming, and what would you check?
  • How does CUPED combine with cluster randomisation on account_id, and at which grain is the covariate built?

Separate a definition edit from a behaviour change

Easy · definition-change · metric-versioning · cohorts

Seven-day activation rate is documented as core actions on at least 2 distinct UTC dates inside the first seven days after account_created_at_utc, and every cohort published through last week was computed on that definition. The dashboard now reads 38% for the signup cohort that started Monday, against 31% for the cohort before it. Two things landed in that week: an onboarding change, and a commit to the metric's SQL that you have not read yet and that came with no documentation update. You have dim_user, fct_event and the metric's version history. Deliver two numbers: how many of the 7 points are behaviour and how many are the definition. Then say which published cohorts are no longer comparable.

Approach

  1. Diff the metric SQL across the commit boundary and enumerate every changed predicate individually: the distinct-day threshold, the is_internal exclusion, the is_core_action source, the window bounds, and the identity key. A single commit routinely changes more than one, and each has its own sign.
  2. Dual-run both definitions over the same 26 cohort weeks. The definition effect is the level shift between the two series computed on identical data, which isolates it from anything that happened in the world.
  3. Read the behaviour effect only from the old-definition series, comparing the new cohort to its own trailing cohorts. The new-definition series cannot answer the behaviour question because it has no pre-period.
  4. Explain the mechanism of the changed predicate rather than only its size. Lowering a distinct-day threshold from 2 to 1 admits every user who acted once in a single session, which is a large and low-intent population, so a several-point jump is the expected magnitude and not a surprise.
  5. State the restatement plan: pick one definition, backfill the full history on it, and annotate the break date. A series carrying two definitions is not a time series.

Follow-up

  • The onboarding team wants the 1-day threshold kept because it is easier to move. What is your argument, and what would change your mind?
  • How would you make a definition change visible to every consumer of this metric without relying on people reading a changelog?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

  • Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
  • Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
  • Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

  • Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
  • On the third attempt, write the grain of every CTE as a comment before writing its body.
  • Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

  • Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
  • Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
  • Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

  • Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
  • Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
  • Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

  • Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
  • Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
  • Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

  • Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
  • Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
  • Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

  • Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
  • Re-read only your own notes from this week, and open no new material.
  • Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

Describe a situation where you had to collaborate with a difficult team member. What was your approach?

Medium · behavioural and stakeholder questions

Approach

  1. Pick a story where you drove the decision, not one where you observed it.
  2. State the situation in two sentences and spend the rest on your reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • What did you decide not to do, and why?
  • What would you do differently if you ran that project again?

Explain a wide interval to a non-technical executive

Medium · communication · uncertainty · pricing

A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.

Approach

  1. Recognise what is being probed: whether you can carry uncertainty into a decision instead of either hiding it or hiding behind it. The generic answer promises to explain the interval in plain English; the strong one replaces the question 'what is the number' with 'across this range, where does the decision change'.
  2. Find the threshold before you draft anything. Ask what the pricing case assumes, then compute the conversion drop at which the higher price stops adding revenue: price uplift on the conversions kept against the revenue lost from conversions forgone. That single figure is what makes the range legible.
  3. Restate the estimate and both bounds in the unit the audience already reasons in. Convert percentage points into monthly first-paid conversions at current trial volume, then into mrr_cents_constant_fx, so the slide reads as money per month rather than as statistics.
  4. Place the range against the break-even and say which part of it sits on each side. If most of the range clears the threshold, that is a recommendation to proceed with a monitoring plan; if the range straddles it, that is a recommendation to narrow the range first.
  5. Name what would narrow it and what that costs in weeks, then give one recommendation with an explicit condition for revisiting it. Uncertainty stated without a next step is read as indecision and the midpoint gets used anyway.

Follow-up

  • The executive says to give the midpoint and they will manage the risk. What do you do?
  • How does the slide change if the interval were a 4.6pp to 0.2pp drop, with no positive outcomes in range?
  • Why is a geo holdout the credible read here rather than the attributed channel numbers you already have?

Disagree with a product manager's roadmap claim using data

Medium · causal inference · selection bias · stakeholder

A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.

Approach

  1. Recognise what is being probed: whether you can separate a number being right from an inference being wrong, and do it without costing the PM face. The generic answer recites that correlation is not causation; the strong one names the specific confound and proposes the cheapest design that could distinguish the explanations.
  2. State the alternative concretely. Accounts that connect an integration in week one are accounts that already have a workflow and a technical owner, so week-one intent plausibly drives both the connection and week-four retention. The selection is on intent, which no amount of post-hoc adjustment observes.
  3. Order the discriminating analyses by cost. First, condition on pre-connection activity by comparing retention within strata of week-one core-action count, which removes the crude version of the confound but not unobserved intent. Second, look for variation in integration availability that was unrelated to intent, such as a staggered release or an outage window. Third, an encouragement design that randomises a prompt to connect and reads the intent-to-treat effect on week-four retention, which is the only version that identifies an effect.
  4. Run the timing check, because it is nearly free and it is the most persuasive single piece of evidence. If the retention advantage among connectors is already visible before any of them connected, the causal story is largely finished.
  5. Pre-commit to what would change your mind and say it before you show anything: if the gap survives stratification and the encouragement arm moves week-four retention at all, the feature has a case and you will say so.
  6. Open the 1:1 by agreeing with the true part, that the correlation is real and worth chasing, then ask what effect size the roadmap plan assumes. That makes the size of the claim the topic instead of its authorship.

Follow-up

  • The encouragement test needs six weeks and the roadmap locks in two. What do you recommend in the interim?
  • Stratifying on week-one activity closes half the gap. What do you conclude, and what do you still not know?
  • How would you word this in the roadmap document so the PM's original number is reframed rather than deleted?