Interview guides · Extend

Extend Data Scientist Interview Questions & Guide 2026

Practice explaining analysis before a Extend data scientist interview: the recommendation, the uncertainty, the follow-ups.

The 60-second brief

The Data Scientist role at Extend is pivotal in driving data-informed decisions that enhance the company’s offerings and user experiences. As a Data Scientist, you will engage deeply with complex datasets, employing statistical analysis and machine learning to derive insights that can significantly influence product strategy and business outcomes. Your work will directly impact the development of innovative solutions that align with Extend’s mission of providing seamless financial services to its users.

How much statistics you need depends on the flavour of the seat. Experiment-facing work wants you deep enough to notice that repeated looks at accumulating data inflate the false positive rate of a fixed-sample test; modelling-facing work wants estimation and honest uncertainty intervals.

Focus areas

Split voluntary churn from involuntary payment failure
Measure catalogue breadth beyond head consumption
Define a qualified stream and defend it

Overview

In this role, you will collaborate with multidisciplinary teams, including product managers, engineers, and other data professionals, to tackle challenging problems. This collaborative environment fosters creativity and innovation, allowing you to explore and implement cutting-edge analytical techniques. Expect to work on high-impact projects that may involve predictive modeling, data visualization, and experimentation, contributing to the evolving landscape of financial technology.

The Extend interview process

Extend candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screening

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Hiring Manager Call

Underneath the questions about your past work sits a resourcing question. Given four things worth doing and one of you, which gets done and what happens to the rest? Managers ask because that is the daily texture of the job, and because the answer shows whether you rank work by effort or by what it changes. The weak version sorts by personal interest or by whoever asked most insistently. The strong version ties each candidate piece of work to a decision somebody downstream is waiting on, and then names the one you would drop and who you would tell.

What to demonstrate

Whether you rank work by the decision it unblocks or by how interesting the method is
How you describe a request you declined, and whether you can say who you said it to
Whether your sense of how long something takes survives one follow-up question about the messy part
How you decide something is good enough to hand over unfinished

How to prepare

Write out your current queue and, next to each item, the decision that stays stalled until it lands. Anything with no waiting decision becomes your example of work you would cut
Rehearse turning down a plausible stakeholder request out loud, including the smaller alternative you offered instead
Have one case where you shipped a rough answer early and one where you refused to, with the reason that separated them

Take-Home Assessment

The clock is part of the test. Three to six hours is not enough to do everything the dataset supports, so the submission mostly reveals how you spend a fixed budget against an open question. A reviewer sees which paths you took and, by absence, which you abandoned. Work that runs out of time inside the analysis ships a thin conclusion, while work that cuts scope early protects the last hour for writing. The most reliable way to lose here is to leave the scoping decision implicit, so it reads as something you missed rather than something you chose.

What to demonstrate

Whether the scope you settled on is presented as a decision with a reason, rather than left for the reader to infer from what is missing
Whether the depth of the work is consistent with the stated time budget, instead of several half-finished directions left open
Whether the closing section reads as something written on purpose rather than assembled from whichever cells survived

How to prepare

Run a timed rehearsal on a public dataset with a hard stop, holding the final sixty minutes for writing no matter where the analysis has got to
Before opening the data, list the questions it could plausibly answer, pick one, and keep the discarded ones as a short note on what you did not attempt and why
Commit a one-line finding after each analysis step so the writeup is assembled from recorded results rather than from memory at midnight

Onsite Interviews

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Reading recommendation quality from click-through or play-through on logged slates.

Slate position drives interaction independently of relevance, and the ranker that produced the log also decided what got exposure, so logged data is confounded by the policy you are trying to evaluate. Inverse-propensity estimators fix this only when the logging propensities are known and every item had a non-zero chance of being shown; without exploration slots that positivity condition fails silently and the estimate is confidently wrong. The visible symptom is a feedback loop: offline metrics improve, consumption concentrates further into the head of the catalogue, and the online test does not reproduce the offline gain.

Comparing consumption week over week across the release calendar and the rights calendar.

A major release, a season drop or a live event produces a spike that dwarfs almost any treatment effect, and the effect is not confined to the new title because it pulls attention from everything else in the same window. Separately, licensed content leaves the catalogue when its window expires, so consumption falls with no product change and the drop is attributed to whatever shipped that week. Both need to be handled by an explicit control: a comparison period chosen for calendar equivalence, a covariate for scheduled releases, or a pre-registered rule for excluding a window, decided before the numbers are seen.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Reading an observational correlation as a causal effect

Name the confounder you are most worried about and the design that would remove it: an experiment, a difference-in-differences with a checked pre-period trend, an instrument, or a regression discontinuity. When none is available, state which direction the bias likely runs and bound the claim accordingly.

Questions & practice

Write a function to calculate the mean and standard deviation of a dataset.

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Quantify uncertainty explicitly rather than reporting a point estimate alone.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

How would you implement a decision tree from scratch?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Permutation test for hours per account across two ranker arms

Medium · permutation-test · heavy-tails · experiment-analysis

arm_hours holds one row per account: account_id, arm in {control, treatment}, qualified_hours over a seven-day window. Roughly 40 thousand accounts per arm, about 38 percent of them at zero hours, and the non-zero tail is long. Without calling a library test function, write a permutation test on the difference in mean hours with 10,000 relabellings. Then run it as two parts: the difference in the share of accounts with any hours, and the difference in mean hours among accounts with hours. Report all three and say which belongs in the readout.

Approach

Shuffle the labels, not the data. Draw a permutation of the arm indicator over accounts, which is the unit that was randomised, and hold the hours vector fixed.
Make each replication O(n): precompute the grand sum and the arm sizes, so a shuffled difference is the treated subset sum over n_t minus (grand sum minus that subset sum) over n_c. Ten thousand replications then take seconds instead of a minute.
Use the two-sided p-value (1 + count of permuted absolute differences at or above the observed) divided by (B + 1). The plus one is not cosmetic: it makes the p-value valid rather than optimistic, and it means the smallest reportable value here is 1/10001, not zero.
For the two-part version, run the same machinery on the 0/1 indicator for the rate, then on the non-zero subset for the conditional mean, and say plainly that conditioning on a post-treatment outcome breaks the randomisation, so the conditional arm is descriptive rather than causal.
Report the rate test and the overall mean test as the result, with the conditional mean as colour, and give the effect size in hours beside each p-value, because at 80 thousand accounts almost anything is detectable.

Worked solution (30 min)

Pull hours into a float array and arm into a boolean array; record n_t, n_c, the grand sum and the observed difference of means.
Loop B = 10,000 times, taking np.random.permutation(n)[:n_t] as the treated index set and accumulating hours[idx].sum().
Convert each shuffled treated sum into a difference of means, compare its absolute value against the observed, count, and apply the plus-one form.
Repeat with hours replaced by (hours > 0).astype(float) for the rate test.
Subset to hours above zero, re-run for the conditional mean, and assemble a three-row result carrying effect, p-value and the unit each is defined on.

Expected result: Three rows: the mean-hours difference with a permutation p-value bounded below by 1/10001; the streaming-rate difference in percentage points with its own p-value; and the conditional mean difference among non-zero accounts, labelled descriptive because it conditions on a post-treatment outcome.

Verify it yourself

The permutation distribution of the mean difference is centred on zero within its own Monte Carlo error. A centre away from zero means the labels were not actually being reshuffled between replications.
The permutation p-value on the rate agrees to two decimal places with a two-proportion z-test, which is a fair cross-check because a 0/1 indicator is not heavy-tailed.
Doubling B leaves the p-value stable to within roughly 1 over sqrt(B). A p-value that moves a lot between runs is not yet resolved.
Multiplying every hours value in both arms by 60 leaves all three p-values identical under the same seed and scales only the effect sizes, since a permutation test is invariant to a common positive rescaling.

Follow-up

The permutation p-value on the mean is 0.03 and the rate test is flat. What is the most likely explanation, and does it change the decision?
How would CUPED on pre-period hours change your power here, and what would disqualify a covariate?
Accounts are households. Does that affect the validity of this test, or only its interpretation?

Describe how you would optimize a SQL query for better performance.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Sessionise playback into listening sessions with an idle gap

Medium · window functions · gaps and islands · sessionisation

fct_stream has profile_id, stream_id, content_version_id, started_at, ended_at (nullable when the client stopped reporting) and played_seconds. Collapse consecutive streams by the same profile into listening sessions: a new session begins when a stream starts more than 30 minutes after the latest end reached by any of that profile's earlier streams. One profile can play on two devices at once, so streams overlap and 'the previous stream' is not well defined — the idle gap is measured against the running maximum end, not against the row immediately before. Treat a NULL ended_at as started_at plus played_seconds. Return one row per session with profile_id, session_start, session_end, stream_count, distinct content_version_id count and total played_seconds. Use window functions; a self-join on fct_stream is not an acceptable answer.

Approach

In a first CTE compute effective_end = COALESCE(ended_at, started_at + played_seconds * INTERVAL '1 second'). This both fills the abandoned-client case and bounds it, since played_seconds is advancing playback only and cannot run past the true end.
Take prev_max_end = MAX(effective_end) OVER (PARTITION BY profile_id ORDER BY started_at, stream_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING), and flag is_new_session = (prev_max_end IS NULL OR started_at > prev_max_end + INTERVAL '30 minutes'). LAG(effective_end) is the reflex here and it is wrong under concurrent devices: with A 10:00-12:00, B 10:05-10:10 and C starting 10:45, LAG hands C the end of B at 10:10, so C opens a new session 35 minutes into a session still running until 12:00. The NULL branch is not defensive noise either: the first row of every profile has no preceding frame and would otherwise fall outside any session.
Number the sessions with SUM(is_new_session::int) OVER (PARTITION BY profile_id ORDER BY started_at, stream_id ROWS UNBOUNDED PRECEDING). A running count of boundaries is the gaps-and-islands device. Specify ROWS rather than letting the default RANGE frame apply, because RANGE lumps peer rows that share a started_at into one frame position — the same reason the ORDER BY carries stream_id as a tie-break.
Group by (profile_id, session_number) and aggregate: MIN(started_at), MAX(effective_end), COUNT(*), COUNT(DISTINCT content_version_id), SUM(played_seconds).
The running maximum makes the sessions disjoint in wall-clock time, but it does not make SUM(played_seconds) an elapsed-time figure: two devices playing at once contribute overlapping seconds, so the total can exceed session_end minus session_start. Report it as consumption and say which of the two the requester asked for.

Worked solution (30 min)

Materialise the effective_end CTE and check for rows where effective_end < started_at, which means negative played_seconds or a clock-skewed ended_at and must be quarantined before anything else.
Add the running MAX and the boundary flag; verify that exactly one row per profile has prev_max_end IS NULL and that it is flagged as a session start.
Add the running SUM to number sessions; confirm the numbers are dense and begin at 1 within each profile.
Aggregate and reconcile: SUM(stream_count) across all sessions must equal the input row count.
Run the overlap fixture — one profile with A 10:00-12:00, B 10:05-10:10 and C at 10:45 — and confirm it collapses to one session. With LAG(effective_end) in place of the running MAX it splits into two, and the second session's start, 10:45, sits inside the first session's span.

Expected result: One row per (profile_id, session_number). Session numbers are dense from 1 within each profile, SUM(stream_count) equals the input row count exactly, and a profile's sessions are disjoint in wall-clock time: each session_start exceeds the previous session's session_end by more than 30 minutes.

Verify it yourself

Input row count equals SUM(stream_count); a shortfall means rows fell out of the window partition or the NULL branch of the boundary flag.
For any profile, each session's session_start exceeds the previous session's session_end by more than 30 minutes. This holds only because the boundary is measured against the running maximum end — under LAG(effective_end) the overlap fixture above produces a session whose session_start, 10:45, precedes the prior session_end of 12:00.
Changing the gap from 30 to 60 minutes reduces the session count or leaves it unchanged, and never raises it.

Follow-up

Two devices on one profile play simultaneously. Is that one session or two, and what changes in the query for each answer?
Re-run at a 10-minute and a 60-minute idle gap. How far does session count move, and which number do you publish and defend?
Can SUM(played_seconds) for a session ever exceed session_end minus session_start, and what does it mean when it does?

You are tasked with improving user retention. What data would you analyze, and what steps would you take?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Given a dataset with customer transaction history, how would you identify patterns of fraudulent behavior?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

Discuss how you would approach a situation where your data analysis contradicted the established business strategy.

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize your tasks when working on multiple projects?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

How would you design an A/B test for a new feature in a financial app?

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
State the primary metric and the minimum effect worth shipping, then size the test.
Say whether units interfere with each other, and switch design if they do.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

Explain a recent trend in data science that you find interesting.

Medium

Approach

Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Triage a sample ratio mismatch before reading segment cuts

Easy · sample ratio mismatch · multiple comparisons · experiment hygiene

A 50/50 account-randomised test of a new detail-page layout has run ten days. The scorecard reports 2,014,880 accounts in control and 1,997,340 in treatment. It also reports 22 pre-declared segment cuts across signup_platform, device_type and signup_country, of which three are significant at p < 0.05, including a strongly negative one on game_console. The PM wants to ship with a console carve-out. State what you check first and what it returns, what the three segments are worth, and what would have to be true before any part of this scorecard can be read.

Approach

Run the sample ratio check before looking at any effect: a chi-square goodness-of-fit against the intended 50/50 on raw assignment counts, one degree of freedom. Treat p < 0.001 as a hard stop rather than a warning banner.
If it fires, stop reading effects entirely. Unequal counts mean the arms are not exchangeable, so every estimate on the scorecard, overall and per segment, carries a bias of unknown size and unknown direction.
Hunt the cause in the order that explains most mismatches: assignment happening downstream of a redirect or a feature-flag fetch that fails at different rates per arm, exposure logging dropped on one arm's code path, bot and is_test_account filtering applied after assignment rather than before, and deduplication of accounts that were reassigned.
Treat the segments as a multiplicity problem even in a clean test. Twenty-two cuts at alpha 0.05 give 1.1 false positives in expectation, and the chance of at least one is 1 - 0.95^22 = 68 percent, so three hits is barely news.
Apply Benjamini-Hochberg at FDR 0.05 across the 22 p-values, and apply it as the step-up procedure it is: order the p-values ascending, compare the k-th against k * 0.05 / 22, take the largest k that passes, and reject everything at or below it. Require any survivor to come with a pre-registered mechanism before it becomes a product carve-out instead of a post-hoc story.

Worked solution (15 min)

Expected count per arm = (2,014,880 + 1,997,340) / 2 = 2,006,110, so the deviation is 8,770 accounts.
chi-square = 2 * 8,770^2 / 2,006,110 = 2 * 76,912,900 / 2,006,110 = 76.7 on one degree of freedom.
Convert: z = sqrt(76.7) = 8.76, so p is on the order of 1e-18. The realised split is 50.22 / 49.78.
Set the segments aside, and write down the Benjamini-Hochberg rule correctly for when there is a valid scorecard to apply it to. Order the 22 p-values ascending and find the largest k with p_(k) <= k * 0.05 / 22. The ladder is 0.00227 at k = 1, 0.00455 at k = 2, 0.00682 at k = 3, rising linearly to 0.05 at k = 22. Because it steps up, that largest k also rejects every smaller p-value beneath it: p-values of 0.001, 0.004 and 0.006 would all be rejected, even though two of them sit above 0.00227. The single threshold 0.05 / 22 = 0.00227 is Bonferroni, a different and more conservative rule.

Expected result: chi-square approximately 76.7 on 1 df, p on the order of 1e-18: an unambiguous sample ratio mismatch, not a rounding artefact. The scorecard is unreadable, nothing ships, and the work is to find the assignment or exposure-logging asymmetry and re-run. Separately, under the null 22 cuts yield three or more hits about 9.5 percent of the time, so the segment pattern is unremarkable even in a valid test, and whether any of the three would survive Benjamini-Hochberg depends on where they sit on the k * 0.05 / 22 ladder, not on a single fixed threshold.

Verify it yourself

Under a true 50/50 split at this size the standard deviation of one arm's count is 0.5 * sqrt(4,012,220) = 1,002 accounts, so 8,770 is 8.8 standard deviations out; a gap of a few hundred would be ordinary.
Recomputing the ratio on raw assignment events rather than on the deduplicated scorecard population should reproduce the same imbalance. If it does not, the loss is in the analysis pipeline rather than in assignment.
Under the null the expected number of significant cuts among 22 at alpha 0.05 is 1.1, and the probability of three or more is 9.5 percent.
Benjamini-Hochberg must reject at least as many hypotheses as Bonferroni on the same p-values, never fewer. An implementation that rejects fewer, or that compares every p-value against 0.05 / 22, is running Bonferroni under the wrong name and will discard real segment effects.

Follow-up

An engineer finds that assignment happens after a client-side flag fetch that times out more often on smart_tv. Is that a mismatch cause, and can the test be salvaged by excluding those accounts?
Suppose the counts had been balanced. What evidence would make the game_console result worth acting on?
Under Benjamini-Hochberg, a p-value of 0.006 can be rejected among these 22 cuts while a p-value of 0.004 elsewhere in the same family is not. Explain how both can be true at once.

Involuntary churn share jumped after a dashboard rebuild

Medium · definition-change · churn · late-resolving-data

Involuntary share of gross churn — fct_subscription_period rows with renewal_outcome = 'failed_involuntary' over rows with renewal_outcome in ('cancelled_voluntary','failed_involuntary'), keyed on period_end_ts month — reads 31% for last month against 19% for the three months before. Billing reports no change in decline rates and dunning_attempts per failed period is unchanged. Using fct_subscription_period (period_end_ts, renewal_outcome, payment_status, dunning_attempts, cancel_requested_ts, billing_provider), determine whether involuntary churn actually rose. Deliverable: the corrected month-on-month series and the specific change that produced the step.

Approach

Check the month's maturity before anything else. Periods ending recently still have retries in flight, and a row reading 'failed_involuntary' today becomes payment_status = 'retried_paid' next week. Recompute every one of the last twelve months as of a fixed age — period_end_ts plus the maximum retry window — instead of as of today.
If the step survives age alignment, hunt for a definition change: diff the model or query against the version that produced 19%, and check specifically whether the month is now keyed on cancel_requested_ts rather than period_end_ts, and whether 'refunded' or 'chargeback' rows have entered one side of the ratio.
Settle it with one query rather than a narrative. Run the old definition and the new definition on the same month. Two numbers from one month under two definitions is decisive; two numbers from two months under one definition is not.
If a definition did change, restate history rather than leaving a step in the chart: run the new definition back across all twelve months and publish both series with the cutover marked, so nobody re-discovers this in a quarter.
Cut the horizon-closed series by billing_provider before concluding nothing moved. Card-direct, app-store and carrier billing have different retry mechanics and different success rates, so a mix shift between them moves the ratio with every provider's own rate flat.

Follow-up

How would you set the dunning horizon from the data instead of taking billing's stated maximum?
Under cancel-at-period-end, the cancellation request and the entitlement end fall in different months. Which timestamp keys this metric, and what does the other choice do to the curve?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

How do you handle missing values in a dataset?

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Scope a one-line request about home row performance

Easy · scoping · stakeholder communication · metric definition

A director messages you: "Is the new home row working?" Nothing else. A new ranker_version has been serving a fraction of profiles for eleven days. You have fct_impression (surface, slate_position, ranker_version, is_exploration_slot, logging_propensity, experiment_assignment_id, was_clicked) and fct_stream (impression_id, start_source, is_qualified, played_seconds). You get a fifteen-minute call before they go into a rollout meeting. Deliverable: the three questions you ask before writing any SQL, the single primary metric you commit to with its guardrail, and the questions you tell them this data cannot answer.

Approach

The probe is whether you convert a vague request into a decision before producing a number. Ask what happens at each answer — rollback, widen, iterate — because a question whose answer changes nothing is a report request and should be scoped as one.
Pin the unit of analysis out loud. fct_impression is at (profile, slate, slot) grain and experiment_assignment_id is per assignment, so the comparison must be aggregated to the assignment unit first; comparing impression-level rates lets a change in slate length move the metric on its own.
Commit to one primary metric from the tree — qualified hours per active account-week for assigned accounts — and name the guardrail pair explicitly: share of qualified streams with start_source = 'autoplay_continuation', and median completion_ratio within content_type. A ranker can lift qualified stream counts by queueing short items that clear the 30-second threshold, and the guardrail is the only thing that catches it.
State the refusals with structural reasons, not time reasons: eleven days gives no matured cohort, so month-6 retention and net revenue per active account-month are unanswerable; and logging_propensity is populated only where is_exploration_slot = true, so the positivity condition for an off-policy estimate fails outside those slots.
Write the scope back in one paragraph — decision, metric, guardrail, the date the read becomes valid — and get it agreed in the thread before querying, so the number that arrives is the number that was asked for.

Follow-up

They reply "just give me click-through by slate position." What do you say, and what would that number actually tell them?
The eleven days include a weekend and a large release landing on day six. Does that change the metric you commit to, or only the read date?
What would have to be true for you to be willing to answer the retention question from this experiment?

Disagree with a product manager about a completion metric

Medium · disagreement · metric design · influence

A product manager proposes making duration-normalised completion rate the team's primary metric for the quarter, and wants a plain global unweighted version "so it's simple enough to put on a wall." You expect the unweighted version to move several points on catalogue mix alone, with no change in how satisfying anything was. You have one meeting, they own the roadmap, and you will work with them for years. Deliverable: the argument, the evidence you bring, and the fallback you accept if they still want the simple version.

Approach

The probe is whether you can lose the decision without damaging either the metric or the relationship. Bring the failure already reproduced: recompute the unweighted global rate over the last two quarters and point at the weeks it moved several points where the only input that changed was which content people played.
Decompose the variance instead of objecting. Attribute the movement in the unweighted rate to between-cell mix — (content_type, duration decile) — versus within-cell movement. A decomposition is an argument; an assertion that mix matters is a preference.
Concede the real cost honestly: the mix-weighted version is harder to explain and harder to recompute. Bring the mitigation rather than dismissing the objection — weights fixed from a stated reference month, published once, so anyone can reproduce the number without redoing the weighting.
Offer the fallback deliberately and in advance: ship both, with the unweighted rate labelled a diagnostic and the weighted one as the decision metric, and pre-agree the one observation that would mean the simple version has misled the team.
Leave the decision with them, with the consequence written down before it happens. That is what makes a later correction a shared prediction coming true rather than a retrospective argument about who was right.

Follow-up

They ship the simple version and it moves four points during a heavy release week. How do you raise it without saying you told them so?
Is there a case where the unweighted rate is genuinely the right primary metric?
Suppose you are wrong and the mix effect turns out to be small. What would you change about how you argued this?