Interview guides · Mutual of Omaha
Mutual of Omaha Data Scientist Interview Questions & Guide 2026
SQL practice for Mutual of Omaha data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.
The 60-second brief
As a Data Scientist at Mutual of Omaha, you play a pivotal role in leveraging data to drive insights and inform strategic decisions. Your expertise in statistical analysis, machine learning, and data wrangling is crucial for optimizing Mutual of Omaha's products and improving customer experiences. This role allows you to work on a variety of projects that directly impact the business, from enhancing insurance models to predicting customer needs and behaviors.
If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.
Focus areas
- Define a qualified stream and defend it
- Separate release-calendar spikes from underlying trend
- Diagnose rebuffering by device, network and POP
Overview
Your contributions will not only enhance operational efficiency but also support the development of innovative insurance solutions tailored to meet diverse customer requirements. Collaborating with cross-functional teams, you will engage in complex problem-solving that involves high volumes of data, helping shape the future of Mutual of Omaha's offerings in a competitive market. The work environment is dynamic and supportive, encouraging you to explore cutting-edge technology and methodologies in data science.
The Mutual of Omaha interview process
Mutual of Omaha candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.
HR Screening
Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.
What to demonstrate
- Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
- Whether your reason for wanting the role points at the work itself rather than the company's reputation
- Whether your language signals the level being screened for: what you decided yourself versus what you were handed
How to prepare
- Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
- Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
- Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands
Technical Interviews
A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.
What to demonstrate
- Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
- Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
- Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.
How to prepare
- Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
- Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
- If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows
Collaborative Environment
Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.
What to demonstrate
- Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
- Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
- Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs
How to prepare
- For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
- Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
- For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.
Live Coding Exercises
Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.
What to demonstrate
- Whether the query you write matches the plan you just described
- What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
- Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly
How to prepare
- Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
- Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
- Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock
Project Discussions
An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.
What to demonstrate
- Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
- Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
- How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact
How to prepare
- Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
- For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
- Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.
Where candidates lose points
Testing hours, revenue or completion with a difference in means on a heavy-tailed distribution.
Listening and viewing hours per account are strongly right-skewed and content popularity is close to power-law, so the variance of a sample mean is dominated by a few accounts and the central limit approximation converges slowly at realistic sample sizes. A t-test on mean hours can flip sign when one heavy account's week changes, and an experiment can appear significant because a single title released into one arm's window. Capping at a pre-registered percentile, or decomposing into a rate (did they stream at all) and a conditional intensity, controls the variance, at the stated cost that capping biases toward zero exactly when the true effect lives in the tail.
Reading recommendation quality from click-through or play-through on logged slates.
Slate position drives interaction independently of relevance, and the ranker that produced the log also decided what got exposure, so logged data is confounded by the policy you are trying to evaluate. Inverse-propensity estimators fix this only when the logging propensities are known and every item had a non-zero chance of being shown; without exploration slots that positivity condition fails silently and the estimate is confidently wrong. The visible symptom is a feedback loop: offline metrics improve, consumption concentrates further into the head of the catalogue, and the online test does not reproduce the offline gain.
Reporting a mean for a heavy-tailed metric without saying what it hides
For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.
Crediting a treatment for regression to the mean
Selecting a group because it is extreme (lowest-engagement users, accounts having their worst month, the bottom decile of a score) moves that group's expected next-period value back toward the average even under no treatment, by exactly as much as the selecting measure is imperfectly correlated with its own later value. Compare against units that met the same selection rule and went untreated, or use two pre-periods so the bounce-back is visible before the intervention starts. A pre-post number on a group chosen for being extreme measures the selection rule, not the treatment.
Questions & practice
Can you write a function to calculate the mean and standard deviation of a list of numbers?
Medium · statistics and probability
Approach
- Write down the assumption the method needs before you use the method.
- Sanity-check the answer against a simple bound or a simulated case.
- Say what the estimate is of, and over what population it generalises.
Follow-up
- Which assumption here is most likely to be violated in practice?
- What sample size would you need to detect an effect half this size?
How would you approach building a predictive model for customer churn?
Medium · machine learning and modelling
Approach
- Check what information would not exist at prediction time, and exclude it.
- Say how the offline result would be validated online before it is trusted.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- What would you monitor after launch to know the model is still valid?
What are some common metrics used to evaluate model performance?
Medium · machine learning and modelling
Approach
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- Where could label leakage enter this setup?
- How would you choose the decision threshold, and who owns that choice?
Duration-decile-weighted completion rate with fixed reference weights
Hard · metric-definition · standardisation · pandas
Implement this metric. Numerator: qualified streams with completion_ratio at or above 0.9. Denominator: qualified streams with a non-null duration_seconds, so live events are out. Compute the rate inside each (content_type, duration decile) cell, then aggregate with catalogue-mix weights fixed from a reference month. You get streams and content for the last eight weeks plus ref_streams for the reference month. Return the weighted index and the unweighted global rate for each of the eight weeks, and the share of reference weight your cells actually covered.
Approach
- Fix the decile boundaries from the reference month, within content_type, over that month's qualified streams. Not over the catalogue, and not per week: the weights and the cells have to be defined on the same population or the weighted sum is adding rates over cells the weights do not describe.
- Store the boundaries explicitly and bin every week against them with pd.cut, with open-ended outer edges, so a duration longer than anything in the reference month still lands in the top cell instead of becoming NaN and quietly leaving the denominator.
- Weights are the reference month's share of qualified streams per (content_type, decile) cell, summing to one across all cells. Apply them to each week's cell rates and report the covered weight separately, because a week missing a cell entirely gives a renormalised index, and renormalising silently is how the series gains a step change nobody can explain.
- Keep the unweighted rate beside it. The pair is the deliverable: the weighted line is the answer, and the gap between the two is the size of the mix effect you removed, which is the first thing anyone reading it will ask about.
- Sanity-test the whole construction by feeding the reference month back in as the current week; the weighted and unweighted rates must then be identical to floating-point error.
Worked solution (40 min)
- Join content onto ref_streams, filter to is_qualified with duration_seconds not null, and take within-content_type deciles of duration_seconds at quantiles 0.1 through 0.9, replacing the outer edges with negative and positive infinity.
- Weights: value counts of (content_type, decile) over the reference month, divided by that month's total qualified, non-null-duration streams.
- For each of the eight weeks, filter and join identically, bin with pd.cut against the stored per-type boundaries, and compute each cell rate as the mean of (completion_ratio at or above 0.9), keeping the cell's stream count alongside.
- Weighted index = sum(weight times rate) over cells present that week, divided by the sum of weight over those same cells; record that divisor as covered_weight.
- Unweighted rate = the week's overall mean of the same indicator. Assemble the eight-row output.
Expected result: An eight-row frame of week_start, weighted_rate, unweighted_rate and covered_weight, where covered_weight is at most 1.0 and the two rates coincide only in a week whose cell mix matches the reference month's.
Verify it yourself
- Feeding ref_streams in as the current week returns weighted_rate equal to unweighted_rate to floating-point error, with covered_weight exactly 1.0.
- Duplicating every stream in one cell changes the unweighted rate and leaves the weighted rate unchanged. This is the point of the metric and the fastest proof the weighting is wired up.
- Each reference decile holds about ten percent of that content_type's reference streams. A cell holding thirty percent means ties in duration_seconds collapsed the quantile edges, which is common where many items share a rounded runtime.
- No row with content_type of live_event, or with a null duration_seconds, survives into either the reference frame or any weekly frame.
Follow-up
- The weighted index is flat and the unweighted rate fell four points. What shipped?
- When would you refresh the reference month, and what do you owe the series when you do?
- Podcast episodes and film have very different completion shapes. Would you ever report one number across them at all?
Solve a live coding challenge related to data manipulation or analysis.
Medium · sql and data manipulation
Approach
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
Follow-up
- How does the query change if the join becomes one-to-many?
- How would you verify this result without re-running the same query?
Net revenue per active account-month without fanning out periods
Hard · fan-out · revenue recognition · grain
fct_subscription_period has account_id, period_start_ts, period_end_ts, net_amount_usd and payment_status. fct_stream has account_id, started_at, played_seconds and is_qualified. Compute net revenue per active account-month for March 2026: the numerator is subscription revenue over periods with payment_status in ('paid','retried_paid'), recognised pro rata by the days of each period falling inside March; the denominator is distinct accounts with at least one qualified stream in March. A colleague's draft joins the two tables on account_id, filters streams to qualified March rows, and sums net_amount_usd; its numerator is roughly forty times too large. Explain the factor and write the correct query.
Approach
- Name the mechanism and decompose the number. The join is one period to many streams, so each period row is duplicated once per matching stream and its net_amount_usd is added that many times: the naive sum equals the un-prorated period sum weighted by each account's March qualified-stream count, which is the net_amount_usd-weighted mean stream count over the accounts the inner join keeps. The distance from there to the correct numerator has two further parts, in opposite directions — pro-rating shrinks the correct figure, while the inner join has dropped every period belonging to an account that streamed nothing in March. A weighted mean near thirty streams against a pro-rata factor near 0.75 is what puts the draft around forty times high; the number is not arbitrary, which is why it looks plausible enough to ship.
- Adopt the rule that produces the fix: two fact tables never meet at row grain. Aggregate each to account grain first and join the aggregates, or — better here — do not join at all, because the numerator and the denominator are independent scalars that share no key.
- State the interval convention before writing the arithmetic. Treating periods as half-open [period_start_ts, period_end_ts) makes consecutive periods tile without overlap; overlap_days = GREATEST(0, LEAST(period_end_ts, DATE '2026-04-01') - GREATEST(period_start_ts, DATE '2026-03-01')) in days, and the contribution is net_amount_usd * overlap_days / total_days_of_period. An annual period contributes roughly 31/365 of its amount.
- Build the denominator on its own: COUNT(DISTINCT account_id) FROM fct_stream WHERE is_qualified AND started_at >= '2026-03-01' AND started_at < '2026-04-01'. Do not restrict it to paying accounts — the metric deliberately holds subscription and advertising revenue to one engaged-account denominator, and filtering to payers is what makes a tier migration look like two unrelated numbers moving.
- State the closing rule with the result: the month must be at least 45 days closed, because refunds and chargebacks are recorded against periods late and an open month always reads high.
Worked solution (35 min)
- Reproduce the bug deliberately: compute the naive joined sum, and beside it the un-prorated SUM(net_amount_usd) over the same period rows restricted to accounts with at least one qualified March stream. Their ratio is the fan-out multiple, and it is the only ratio in this problem that has a closed form.
- Write the overlap expression and verify that one period's contributions, summed across every month it touches, reconstruct its net_amount_usd exactly.
- Compute the denominator independently and combine the two as one-row CTEs joined with CROSS JOIN, so neither side can fan out the other.
- Re-run for February and March and confirm no revenue is counted in both, which is the boundary test for the half-open convention.
Expected result: A single row of numerator_usd, active_accounts and their quotient. Each period's pro-rata contributions summed over all months it spans reconcile to its net_amount_usd within floating-point rounding.
Verify it yourself
- Naive joined sum divided by the un-prorated SUM(net_amount_usd) over the same period rows — restricted to the accounts the inner join keeps, meaning those with at least one qualified March stream — equals the net_amount_usd-weighted mean qualified-stream count over those accounts. Do not expect that identity against the correct numerator: that one is pro-rated to March and also carries the periods of accounts that streamed nothing, so the two differ by more than a single factor.
- A period's contributions across every calendar month it touches — thirteen months for an annual period that does not begin on the 1st — sum to its net_amount_usd.
- No period contributes anything to March when period_end_ts <= 2026-03-01 or period_start_ts >= 2026-04-01.
Follow-up
- One account holds two overlapping periods after a plan migration. Whose revenue counts for March, and does your pro-rata still sum to each period's own total?
- Advertising revenue belongs in this numerator. What grain does it arrive at, and what breaks in your query when you add it?
- How would you turn this into a monthly time series without rewriting the date literals, and what would you do about periods that span three months?
How do you prioritize tasks when faced with multiple deadlines?
Medium · product sense and metrics
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Fix the population and the time window before naming any metric.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
If given a dataset with multiple variables, how would you determine which features are most significant?
Medium · product sense and metrics
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
Can you describe a project where you had to clean a messy dataset?
Medium
Approach
- Clarify what is being asked and what a complete answer would contain.
- Work from the decision backwards to the evidence you would need.
- State your assumptions explicitly before working the problem.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Triage a sample ratio mismatch before reading segment cuts
Easy · sample ratio mismatch · multiple comparisons · experiment hygiene
A 50/50 account-randomised test of a new detail-page layout has run ten days. The scorecard reports 2,014,880 accounts in control and 1,997,340 in treatment. It also reports 22 pre-declared segment cuts across signup_platform, device_type and signup_country, of which three are significant at p < 0.05, including a strongly negative one on game_console. The PM wants to ship with a console carve-out. State what you check first and what it returns, what the three segments are worth, and what would have to be true before any part of this scorecard can be read.
Approach
- Run the sample ratio check before looking at any effect: a chi-square goodness-of-fit against the intended 50/50 on raw assignment counts, one degree of freedom. Treat p < 0.001 as a hard stop rather than a warning banner.
- If it fires, stop reading effects entirely. Unequal counts mean the arms are not exchangeable, so every estimate on the scorecard, overall and per segment, carries a bias of unknown size and unknown direction.
- Hunt the cause in the order that explains most mismatches: assignment happening downstream of a redirect or a feature-flag fetch that fails at different rates per arm, exposure logging dropped on one arm's code path, bot and is_test_account filtering applied after assignment rather than before, and deduplication of accounts that were reassigned.
- Treat the segments as a multiplicity problem even in a clean test. Twenty-two cuts at alpha 0.05 give 1.1 false positives in expectation, and the chance of at least one is 1 - 0.95^22 = 68 percent, so three hits is barely news.
- Apply Benjamini-Hochberg at FDR 0.05 across the 22 p-values, and apply it as the step-up procedure it is: order the p-values ascending, compare the k-th against k * 0.05 / 22, take the largest k that passes, and reject everything at or below it. Require any survivor to come with a pre-registered mechanism before it becomes a product carve-out instead of a post-hoc story.
Worked solution (15 min)
- Expected count per arm = (2,014,880 + 1,997,340) / 2 = 2,006,110, so the deviation is 8,770 accounts.
- chi-square = 2 * 8,770^2 / 2,006,110 = 2 * 76,912,900 / 2,006,110 = 76.7 on one degree of freedom.
- Convert: z = sqrt(76.7) = 8.76, so p is on the order of 1e-18. The realised split is 50.22 / 49.78.
- Set the segments aside, and write down the Benjamini-Hochberg rule correctly for when there is a valid scorecard to apply it to. Order the 22 p-values ascending and find the largest k with p_(k) <= k * 0.05 / 22. The ladder is 0.00227 at k = 1, 0.00455 at k = 2, 0.00682 at k = 3, rising linearly to 0.05 at k = 22. Because it steps up, that largest k also rejects every smaller p-value beneath it: p-values of 0.001, 0.004 and 0.006 would all be rejected, even though two of them sit above 0.00227. The single threshold 0.05 / 22 = 0.00227 is Bonferroni, a different and more conservative rule.
Expected result: chi-square approximately 76.7 on 1 df, p on the order of 1e-18: an unambiguous sample ratio mismatch, not a rounding artefact. The scorecard is unreadable, nothing ships, and the work is to find the assignment or exposure-logging asymmetry and re-run. Separately, under the null 22 cuts yield three or more hits about 9.5 percent of the time, so the segment pattern is unremarkable even in a valid test, and whether any of the three would survive Benjamini-Hochberg depends on where they sit on the k * 0.05 / 22 ladder, not on a single fixed threshold.
Verify it yourself
- Under a true 50/50 split at this size the standard deviation of one arm's count is 0.5 * sqrt(4,012,220) = 1,002 accounts, so 8,770 is 8.8 standard deviations out; a gap of a few hundred would be ordinary.
- Recomputing the ratio on raw assignment events rather than on the deduplicated scorecard population should reproduce the same imbalance. If it does not, the loss is in the analysis pipeline rather than in assignment.
- Under the null the expected number of significant cuts among 22 at alpha 0.05 is 1.1, and the probability of three or more is 9.5 percent.
- Benjamini-Hochberg must reject at least as many hypotheses as Bonferroni on the same p-values, never fewer. An implementation that rejects fewer, or that compares every p-value against 0.05 / 22, is running Bonferroni under the wrong name and will discard real segment effects.
Follow-up
- An engineer finds that assignment happens after a client-side flag fetch that times out more often on smart_tv. Is that a mismatch cause, and can the test be salvaged by excluding those accounts?
- Suppose the counts had been balanced. What evidence would make the game_console result worth acting on?
- Under Benjamini-Hochberg, a p-value of 0.006 can be rejected among these 22 cuts while a p-value of 0.004 elsewhere in the same family is not. Explain how both can be true at once.
Weekly listening hours fell; check the calendar first
Easy · seasonality · data-completeness · metric-hygiene
A dashboard reports qualified hours per active account-week — sum(played_seconds)/3600 over fct_stream rows with is_qualified, divided by distinct (account_id, ISO week) pairs having a qualified stream — down 9% against the prior week. It is being read on a Wednesday and the current week is still open. Two public holidays fell inside the prior week. Using fct_stream (started_at, event_date, played_seconds, is_qualified, account_id, play_territory), establish whether anything real changed. Deliverable: a corrected comparison plus a written rule for how this chart is read from now on.
Approach
- Stop comparing an open week to a closed one. Either truncate the prior week at the same weekday the current week has reached, or exclude the open week entirely. A Wednesday read covers three of seven days and the four missing days include both weekend days.
- Establish the day-of-week profile before judging any week: qualified hours by ISO weekday across the last eight weeks. A truncated week is not a scaled-down full week, it is a systematically weekday-heavy one, and the ratio distorts in a specific direction — the numerator loses the two highest-hour days while the denominator only loses accounts who stream exclusively at weekends.
- Test partition completeness separately from seasonality. Recompute a day that closed two weeks ago and compare it against the value stored at the time. Offline playback uploads in bulk when the device reconnects, so a recent event_date keeps accruing for days; and if the pipeline stamps event_date at ingest rather than from started_at, those hours land on the wrong day entirely, which is a different defect with a different fix.
- Separate the holiday from a trend. Cut the prior week by play_territory and split territories that observe the holidays from those that do not, then recompute the week-over-week change on the non-observing set. A holiday that lifts consumption in two large markets raises the prior week's baseline, so the current week reads as a fall with no behaviour changing anywhere.
- Restate the comparison as complete-week against complete-week with the territory cut shown, and write the reading rule down: the current week is suppressed until it closes plus a stated late-arrival lag, and holiday weeks are annotated on the chart rather than explained in a thread each time.
Follow-up
- The holiday raised hours but also raised the active-account denominator. Which way does that push the ratio, and is the sign consistent with what you observed?
- What evidence sets the late-arrival lag, rather than picking a round number of days?
Suggested practice checklist
Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.
Day 1 — Fix the scope and set a baseline
- Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
- Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
- Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.
Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.
Day 2 — One query pattern, written three times
- Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
- On the third attempt, write the grain of every CTE as a comment before writing its body.
- Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.
Deliverable: Three independent versions of the same query plus a note on what changed between them.
Day 3 — Only the statistics you will be asked to defend
- Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
- Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
- Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.
Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.
Day 4 — One case, and the assumptions holding it up
- Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
- Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
- Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.
Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.
Day 5 — Your own work, timed
- Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
- Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
- Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.
Deliverable: Two timed narratives with one defensible number in the opening line.
Day 6 — The one full rehearsal, in a longer weekend block
- Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
- Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
- Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.
Deliverable: Mock notes naming three failure moments with a specific fix written under each.
Day 7 — Taper
- Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
- Re-read only your own notes from this week, and open no new material.
- Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.
Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.
Where your data came from
Data people depend on systems owned by other teams, and much of the job is negotiating for instrumentation, access, or a fix to a broken pipeline. Prepare an example of getting something changed upstream that you did not control. Describe what you asked for, what you traded, and how you worked while you waited.
Describe a challenge you faced while working on a team project and how you handled it.
Medium · behavioural and stakeholder questions
Approach
- Pick a story where you drove the decision, not one where you observed it.
- State the situation in two sentences and spend the rest on your reasoning.
- Name the disagreement or constraint, and how you resolved it with evidence.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Describe a time when you had to make a decision based on incomplete data.
Medium · behavioural and stakeholder questions
Approach
- Quantify the outcome, including what you would not claim credit for.
- Name the disagreement or constraint, and how you resolved it with evidence.
- Close with what you would do differently, concretely.
Follow-up
- What would you do differently if you ran that project again?
- What did you decide not to do, and why?
Defend a flat result on a flagship discovery launch
Medium · defending findings · statistical power · experiment readout
A new ranker ran to 50 percent of accounts for four weeks. Your read: qualified hours per active account-week is +0.4 percent with a 95 percent interval of [-0.9 percent, +1.7 percent]. Meanwhile the share of qualified streams with start_source = 'algorithmic_slate' rose six points and the share with start_source = 'search' fell five. The team reads the slate shift as the win and wants to ship. The launch review is Friday. Deliverable: a five-minute verbal position and the one table you put on the screen.
Approach
- The probe is whether you can hold a position under social pressure without overclaiming in the other direction. Separate the two claims explicitly: the start-source shift is well measured and real; the hours effect the launch was justified on is not distinguishable from zero at this sample size. Those are different statements and only one is contested.
- Do the power arithmetic before the meeting, using the observed per-account variance on a right-skewed hours metric. Arrive able to say "at this n we could not have detected less than X percent", which is a fact, instead of "it didn't work", which is an opinion the room can simply disagree with.
- Show the substitution as arithmetic, not interpretation: decompose treatment hours by start_source and show slate hours rising by approximately what search hours fell. A start-source mix shift with no change in total is exactly what displacement looks like, and the decomposition sums to the total so there is nothing to argue about.
- Refuse the symmetric overclaim. The interval's upper bound is commercially meaningful, so "the ranker does nothing" is not supportable either; say that out loud, because it is what makes the rest of your position credible.
- Offer the falsifiable next step with a price: a longer read powered for the hours effect, or CUPED on a pre-period consumption covariate measured before assignment, with the required account-weeks stated. Put the disagreement on the pre-registered metric, not on the team's judgement.
Follow-up
- The PM argues start-source mix is a leading indicator of retention. How would you test that claim, and how long would it take?
- Suppose the interval had been [+0.1 percent, +1.9 percent]. Does your position change, and by how much?
- The decision goes against your read and the ranker ships. What do you do on Monday?