cswork / libraryOpen practice workspace

Interview guides · Woolworths Group

Woolworths Group Data Scientist Interview Questions & Guide 2026

Modeling drills for a Woolworths Group data scientist interview: baselines, regression interpretation, class imbalance, model defense.

The 60-second brief

A Data Scientist at Woolworths Group operates at the intersection of massive-scale retail data and cutting-edge artificial intelligence. As one of Australia’s largest organizations, Woolworths Group leverages data to optimize complex supply chains, personalize customer experiences through loyalty programs, and drive strategic decision-making across its vast network of stores and digital platforms. Your work directly influences how millions of customers interact with the brand, making this a role where analytical rigor meets tangible, real-world impact.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

  • Report only matured cohorts for loss metrics
  • Reconcile amounts in minor units and currency
  • Read vintage curves, not blended portfolio averages

Overview

You will join a team dedicated to solving high-stakes problems, such as developing propensity models to predict customer behavior or enhancing operational efficiency through advanced machine learning. The environment is fast-paced and data-rich, requiring you to translate ambiguous business challenges into structured technical solutions. Whether you are working on supply chain logistics or digital retail products, you will be expected to balance technical sophistication with a clear understanding of the Woolworths Group commercial landscape.

Focus your preparation on demonstrating how your technical models directly map to business outcomes. At Woolworths Group, the ability to explain the "why" behind your data is as important as the model itself.

The Woolworths Group interview process

Woolworths Group candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Case Study Interview

Underneath the business framing, this round is usually asking whether you can turn a fuzzy goal into a quantity that could be computed from data such a business would plausibly hold. That means a metric with a stated numerator, denominator, eligibility rule and time window, plus an honest account of the conditions under which it would mislead you. Answers come apart when a candidate names a familiar metric and never defines it, because every follow-up then lands on an ambiguity that was left open and the candidate has to invent the definition under pressure.

What to demonstrate

  • Whether a named metric arrives with its denominator, eligibility rule and window attached rather than assumed
  • Whether the measure follows from the mechanism you proposed, or is a recognisable metric retrofitted to it afterwards
  • Whether you name a guardrail that would reveal the gain came from somewhere you did not want it to come from
  • Whether you can say what data the plan requires and what you would settle for if that logging were never implemented

How to prepare

  • Take five metrics you reach for by reflex and write each as one sentence containing numerator, denominator, eligibility rule and time window. The ones you cannot finish are the ones that will fail under follow-up.
  • For a product you use daily, write the measurement plan you would propose for a change to it: primary metric, one guardrail, the unit of analysis, and the table the numbers would come from.
  • Practise the substitution question. For three metrics you like, write what you would measure instead if the event you depend on were not being logged.

Behavioral Round

Most of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.

What to demonstrate

  • Whether you can state the other side's argument accurately before you explain why you disagreed
  • What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
  • Whether you distinguish being overruled from being wrong, and can give an example of each

How to prepare

  • Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
  • For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
  • Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.

Where candidates lose points

Reading the most recent months of fraud and dispute rates as final

Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.

Recalibrating an underwriting cutoff on approved and funded applicants only

Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.

Generalising beyond the population the sample actually supports

State the frame the sample was drawn from and where it diverges from the population you want to talk about: time window, platform, geography, opt-in. If a group is excluded from the frame, either weight to known margins or narrow the claim rather than quietly extending it.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Questions & practice

Build a vintage delinquency table without pivot or unstack

Easy · vintage analysis · cohorts · groupby · cumulative max

fct_loan_performance_monthly gives loan_id, origination_month, months_on_book, days_past_due, charge_off_flag and restructured_flag. Produce a DataFrame with one row per origination_month and columns for months_on_book 0 through 12, each cell holding the share of that vintage's funded loans that had ever reached 90 or more days past due, or charge-off, by that age. You may not use pivot, pivot_table, crosstab or unstack. Cells for ages a cohort has not yet reached must be NaN rather than zero.

Approach

  1. Define the per-row indicator as days_past_due >= 90 or charge_off_flag, then take a cumulative maximum of it per loan ordered by months_on_book, because the metric is reached-by-age-m, not in-that-state-at-age-m.
  2. Deal with restructuring before the cumulative max. Restructuring resets days_past_due, so a restructured loan re-enters at current and, without the cumulative maximum carrying its pre-restructure worst state, reads as a cure.
  3. Fix the denominator once as the count of distinct loan_id per origination_month across the whole cohort. Prepaid and charged-off loans stop producing rows, so a denominator recomputed at each age silently shrinks exactly where losses land.
  4. Aggregate with groupby(['origination_month','months_on_book'])['ever_90'].sum(), then pre-build the output frame indexed by sorted origination months with integer columns 0 to 12 and assign from the grouped Series by .loc on its index.
  5. Mask cells beyond each cohort's maximum observed months_on_book so an immature cell reads NaN instead of an artificially low rate.

Follow-up

  • Two adjacent vintages diverge at months_on_book 6. How would you separate seasoning, mix shift and a genuine credit-quality change?
  • The three most recent vintages look best on this table. What do you check before saying so?
  • How does the table change if charge-off policy moved from 180 to 120 days past due partway through the series?

Estimate a delinquency roll-rate matrix and project twelve months

Hard · roll rates · markov chain · survivorship

fct_loan_performance_monthly gives loan_id, as_of_month_end, months_on_book, delinquency_bucket, charge_off_flag, prepaid_in_full_flag and restructured_flag. Build a month-to-month transition matrix over the five delinquency buckets plus absorbing charged_off and prepaid states. Loans that stop appearing must be routed to an absorbing state rather than dropped. Project the current book forward 12 months by repeated matrix multiplication and report the projected share reaching charge-off. Handle restructured_flag explicitly, and name one place the Markov assumption fails on this data.

Approach

  1. Build consecutive month pairs per loan by shifting as_of_month_end within loan_id, then verify the shifted value is exactly one month later. A gap is not a transition, it is an exit you have not resolved yet.
  2. Resolve exits before counting anything. A loan whose last row carries charge_off_flag moves to charged_off, one carrying prepaid_in_full_flag moves to prepaid, and one that disappears with neither is a data question to raise rather than silently discard, because discarding it is survivorship that inflates every cure rate.
  3. Count pairs into a 7 by 7 matrix and row-normalise. Assert every row sums to one and the two absorbing rows are the identity; a row that does not sum to one means exits were dropped.
  4. Decide and state the restructure rule. Restructuring resets days_past_due, so a dpd_60_89 to current move on a restructured loan is not a cure. Either give restructured loans their own state or carry the pre-restructure bucket, but do not let that move land in the cure cell.
  5. Project by taking the current bucket distribution as a row vector and multiplying by the matrix twelve times. Report the charged_off entry, and report it again from an all-current starting vector so the reader can see how much of the projection comes from loans that are already delinquent today.
  6. State the homogeneity failure plainly: transition rates depend strongly on months_on_book, so one pooled matrix applied to a book with a young mix understates early-life delinquency. If the mix is moving, estimate separate matrices by seasoning band.

Worked solution (45 min)

  1. Sort by loan_id and as_of_month_end, shift to form (from_state, to_state) pairs, and flag pairs whose month gap is not exactly one.
  2. For each loan's final row, assign the absorbing destination from charge_off_flag or prepaid_in_full_flag, and list loans that vanish with neither as an exception count to report.
  3. Apply the restructure rule, then build the 7 by 7 count matrix with a cross-tabulation over ordered state categories and row-normalise it.
  4. Assert row sums equal one and absorbing rows are the identity, then take the current month's bucket distribution as a row vector.
  5. Multiply twelve times, report the charged_off component, and repeat from an all-current vector for comparison.

Expected result: A 7 by 7 row-stochastic matrix with identity rows for charged_off and prepaid, a current-to-current diagonal typically above 0.95, roll rates rising with bucket depth, and a 12-month projected charge-off share of a few percentage points from an all-current start and materially higher from the actual book.

Verify it yourself

  • Every row sums to one within floating-point tolerance, and both absorbing rows are exactly the identity.
  • No mass sits in cells that skip a bucket upward, such as dpd_1_29 to dpd_60_89 in one month; if there is, the pairs are not actually consecutive months.
  • The projection from an all-current start is strictly below the projection from the real book distribution.
  • Backtest by estimating on months up to t minus 12 and comparing the projected distribution with the realised one at t; a systematic optimistic gap points at dropped exits or at seasoning, not at noise.

Follow-up

  • How would you validate the projection against what actually happened, and over what window?
  • The cure rate out of dpd_30_59 rose five points last quarter. What are the candidate explanations and how would you separate them?
  • When would you prefer a vintage curve to a roll-rate projection, and why?

Write integrity checks for the authorization and settlement lifecycle

Easy · data quality · minor units · fx reconciliation

You are given fct_payment_authorization as a pandas DataFrame with auth_id, requested_at, amount_minor, transaction_currency, auth_result, decline_reason_code, is_reversal, parent_auth_id, captured_at, captured_amount_minor, settled_at, settlement_amount_minor, settlement_currency and settlement_fx_rate. Write a function returning one row per integrity check with the check name, failing row count, failing share and up to five example auth_id values. Cover at least six checks, one of which reconciles captured_amount_minor against settlement_amount_minor through settlement_fx_rate. Partial capture, zero-amount verification and a decline with no capture are all legitimate and must not be flagged.

Approach

  1. Separate contract violations from observations before writing any code: an approved row carrying a decline_reason_code is structurally impossible, while a capture two days after requested_at is merely slow and belongs in a different severity tier.
  2. Express each check as a boolean mask over the whole frame and collect the masks in a dict, so the summary table is one comprehension over mask.sum() rather than a row loop.
  3. For the reconciliation, leave minor units before comparing: expected = captured_amount_minor / 10exponent[transaction_currency] * settlement_fx_rate * 10exponent[settlement_currency]. Build the exponent table covering zero-decimal and three-decimal currencies instead of assuming two everywhere.
  4. Guard the legitimate cases explicitly so each mask fires only on the genuine contradiction: captured_amount_minor below amount_minor is partial capture, amount_minor of zero on an approved row is account verification, a null captured_at on a declined row is correct.
  5. Sort the output by failing share times a stated severity weight, because a check firing on 0.01 percent of rows can still be the one that breaks a ledger reconciliation.

Follow-up

  • Which of these would you run as a blocking pipeline assertion and which as a monitored metric, and why?
  • The FX check fails on 3 percent of rows, all in one settlement currency. How do you decide between a data bug and a rounding convention?
  • How would you detect that a currency's minor-unit exponent is wrong in your reference table, using only the transaction data?

How would you use SQL window functions to calculate a rolling average of customer spend over the last 30 days?

Medium · sql and data manipulation

Approach

  1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  2. Say which table is the grain you start from, and join outward from it.
  3. State the window function and its partition and ordering out loud before writing it.

Follow-up

  • How does the query change if the join becomes one-to-many?
  • What breaks if events arrive late or out of order?

How do you optimize a query that is performing slowly on a multi-terabyte dataset?

Medium · sql and data manipulation

Approach

  1. Compute rates by summing numerator and denominator separately, never by averaging rates.
  2. Check whether any join is one-to-many before aggregating, or the sums inflate.
  3. State the window function and its partition and ordering out loud before writing it.

Follow-up

  • What breaks if events arrive late or out of order?
  • How does the query change if the join becomes one-to-many?

Given a table of transaction logs, how would you identify the first and last purchase date for every customer?

Medium · sql and data manipulation

Approach

  1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  2. Compute rates by summing numerator and denominator separately, never by averaging rates.
  3. Say which table is the grain you start from, and join outward from it.

Follow-up

  • What breaks if events arrive late or out of order?
  • How does the query change if the join becomes one-to-many?

Accident-quarter loss ratio on earned rather than written premium

Hard · loss ratio · earned vs written · development age

From fct_policy_period_monthly, compute the accident-quarter loss ratio by product_line: incurred losses, being paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, over earned_premium_minor for the same accident quarter. State explicitly whether loss_adjustment_expense_minor is included and apply that choice consistently. Also output the same ratio computed on written_premium_minor so the two can be compared. The table holds current values with no valuation-date snapshot. Say in one line which comparison this schema cannot support and what you would need to support it.

Approach

  1. Derive the accident quarter from as_of_month with date_trunc, and note that the table already attributes losses to the month of the loss event while earning premium pro rata into the same month, which is what makes the two sides comparable at all.
  2. Aggregate earned_premium_minor, written_premium_minor and the three loss components to product_line and accident quarter in one pass, keeping loss adjustment expense as its own column so the inclusion choice is a final-select decision rather than something buried in a CTE.
  3. Compute both ratios side by side and a third column for their difference, because the size and sign of that difference is a direct read on whether the book grew or shrank in the quarter.
  4. State the limitation plainly: every row carries today's reserve estimate, so each accident quarter is observed at a different development age and a cross-quarter comparison mixes development with underwriting. A fixed development age needs a valuation-date dimension, that is one row per accident period per valuation, which this table does not have.
  5. Guard against the mirror-image error on the numerator by confirming ibnr_reserve_minor is non-zero on recent quarters; if it is null or zero there, the recent periods are understated twice over and the series is not usable.

Worked solution (40 min)

  1. CTE quarterly: group fct_policy_period_monthly by product_line and date_trunc('quarter', as_of_month), summing earned_premium_minor, written_premium_minor, paid_loss_minor, case_reserve_minor, ibnr_reserve_minor and loss_adjustment_expense_minor.
  2. Final SELECT: build incurred_minor as the three loss components plus the LAE column, with the LAE inclusion written as a named expression so the choice is visible on the page.
  3. Emit loss_ratio_earned and loss_ratio_written, both cast to numeric, plus their difference and the written-to-earned premium ratio.
  4. Order by product_line and accident quarter, and append the one-line note about the missing valuation dimension to the query as a comment.

Expected result: One row per product_line and accident quarter with earned and written premium, incurred losses, both ratios and their difference. In quarters where written premium exceeds earned, loss_ratio_written is strictly below loss_ratio_earned, and the gap widens with the growth rate.

Verify it yourself

  • For a mature product line with flat volume, summed earned premium over a full policy term is close to summed written premium, so the two ratios converge there.
  • The sign of loss_ratio_earned minus loss_ratio_written matches the sign of written premium minus earned premium in every quarter, which is an arithmetic identity and not a data finding.
  • Recent accident quarters have non-zero ibnr_reserve_minor; a zero there means the numerator is missing its tail and the ratio is understated.
  • Within a quarter, loss_ratio_earned divided by loss_ratio_written equals written_premium_minor divided by earned_premium_minor exactly, because both ratios share one numerator. Any deviation means the LAE term was built into one ratio and not the other, which is the real test that the inclusion choice was applied consistently.
  • Recomputing with LAE excluded scales each quarter's ratio by incurred_minor / (incurred_minor + lae_minor). That factor is not constant across quarters unless LAE happens to be booked as a fixed fraction of incurred losses, so a uniform shift is evidence about how LAE is loaded, not evidence that the query is self-consistent — use the previous check for that.

Follow-up

  • Written premium exceeds earned premium by 18 percent this quarter and by 3 percent two years ago. What happened to the book, and what does it do to each ratio?
  • How would you build a development triangle from a valuation-dated version of this table, and what would you use the chain-ladder factors for?
  • Statutory presentation conventionally takes the expense ratio on written premium while the loss ratio uses earned. How do you avoid a combined ratio that quietly mixes the two bases?

How do you balance short-term conversion gains with long-term customer retention when designing product metrics?

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

How would you design a metric to measure the success of a new loyalty program feature?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

A key engagement metric has dropped suddenly; how would you perform a root cause analysis to diagnose the issue?

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

What are the common pitfalls in A/B testing, and how do you mitigate them?

Medium · a/b testing and experimentation

Approach

  1. State the primary metric and the minimum effect worth shipping, then size the test.
  2. Name the guardrails that would stop a launch even on a positive primary result.
  3. Say whether units interfere with each other, and switch design if they do.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • What would you do if you could not randomise at all?

How do you determine the required sample size for an experiment to achieve statistical significance?

Medium · a/b testing and experimentation

Approach

  1. Name the guardrails that would stop a launch even on a positive primary result.
  2. Say whether units interfere with each other, and switch design if they do.
  3. State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • How would you handle interference between treated and control units?

Size a two-week test on the ecommerce authorization path

Easy · power · mde · ratio metrics · clustering

Your team wants to test a new decline-retry policy on the ecommerce channel. The metric is the dollar-weighted authorization approval rate from fct_payment_authorization: approved amount_minor over attempted amount_minor, after collapsing retry chains within a 15-minute window on (card_token_id, merchant_id, amount_minor) and excluding is_reversal rows and zero-amount verifications, all converted to one reporting currency. Baseline is 87 percent on roughly 900,000 collapsed attempts in two weeks. Randomisation is by customer_id, mean six attempts per customer. Give the minimum detectable effect at 80 percent power, two-sided alpha 0.05, and say what you would change if it is too large.

Approach

  1. Compute the count-weighted binomial anchor first, because it is the number everyone expects and you need it to show why it is wrong. With n = 450,000 attempts per arm and p = 0.87, MDE = (1.96 + 0.8416) * sqrt(2p(1-p)/n).
  2. State that the real metric is a ratio of two sums with a random denominator, not a Bernoulli mean, so the binomial standard error is not the right one. Linearise: Var(R) is approximately Var(Y_i - R * D_i) / (n * Dbar^2), where Y_i and D_i are the approved and attempted amount totals for customer i, n is the number of customers, and Dbar is the mean attempted amount per customer.
  3. Apply the clustering correction, since assignment is by customer and attempts within a customer are correlated. Ask for or estimate the intra-customer correlation; the design effect is 1 + (m - 1) * rho with m = 6, so rho = 0.05 gives 1.25 and inflates the MDE by sqrt(1.25) = 1.118.
  4. Do not assume the amount-weighted variance; estimate it by bootstrapping customers from the last eight weeks of history and resampling whole customers, which captures both the skew in amount_minor and the within-customer correlation in one step.
  5. If the MDE exceeds the plausible effect, list the levers in order of cost: extend duration (MDE falls as 1/sqrt(n)), restrict to the segment where the rule can bind at all rather than diluting across all traffic, apply CUPED on the customer's pre-period approved amount, or switch the primary to the count-weighted rate and demote the dollar-weighted rate to a secondary read.

Worked solution (20 min)

  1. Split 900,000 collapsed attempts into 450,000 per arm and compute the binomial anchor: sqrt(2 * 0.87 * 0.13 / 450000) = sqrt(5.027e-7) = 7.090e-4; multiply by 2.8016 to get 1.99e-3.
  2. Derive the customer count: 900,000 / 6 = 150,000 customers, 75,000 per arm, and note that this is the true sample size for inference.
  3. Apply the design effect at rho = 0.05: 1 + 5 * 0.05 = 1.25, so the count-weighted MDE becomes 1.99e-3 * 1.118 = 2.22e-3.
  4. State that the dollar-weighted MDE requires the linearised variance and cannot be derived from p alone; specify the customer-level bootstrap that would produce it and note it will be larger because amount_minor is right-skewed.

Expected result: Count-weighted MDE is about 0.20 percentage points unclustered and about 0.22 percentage points at a design effect of 1.25, against an 87 percent baseline (roughly 0.26 percent relative). The dollar-weighted MDE is not computable from the stated inputs and must be estimated by a customer-level bootstrap; the candidate should say so rather than quote the binomial figure for it.

Verify it yourself

  • z(0.975) + z(0.80) = 1.9600 + 0.8416 = 2.8016; any MDE quoted with a different multiplier is using different power or alpha than stated.
  • Doubling the test duration should shrink the MDE by exactly sqrt(2) = 1.414 if the traffic mix is stable; if a candidate's arithmetic does not, the formula is wrong.
  • Run the bootstrap on two historical periods split A/A: the bootstrap standard error of the difference times 2.8016 should reproduce the analytic MDE.
  • A/A splits of historical data should reject at roughly 5 percent; materially more means the variance model is understated.

Follow-up

  • The rule only changes behaviour on declined attempts, which are 13 percent of traffic. How does restricting the analysis population to attempts that could have triggered the rule change both the MDE and the estimand?
  • How would your sizing change if the split were 90/10 instead of 50/50, and why is the loss more than proportional?
  • Currency: the metric sums amounts converted to one reporting currency. Would you use settlement_fx_rate or a rate table pinned at test start, and what does the choice do to variance?

Manual review rate doubles on the day two changes ship

Hard · definition change · model version · population stability

Manual review rate on fct_loan_application, defined as decided_by = 'manual' over decisions in ('approve','decline'), stepped from 9 to 18 percent on a single day. That day a new model_version went live, and the same release changed which referral paths are recorded as manual. Columns: submitted_at, channel, bureau_score, model_pd_12m, model_version, policy_rule_hits, decision, decided_by, decision_at. Attribute the step across definition, model and population, with a number against each.

Approach

  1. Establish the shape before the cause. A definition or code change produces a step at the deploy timestamp; a population change produces a ramp. Plot at hourly granularity either side of the release and read which one this is.
  2. Dual-run the definition. Recompute the metric under both the old and the new recording rules across several months of history. If the historical series under the new rule sits near 18 percent throughout, that part of the step is definitional and should be restated, not investigated.
  3. Hold the model constant and decompose the residual by policy_rule_hits identifier. A referral rule keyed to a score threshold shows its own step, and naming the rule is what makes the finding actionable to the policy owner.
  4. Test the score, not the applicants. Compare model_pd_12m distributions by model_version on the same applications where shadow scores exist; otherwise compare adjacent weeks with a population stability index, treating the customary 0.1 and 0.25 marks as heuristics rather than tests, and assess calibration separately from ranking.
  5. Test the applicants, not the score. Compare channel mix, bureau_score distribution and null-bureau share across the boundary. If those are stable, population is not a driver and should be reported as approximately zero rather than left unquantified.
  6. Deliver three components that sum to the observed step, each with the team that owns it.

Follow-up

  • If the new model is better calibrated, is a higher referral rate the correct outcome or a threshold nobody retuned?
  • How would you have caught the recording change before it reached the dashboard?
  • What is the swap set here, and which swap group would you inspect first?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

  • Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
  • Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
  • Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

  • Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
  • On the third attempt, write the grain of every CTE as a comment before writing its body.
  • Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

  • Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
  • Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
  • Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

  • Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
  • Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
  • Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

  • Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
  • Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
  • Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

  • Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
  • Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
  • Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

  • Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
  • Re-read only your own notes from this week, and open no new material.
  • Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Explaining a method to someone who will not follow the maths

Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.

Tell me about a time you coached a junior team member or peer. How did you support their development?

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. State the situation in two sentences and spend the rest on your reasoning.
  3. Pick a story where you drove the decision, not one where you observed it.

Follow-up

  • How did you know the outcome was caused by your change?
  • What would you do differently if you ran that project again?

Explain an incomplete dispute chart to a non-technical executive

Easy · dispute maturity · stakeholder communication · right-censoring

A finance lead is looking at first-chargeback rate by transaction month, built from fct_card_dispute joined to fct_payment_authorization on auth_id and attributed to requested_at. The last three months slope sharply down and the lead wants to announce a fraud improvement at tomorrow's review. Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, so those months are not complete. In five minutes, with no statistics vocabulary, explain why the decline is not yet evidence and say exactly what you would put on the slide instead.

Approach

  1. Lead with the mechanism in the listener's own terms, not with the statistical name for it: a dispute is attributed to the month the transaction happened, but it can be filed up to roughly 120 days later, so recent months contain only the disputes filed so far.
  2. Show completeness rather than arguing about the rate: for each transaction month, plot the share of its eventual disputes already filed, estimated from months that are fully matured. The last three months will sit visibly below 100 percent.
  3. Replace the chart with two artefacts: a matured series that stops 120 days back and is labelled final, and a development-factor estimate for the immature months drawn as a dashed range and labelled an estimate.
  4. Hand over one sentence the executive can repeat without you in the room: the recent months look better because the disputes have not arrived yet, not because fewer will arrive.
  5. Offer a weekly signal they can watch instead, such as the risk-score mix of approved volume or the decline-rule hit rate, and state up front what it does and does not predict.

Follow-up

  • The deck ships tomorrow regardless. What exactly goes on the slide, and what wording do you insist on?
  • How would you estimate the development factors, and how would you notice if they had shifted?

Turn a one-line fraud-number request into a scoped brief

Easy · scoping · metric definition · denominators

A stakeholder messages: what is our fraud rate, and is it going up? You have fct_payment_authorization, fct_card_dispute and dim_customer. At least four defensible answers exist: count-weighted or value-weighted, attributed to the transaction month or to the dispute filing month, and gross or net of recoveries and successful representments. You get one reply before someone else produces an uncaveated number. Write that reply: the clarifying questions you ask, the single default you will produce if nobody answers, and what the default excludes.

Approach

  1. Establish the decision behind the question first, because a risk-rule change, a board number and a merchant contract negotiation need different denominators, and asking which one is not stalling.
  2. Offer a short menu rather than an open question: a stakeholder can choose between two named options but cannot specify a denominator from scratch.
  3. Commit to a default so the reply is useful even if nobody answers, for example net fraud loss in basis points of settled volume, attributed to the requested_at month, matured months only.
  4. State the exclusions in the same breath as the default: non-fraud dispute categories, transaction months with less than 120 days of maturity, and first-party abuse that arrives coded as consumer_dispute.
  5. Give a delivery time for the default and a longer one for the fuller cut, so the choice between them carries a visible cost.

Follow-up

  • They come back wanting it by merchant for a contract negotiation. What changes in the definition and in the maturity rule?
  • How would you separate first-party abuse from third-party fraud in this data, and what would you refuse to conclude from the split?