Articles

Monzo Data Scientist Case Study: Turn SQL Results into a Product Recommendation

Prepare for the Monzo Data Scientist case study by checking SQL denominators, customer mix, causal claims, and the evidence behind a product recommendation.

For a Monzo Data Scientist case study, prepare to move from an ambiguous product question to a defensible recommendation. Inspect who the SQL counted, how it defined outcomes, and what could explain the difference. Then choose the next action the evidence supports. A correct query can still answer the wrong decision.

Official: Monzo's April 27, 2026 hiring article distinguishes the take-home SQL/Python task and technical discussion from the final open-ended case study. The latter explores a realistic product or business scenario, assumptions, metrics, trade-offs and reasoning towards a recommendation. It does not say every final case study requires live SQL. Monzo Data Science interview article

The savings-reminder case below uses invented customers and outcomes; it is not a Monzo assignment, product result or internal assessment rubric.

Know which interview conversation you are preparing for

Monzo's official article describes technical review of take-home logic, testing and choices, alongside analytical and communication discussion. Its final case study asks candidates to structure a scenario and challenge assumptions. Prepare to explain the connection between your evidence and the decision, while following the format in your own invitation. Official DS hiring explanation

Candidate evidence boundary: we did not locate two independent completed DS reports establishing the same current recruiting-cycle format. A Monzo employee's May 2022 retrospective mentions metric formulation, SQL, visualization and a case study, but it is historical first-person evidence, not confirmation of a 2026 task or timer. Historical employee account

Do not import an Analytics Engineer modelling exercise or an ML interview's algorithm expectations into this role. Our preparation thesis is narrower: a product recommendation should remain defensible after someone questions its denominator, customer mix and causal interpretation.

Start with a decision rather than a dashboard

Our original scenario concerns a fictional savings feature. A product manager asks whether to roll out a reminder encouraging eligible customers to complete a savings transfer. Existing data compares customers who received the reminder with customers who did not. Delivery was not randomized.

The decision is whether to launch broadly now, continue with a controlled test, or abandon the idea. Clarify the eligible population, how reminders were selected, the analysis cutoff, and what harm a launch could create. An increase in message clicks is not automatically an improvement in the saving behavior the product wants to support.

Define a primary outcome as the proportion of eligible customers completing at least one successful savings transfer within seven days of cohort entry. Pending or failed transfers do not count. A customer with ten successful transfers counts once for this outcome; transfer frequency and amount would be separate metrics.

Our recommendation is to pair that outcome with guardrails such as reminder opt-outs, complaints and unintended friction. These are proposed measures for the invented exercise, not disclosed Monzo targets. Ask the interviewer which customer outcomes and constraints should change the decision.

Preserve the denominator in the SQL contract

The denominator contains eligible customers with a complete seven-day observation window, including customers with no qualifying activity. Starting from successful transfers would exclude non-converters and inflate the rate. Starting from recipients alone would also change the question from the eligible population to a selected exposure group.

Before joining, assert one cohort row per customer. If eligibility records have versions, define the baseline version and selection rule. Do not resolve conflicting reminder flags by arbitrarily taking the most convenient row.

This original BigQuery outline assumes unique cohort rows and a consistent event-time definition:

WITH cohort AS (
  SELECT user_id, segment, reminder_received, enrolled_at
  FROM sandbox.eligible_customers
  WHERE enrolled_at <= TIMESTAMP_SUB(@as_of, INTERVAL 7 DAY)
), outcomes AS (
  SELECT c.user_id, c.segment, c.reminder_received,
         MAX(IF(e.event_id IS NULL, 0, 1)) AS completed_action
  FROM cohort c
  LEFT JOIN sandbox.activity e
    ON e.user_id = c.user_id
   AND e.event_type = 'completed_savings_transfer'
   AND e.occurred_at >= c.enrolled_at
   AND e.occurred_at < TIMESTAMP_ADD(c.enrolled_at, INTERVAL 7 DAY)
  GROUP BY c.user_id, c.segment, c.reminder_received
)
SELECT segment, reminder_received,
       COUNT(*) AS eligible_customers,
       SUM(completed_action) AS converters,
       SAFE_DIVIDE(SUM(completed_action), COUNT(*)) AS conversion_rate
FROM outcomes
GROUP BY segment, reminder_received;

The left join preserves customers with zero events. Conditions on activity stay in the join, so missing activity does not remove a cohort row. Aggregating to customer grain before calculating rates prevents repeated activity from multiplying the denominator. BigQuery query syntax

The outline uses a named timestamp parameter and synthetic table names. It is not a query against Monzo data. A mature event-time window also needs sufficiently complete ingestion: a transfer that occurred yesterday but has not reached the warehouse is still missing from today's analysis.

Diagnose an aggregate result that reverses within groups

Suppose the original exercise produces this complete seven-day cohort summary:

New customers: comparison 80 / 400 = 20%; reminder 15 / 100 = 15%.
Tenured customers: comparison 60 / 100 = 60%; reminder 220 / 400 = 55%.
Observed total: comparison 140 / 500 = 28%; reminder 235 / 500 = 47%.

The aggregate difference is 19 percentage points, apparently favoring the reminder. Yet the reminder group is five percentage points lower within each customer segment. Its higher total is driven by a much larger share of tenured customers, whose baseline conversion is higher.

This is a constructed Simpson's-paradox example: the direction of an aggregate comparison differs from the within-group comparisons because the groups have different weights. No query syntax error is required. The arithmetic is correct while a claim that the reminder caused improvement is unsupported.

Standardize both groups to the same hypothetical 50/50 mix. The comparison becomes (20% + 60%) / 2 = 40%; the reminder becomes (15% + 55%) / 2 = 35%. This is a descriptive sensitivity check, not an estimated causal effect and not proof that the reminder harms customers.

Choose weights that represent the population you want to describe. Equal weights illustrate the reversal; a real target population might have a different composition. State the target mix before choosing weights, and keep both segment counts visible so a reviewer can reconstruct the calculation.

Separate association, attribution and incremental impact

A reminder can be sent preferentially to customers already likely to save. Conversely, it might target customers with unusually low recent engagement. Either selection rule makes recipients and non-recipients different before the message arrives.

The two-segment adjustment only checks the variables shown. Account tenure does not automatically capture balance, prior saving behavior, eligibility rules or seasonal differences. Those are hypotheses about confounding to investigate, not licenses to keep adjusting until a desired result appears.

Official product-analysis context: Monzo's CRM measurement article explains why attributed conversions and incremental impact differ, describing holdouts as part of its measurement approach. That is useful background for asking whether a reminder caused an additional action; it does not reveal the interview task or mandate a specific experiment design. Monzo CRM impact measurement

A click followed by a savings transfer is a sequence of events. It does not identify what the same customer would have done without the reminder. Do not describe the 47% recipient rate as “uplift” without specifying both the comparator and the evidence needed for causal interpretation.

Propose a controlled test with meaningful guardrails

Our proposed next step is a randomized reminder experiment among customers eligible before assignment. Use persistent customer-level assignment when repeated messages and actions can cross sessions. Keep the outcome window, assignment unit, eligible population and analysis rules fixed in advance.

Use an intention-to-treat comparison as the primary analysis: count eligible customers in their assigned group, even if they never open the reminder. Conditioning only on people who opened a message would introduce a post-assignment selection. Delivery and open rates can diagnose execution, but they answer different questions from the effect of offering the reminder policy.

Check assignment and data integrity before interpreting differences. Official methods reference: Microsoft Research describes sample-ratio mismatch as an unexpected, statistically assessed discrepancy between planned and observed variant counts. It can arise in assignment, execution, log processing or analysis. Equal overall counts alone do not establish that every segment or outcome join is trustworthy. Microsoft Research on SRM

Specify a minimum practically useful change, baseline assumptions and the observation duration needed to estimate it. Avoid announcing a sample size from the four-cell illustration: power depends on the chosen effect, variability, significance policy and randomization design. The case does not include a p-value or confidence interval from an actual experiment.

For guardrails, define the denominator as carefully as for conversion. Complaints per assigned customer differ from complaints per message sent, especially if one arm receives more messages. Monitor opt-outs and friction alongside the desired action; choose stopping and review rules before seeing a favorable result.

Turn uncertainty into a concise product recommendation

A decision memo should begin with the action, then evidence, limitations and the next test. Our original memo could read:

Hold the broad rollout and run a randomized eligibility-based test. The observed recipient conversion is 47%, compared with 28% for non-recipients, but the recipient cohort contains more tenured customers. Within both shown segments, recipients convert five percentage points less often; common 50/50 weighting gives 35% versus 40%. These observational comparisons establish neither benefit nor harm. Validate cohort completeness and run a customer-level test of seven-day completed-action conversion, with opt-out and complaint guardrails.

The memo specifies the next action while leaving the causal effect unresolved. If resources prevent an experiment, identify which observational assumptions you would need and which decisions remain too uncertain. A weaker method can support a bounded exploration without supporting a full rollout.

Explain what would change the recommendation. Reliable experimental evidence of meaningful improvement with acceptable guardrails could justify expansion. Evidence of customer harm could justify stopping. A broken assignment or outcome pipeline calls for repairing measurement, not a product verdict.

Handle misleading charts and case-study challenges

If shown only the aggregate bars, ask for population counts, segmentation and assignment logic before recommending launch. Use a zero baseline for rate bars and label percentage-point differences separately from relative percentage changes. The 19-point difference is not a 19% relative increase.

When asked why the reminder performs worse within both groups, offer testable explanations rather than a confident story. The reminder may have targeted customers with lower recent saving activity. Delivery timing may differ, or logging may be incomplete. Name the evidence that would distinguish those hypotheses.

A late change to the outcome definition requires recomputing both numerator and denominator consistently. If the interviewer asks for recurring behavior rather than any completed action, the present seven-day conversion query does not answer it. Define a new measure and acknowledge what the current result can still support.

Keep the answer concise without overstating certainty. A short case-study answer can still state the decision, metric, data checks and strongest limitation. For rehearsal, explain the four-cell reversal aloud, then write the memo without looking at SQL. This is our exercise, not a claimed Monzo timing requirement.

Sources and Further Reading

Research checked October 4, 2026. Follow your own invitation for format and tool rules. All cohort counts, SQL, chart values, metric definitions and recommendations here are original practice material.