Interview guides · Remitly

Remitly Data Scientist Interview Questions & Guide 2026

Practice explaining analysis before a Remitly data scientist interview: the recommendation, the uncertainty, the follow-ups.

The 60-second brief

At Remitly, a Data Scientist is a strategic partner tasked with transforming vast amounts of transactional and behavioral data into actionable insights that power the global movement of money. You are not just crunching numbers; you are a builder and a re-imaginer who helps the company solve complex problems at the intersection of finance, technology, and global trust. Whether optimizing pricing models for the SMB business line or analyzing customer segments across diverse geographies, your work directly influences the financial lives of millions of users.

Most of the loop measures decision-making under uncertainty rather than recall. You are scored on whether you state your assumptions, commit to an estimate you can defend, and say explicitly what evidence would change it.

Focus areas

Size an experiment before anyone launches it
Pick a randomisation unit that respects interference
Decompose a metric move by segment and mix

Overview

This role is highly collaborative and requires a unique blend of technical rigor and business acumen. You will work closely with product, engineering, and leadership teams to design experiments, build predictive models, and provide data-driven recommendations that shape the company’s trajectory. Success at Remitly requires a deep curiosity about human behavior and a commitment to delivering "peace of mind" to customers, making this an ideal position for those who want their analytical contributions to have a tangible, real-world impact.

The Remitly interview process

Remitly candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Hiring Manager Interview

Expect a live problem with pieces of it missing, closer to a conversation than an exam. A metric moved, or somebody wants to know whether a change worked, and you are asked how you would find out. The manager is watching the first ninety seconds, specifically whether you establish what decision hangs on the answer before you start proposing methods. Candidates who open with a technique get steered back. Once the decision is clear, describe what the data would look like if the story were true, and say what you would accept as evidence that it is not.

What to demonstrate

Whether you fix the decision the analysis serves before choosing an approach
How you continue when you are told the data you just asked for does not exist
Whether you state what would change your mind, not only what would confirm the hypothesis you started with
How you size an effect before you have measured it

How to prepare

Take a metric you know well and practise explaining in under two minutes the four things that could have moved it and how you would separate them
Pick a recent launch or experiment and write the single number you would ask for first, plus what you would conclude if it came back flat
Practise being interrupted: have someone remove a data source halfway through your answer and carry on without restarting

Team Interaction

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Technical Screen

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Case Studies

A case has a fixed clock, and a good deal of what is being scored is how you spend it. Thirty to forty-five minutes buys one pass across the whole problem or a deep read of one part of it, and choosing between those is the work rather than a compromise forced on you. Announce the shape early: the structure you are using, the branch you think carries the decision, and what you are setting aside. An answer that is thorough for the first third and silent on the recommendation reads worse than one that is rougher throughout and lands.

What to demonstrate

Whether a visible structure appears in the opening minutes and survives the rest of the case
Whether the depth goes to the branch that carries the decision, rather than the branch you find most comfortable
Whether you say what you are leaving out and why, instead of quietly omitting it and hoping nobody asks

How to prepare

After each practice case, write down the branches you chose not to open and the reason for each, then check whether you said any of them out loud while the case was running. A branch you only cut privately reads to the interviewer as one you missed.
Redo a case you have already worked in half the time, deciding in advance which single branch you keep, then compare which version a listener would find more useful.
Write a two-sentence opening you can reuse, holding the restated question and your plan for the available time, and deliver it within the first ninety seconds of every practice run.

Where candidates lose points

Counting on an identity key that changes underneath the metric

visitor_id is per browser and per device, and it resets on cookie clearance, private browsing and platform privacy changes, so the distinct-visitor count drifts upward for reasons unrelated to reach. Any rate with visitors in the denominator therefore decays over time even when behaviour is constant, and any rate with visitors in the numerator inflates. The stitching at signup makes it worse in both directions: a user who signed up on mobile and returns on desktop is two visitors and one user, while a shared device is one visitor and several users. Decide which key each metric is counted on, write it into the definition, and when comparing a period before and after a platform privacy change, expect a level shift in every visitor-keyed metric and do not attribute it to the product.

Slicing a flat experiment until a segment reaches significance

Testing one metric across twenty segments at a nominal 5% level produces a significant result about two thirds of the time when nothing is happening anywhere, and the segment that surfaces is by construction the one with the most favourable noise. The reported effect in that slice is then badly overstated, because selection on significance conditions the estimate on being large. What makes it dangerous rather than merely wrong is that a post-hoc segment always has a plausible story attached, so it survives the meeting. The controls are declaring the small number of segments of interest before launch, correcting across the ones tested, and treating anything discovered afterwards as a hypothesis that needs its own adequately-powered test rather than a finding.

Reading experiment results before checking the arm split

Compare observed arm counts against the intended allocation ratio, not an assumed even split, and set the alarm far below the conventional 0.05: at 0.05 roughly one healthy experiment in twenty trips it, which is why sample-ratio checks usually run at p < 0.001 or stricter. The test's power scales with sample size, so it misses a real diversion on a small experiment and fires on an imbalance too small to move the estimate on a very large one. A flag means go find the assignment or logging fault before reading any outcome, not report a mismatch.

Comparing periods without accounting for seasonality or day-of-week

Compare whole weeks against whole weeks and check whether the same swing appeared in prior cycles or prior years before attributing it to anything you changed. Weekday and weekend populations often differ enough that a Tuesday-to-Saturday comparison is meaningless.

Questions & practice

If we were to launch a new subscription model for our SMB customers, what data points would you prioritize?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Walk me through your process for building a financial model from scratch.

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

Rebuild per-visitor ordering without groupby convenience methods

Easy · pandas · vectorisation · window logic

You have a DataFrame of 2 million fct_event rows with visitor_id, occurred_at_utc and event_id, unsorted and containing duplicate timestamps within a visitor. Produce three new columns: event_rank, the 1-based position of the event within its visitor ordered by occurred_at_utc; seconds_since_prev, the gap to that visitor's previous event, NULL for the first; and is_first_for_visitor. You may use sort_values, shift, cumsum, numpy and boolean masking. You may not use groupby.transform, groupby.apply, groupby.cumcount, groupby.rank or merge_asof. Break timestamp ties on event_id.

Approach

Sort once by ['visitor_id', 'occurred_at_utc', 'event_id'] and reset the index. The whole exercise reduces to row arithmetic on a sorted frame, and the tiebreak on event_id is what makes the result reproducible across runs.
Mark visitor boundaries with is_first = df['visitor_id'].ne(df['visitor_id'].shift()). This is the single fact every other column derives from.
Compute seconds_since_prev as the diff of the timestamp column, then overwrite it with NaT/NaN wherever is_first is True. The shift crosses the boundary between visitors and will otherwise hand the first row of each visitor the last event of the previous one.
Build event_rank from a running counter that resets at boundaries: take a global cumulative position (np.arange(len(df))) and subtract, per row, the global position at which that visitor started. Get the start position by forward-filling the positions where is_first is True, which is a cumsum-free reset and is O(n).
Verify against the forbidden method once, as a test rather than as the implementation, and confirm the two agree on every row.

Worked solution (20 min)

Sort on the three-key tuple and reset_index(drop=True).
Compute is_first via .ne(.shift()), which is True for row 0 because the shifted value is NaN.
pos = np.arange(len(df)); start = pd.Series(np.where(is_first, pos, np.nan)).ffill(); event_rank = (pos - start + 1).astype(int).
gap = df['occurred_at_utc'].diff().dt.total_seconds(); gap[is_first] = np.nan.
Assert event_rank equals df.groupby('visitor_id').cumcount() + 1 on the sorted frame.

Expected result: Three columns on the sorted frame: event_rank starting at 1 for every visitor and increasing by 1 with no gaps, seconds_since_prev null exactly where is_first_for_visitor is True, and is_first_for_visitor summing to df['visitor_id'].nunique().

Verify it yourself

is_first_for_visitor.sum() equals the number of distinct visitor_id values.
seconds_since_prev.isna().sum() equals that same number.
event_rank.max() per visitor equals that visitor's row count, and event_rank is non-negative everywhere.
A visitor with two events at the identical timestamp gets ranks 1 and 2 with a gap of 0.0, ordered by event_id.

Follow-up

The frame does not fit in memory. How does your approach change if you can only process one visitor-partitioned chunk at a time?
occurred_at_utc is client-supplied and sometimes runs backwards within a visitor. Does your seconds_since_prev go negative, and should it?
How would you extend this to reset the counter at every change of surface as well as visitor?

Pair flow starts to completions within a thirty-minute bound

Medium · event-streams · conditional-aggregation · flow-funnel

fct_event holds event_id, flow_id, flow_instance_id, event_name, occurred_at_utc, surface, app_version. Using only rows where event_name is 'flow_started', 'flow_completed' or 'error_shown', compute the daily core-flow completion rate by surface and app_version. Numerator: distinct flow_instance_id whose flow_completed occurs within 30 minutes of its flow_started with no error_shown for the same instance in between. Denominator: distinct flow_instance_id with a flow_started that day. Report rows with a NULL flow_instance_id as a separate coverage count, not inside either side.

Approach

Collapse the stream to one row per flow_instance_id using conditional aggregation: MIN(occurred_at_utc) FILTER (WHERE event_name = 'flow_started') AS started, MIN(...) FILTER (WHERE event_name = 'flow_completed') AS completed, MIN(...) FILTER (WHERE event_name = 'error_shown') AS first_error. This makes the 'first' semantics explicit and avoids a self-join entirely.
Apply the predicates on that single row: completed IS NOT NULL AND completed <= started + interval '30 minutes' AND (first_error IS NULL OR first_error > completed). Note the last clause encodes 'in between' literally, so an error surfaced after a successful completion does not disqualify the attempt.
Bucket by started::date, surface and app_version. Splitting by app_version is not decoration: a completion-rate regression is nearly always confined to one client build, and a pooled daily rate hides it behind the installed base.
Count NULL flow_instance_id rows as a separate coverage figure. They cannot be attributed to an attempt at all, so they belong in neither numerator nor denominator, and their share tells you how much of the rate is unmeasurable.
For any roll-up to week or to all surfaces, re-sum the numerator and denominator rather than averaging the daily rates.

Follow-up

occurred_at_utc is client-supplied. What do you do with an instance whose completion timestamp precedes its start?
flow_instance_id is missing on 4 percent of starts for one app_version. What can and cannot you conclude about that build?
An instance starts at 23:55 and completes at 00:04. Which day owns it, and what does the alternative do to the daily series?

Weekly visit-to-signup conversion split by acquisition channel

Easy · funnel · conversion · aggregation

From fct_session (session_id, visitor_id, started_at_utc, referrer_channel, is_bot_flagged, consent_state) and fct_event (visitor_id, occurred_at_utc, event_name), compute visit-to-signup conversion for one ISO week, split by channel. session_id is the unique key of fct_session. Denominator: distinct visitor_id with a session starting in the week, is_bot_flagged = FALSE and consent_state <> 'denied'. Numerator: those visitors with a 'signup_completed' event in the same week. Label each visitor with the referrer_channel of their first session in the window. Return channel, visitors, signups and rate, plus one all-channel total row.

Approach

Build a visitor spine that is one row per visitor: filter sessions to the week, drop is_bot_flagged and consent_state = 'denied', then take the first session per visitor with ROW_NUMBER() OVER (PARTITION BY visitor_id ORDER BY started_at_utc, session_id) = 1 to carry the channel label. Collapsing to one row here is what makes the channel buckets mutually exclusive and the totals additive.
session_id is the unique key, so that ordering is total and the label is reproducible. If the table carried no unique key you would have to write an explicit tie rule instead, because two sessions on different channels at the identical timestamp would otherwise label the visitor differently between runs.
Attach the outcome as a semi-join (EXISTS on a signup_completed event for that visitor inside the same week) rather than a join to the event table, so a visitor who fires the event twice does not count twice and inflate the numerator past the denominator.
Aggregate with COUNT() as visitors and COUNT() FILTER (WHERE signed_up) as signups, and compute the rate as signups::numeric / NULLIF(visitors, 0) so an empty channel returns NULL rather than a division error.
Produce the total with GROUP BY GROUPING SETS ((channel), ()), which re-sums numerator and denominator for the total row. Averaging the channel rates gives a different and wrong number whenever channel volumes differ, which they always do.
Verify the spine before trusting the output: COUNT(*) must equal COUNT(DISTINCT visitor_id), and the per-channel visitor counts must sum to the total row.

Worked solution (20 min)

Write the filtered session CTE and check its row count against an unfiltered count, so you know how much volume the bot and consent filters removed.
Add the ROW_NUMBER first-session pick and assert one row per visitor.
Add the EXISTS outcome flag and aggregate with FILTER.
Add GROUPING SETS for the total and format the rate to four decimal places.
Spot-check one channel by hand: pull its visitor list, count signups directly, compare.

Expected result: One row per referrer_channel plus a total row, with visitors, signups and a decimal rate. Channels with zero signups appear with rate 0, not missing. The total rate lies strictly between the minimum and maximum channel rate unless one channel holds all the volume.

Verify it yourself

Per-channel visitor counts sum exactly to the total row's visitor count.
COUNT(*) equals COUNT(DISTINCT visitor_id) on the spine CTE.
Replacing the EXISTS with a plain inner join changes the signup count only if duplicate signup_completed events exist; if it does change, say by how much.
Every rate is between 0 and 1.
Re-running the first-session pick with session_id removed from the ORDER BY returns the same labels; if it does not, visitors hold same-timestamp sessions on different channels and the tie rule is doing real work.

Follow-up

The denominator is distinct visitors. If a browser release shortens cookie lifetime, what happens to this rate, and how would you tell that apart from a genuine drop?
A visitor's first session is direct and their signup session is paid search. Your label says direct. When is that the wrong answer for the decision being made?
How do you roll four weeks into a month, and why is averaging the four weekly rates wrong?

How would you measure the impact of a pricing change on customer retention?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

We are seeing a dip in conversion in a specific corridor; how would you investigate the root cause?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

What metrics would you track to determine the health of a new product feature?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

Explain the difference between A/B testing and multivariate testing in a real-world scenario.

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Say whether units interfere with each other, and switch design if they do.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

How would you design an experiment to test the elasticity of our pricing?

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
Say whether units interfere with each other, and switch design if they do.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

Define success for halving the free trial length

Hard · monetisation · cohort horizon · trials

A proposal cuts the free trial from 30 days to 14. Trial-to-paid conversion is defined on fct_subscription_period: numerator, subscription_id whose first row with is_first_paid_period = TRUE has period_status IN ('active','past_due') and period_start_utc no later than 14 days after trial end; denominator, subscription_id whose first row has period_status = 'trialing' and period_start_utc in the cohort week. dim_account (account_id, account_created_at_utc, signup_surface) is also available and joins to fct_subscription_period on account_id. Explain why conversion rate alone cannot decide this, define the metric that can, and state the horizon and lag your readout needs. Deliverable: the decision metric and the readout schedule.

Approach

Show the incomparability numerically before arguing about it: at a fixed calendar readout date, a larger share of the 14-day arm's cohorts have completed trial end plus 14 days plus settlement, so the short arm leads mechanically and the lead shrinks as both arms mature.
Fix the censoring first, and do not mistake a rescaling for a fix. The given conversion rate already has trial starts as its denominator, so 'paid accounts per 1,000 trial starts' is that same ratio multiplied by 1,000 and answers nothing new. What the censoring needs is a fixed cohort age: read both arms at account_created_at_utc plus 60 days, which clears the long arm's full 30 + 14 + 3 = 47-day path to a settled first payment, so neither arm is censored at readout.
Then move the denominator upstream, which is a separate fix and the only one that catches a higher share of a smaller population: paid accounts per 1,000 dim_account rows created in the cohort week. Precondition — this diverges from the conversion rate only when trial length is visible before the trial starts, on pricing pages, in ads or in the signup flow. If the arms are assigned at trial start and nothing upstream differs, trial starts per 1,000 new accounts is equal across arms by construction and the two metrics are the same comparison rescaled. Report that ratio per arm and say which regime you are in rather than assuming one.
Add the leg neither denominator can see: a shorter trial converts users who had less time to reach activation, so read month-3 gross revenue churn and week-4 retention of the converted cohort, and read activation at trial day 7 to see whether the converted population changed composition.
Audit 'past_due' before comparing, since the definition counts it as converted: report the rate with and without it per arm, because a gain that sits entirely inside past_due is a billing artefact rather than a conversion effect.
Be explicit that the twelve-month consequence is not observable inside a planning cycle: pre-register that the decision is made on the 60-day metric, that the twelve-month cohort read will be published as a check, and what action follows if the two disagree.

Worked solution (35 min)

Tabulate, at a fixed calendar readout date, the share of each arm's cohorts that have completed trial end plus 14 days plus 3 days of settlement, and show the gap.
Define the decision metric as paid accounts per 1,000 dim_account rows created in the cohort week, evaluated at account_created_at_utc plus 60 days for both arms, and write the 30 + 14 + 3 = 47 arithmetic that justifies 60 as the common clock.
Report trial starts per 1,000 new accounts per arm beside it, so the reader can see whether the upstream denominator is doing any work in this test or whether it is arithmetically pinned to the conversion rate.
Compute the earliest honest readout date from the enrolment window: 60 days after the last account creation in the window plus a 3-day settlement lag, with no partial-cohort comparison permitted before it.
Add the quality leg: month-3 gross revenue churn, week-4 retention of converted accounts, and day-7 activation per arm.
Report the conversion rate with and without past_due per arm, and state which version the decision uses.

Expected result: A decision metric stated as paid accounts per 1,000 new accounts at a fixed 60-day cohort age measured from account creation, the trial-start rate published per arm so it is explicit whether that denominator change carries any information here, a readout date computed from the enrolment window, and at least two quality metrics capable of overturning a conversion-rate win.

Verify it yourself

Both arms must have identical elapsed time at readout: compute min and max of (readout_date minus account_created_at_utc) per arm and confirm the two ranges match.
Trial starts per 1,000 new accounts must be reported per arm. If that ratio is equal across arms, the conversion rate and the per-new-account metric are the same comparison rescaled and cannot diverge — so never present them as two independent confirmations. Only a gap in that ratio gives the per-new-account metric information the conversion rate does not carry.
The past_due share of the numerator should be similar across arms; a gap there is a payment-failure effect and does not belong in a conversion claim.

Follow-up

What happens to a user who would have converted on day 20, and how would you detect that population in the data?
The short arm has lower day-7 activation but higher conversion. Reconcile those two facts into one story.
What randomisation unit do you use here, and what goes wrong with the obvious alternatives?

Separate a definition edit from a behaviour change

Easy · definition-change · metric-versioning · cohorts

Seven-day activation rate is documented as core actions on at least 2 distinct UTC dates inside the first seven days after account_created_at_utc, and every cohort published through last week was computed on that definition. The dashboard now reads 38% for the signup cohort that started Monday, against 31% for the cohort before it. Two things landed in that week: an onboarding change, and a commit to the metric's SQL that you have not read yet and that came with no documentation update. You have dim_user, fct_event and the metric's version history. Deliver two numbers: how many of the 7 points are behaviour and how many are the definition. Then say which published cohorts are no longer comparable.

Approach

Diff the metric SQL across the commit boundary and enumerate every changed predicate individually: the distinct-day threshold, the is_internal exclusion, the is_core_action source, the window bounds, and the identity key. A single commit routinely changes more than one, and each has its own sign.
Dual-run both definitions over the same 26 cohort weeks. The definition effect is the level shift between the two series computed on identical data, which isolates it from anything that happened in the world.
Read the behaviour effect only from the old-definition series, comparing the new cohort to its own trailing cohorts. The new-definition series cannot answer the behaviour question because it has no pre-period.
Explain the mechanism of the changed predicate rather than only its size. Lowering a distinct-day threshold from 2 to 1 admits every user who acted once in a single session, which is a large and low-intent population, so a several-point jump is the expected magnitude and not a surprise.
State the restatement plan: pick one definition, backfill the full history on it, and annotate the break date. A series carrying two definitions is not a time series.

Follow-up

The onboarding team wants the 1-day threshold kept because it is easier to move. What is your argument, and what would change your mind?
How would you make a definition change visible to every consumer of this metric without relying on people reading a changelog?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

The project you argued against

Saying no well is a senior skill and it is rarely rehearsed. Think of a time you told someone their analysis was not worth doing, or that the experiment could not answer their question at the sample size available. Explain what you offered instead. Refusal without an alternative reads as obstruction rather than judgement.

How do you handle missing or noisy data when building a predictive model?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Handle a request for numbers supporting a decision already made

Hard · integrity · framing · stakeholder

A senior leader has already decided to sunset a plan tier and asks you for the analysis showing it is the right call. Accounts on that tier carry 6% of MRR at constant FX and have the highest licensed-seat utilisation in the book. The leader's support matters to your next review cycle, and the decision is being presented in four days. Deliver what you produce, what you decline to produce, and the exact sentence you will say in the meeting where the number appears on a slide.

Approach

Recognise what is being probed: whether you can find the legitimate request inside an illegitimate framing instead of either complying or refusing on principle. The generic answer promises to push back; the strong one produces something genuinely useful and states its limits in the room, without ambushing anybody.
Separate the decision from the justification. Sunsetting the tier may be correct for reasons the data does not hold, such as support cost, roadmap surface area or sales motion. What you decline is a one-sided document. What you produce is the case read both ways, which also happens to be more useful to the leader.
Build the symmetric analysis: MRR at risk at constant FX, the share of affected accounts with a plausible migration path given seats_licensed and billing_term, the recovery rate assumed for that migration and where it came from, and the downside case in which high-utilisation accounts treat the sunset as a reason to re-evaluate the vendor entirely.
Surface the inconvenient fact privately and early. The highest seat utilisation in the book is a retention signal, and the leader should hold it before the room does, so they can incorporate it rather than be caught by it.
Agree the meeting sentence in advance with the leader, so that nobody is surprised. Something to the effect that the tier is 6% of MRR and its accounts are the most heavily used in the book, and that the case for sunsetting rests on cost and focus rather than on revenue. That is true, it supports the decision on its real grounds, and it stops the deck claiming the numbers endorse it.
Decide your own line before you need it: what you will not put your name to, and that the route if asked anyway is your own manager rather than a confrontation in the meeting.

Follow-up

The deck circulates with your analysis included and the downside case removed. What do you do, and by when?
What changes if the honest analysis says the sunset is clearly the wrong call?
How do you write the same memo when the leader is your skip-level and the meeting is tomorrow?

Turn an ambiguous onboarding question into a measurable metric

Easy · scoping · metric definition · stakeholder

Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.

Approach

Recognise what is being probed: whether you convert a goal into a computable predicate without stalling for requirements or guessing in silence. Listing clarifying questions is the generic answer; shipping a defensible default alongside them is the strong one, because the review is in two days and it will happen with or without you.
Infer the decision behind the request. A question about whether onboarding works, arriving before a planning cycle, usually means whether to staff it next quarter. That points at a rate with visible headroom over several cohorts, not at a descriptive dashboard.
Write the three questions so that each one changes the SQL. Which population, all signups or only self-serve from dim_user.signup_surface. What counts as working, reaching a core action or completing the onboarding flow_id. Against what bar, last quarter's cohorts or a stated target.
Propose the default explicitly: seven-day activation on weekly signup cohorts. Numerator, users with is_core_action = TRUE events on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). Denominator, the signup cohort with is_internal = FALSE. Publish with an eight-day lag, and state that the two-distinct-days threshold is a frozen choice rather than a discovery.
Name the exclusion in the same breath as the number. The series shows whether users activate; it does not establish that onboarding caused the level, which needs a staged rollout or an experiment.

Follow-up

The director replies that they meant the onboarding flow specifically, not activation. What changes in the query and in the caveats?
Your cohort metric needs an eight-day lag and the review is in two days. What do you present, and how do you label it?
Two of your three questions come back unanswered. Which one do you refuse to proceed without?