cswork / libraryOpen practice workspace

Interview guides · Ipsos

Ipsos Data Scientist Interview Questions & Guide 2026

Diagnose a metric move for Ipsos data scientist prep: split by segment, rule out a logging change, separate mix shift from behaviour.

The 60-second brief

As a Data Scientist at Ipsos, you will sit at the intersection of advanced analytics and global market research. Ipsos is one of the world's leading market research firms, which means the data you work with is incredibly diverse, ranging from large-scale consumer surveys and social media listening streams to complex behavioral tracking databases. Your primary responsibility is to transform raw, unstructured, or complex data into sophisticated predictive models, segmented audience profiles, and automated analytical solutions that directly influence business strategies for global brands.

How much statistics you need depends on the flavour of the seat. Experiment-facing work wants you deep enough to notice that repeated looks at accumulating data inflate the false positive rate of a fixed-sample test; modelling-facing work wants estimation and honest uncertainty intervals.

Focus areas

  • Define numerator, denominator and window precisely
  • Separate novelty effects from durable behaviour change
  • Pick a randomisation unit that respects interference

Overview

The impact of this role is massive because Ipsos relies on its data science team to innovate beyond traditional research methodologies. You will build and scale machine learning pipelines, apply natural language processing (NLP) to open-ended survey text, and design statistical weighting methodologies that ensure data accuracy. It is a highly collaborative and intellectually stimulating environment where your technical output directly shapes the decisions of Fortune 500 executives and public policymakers.

The Ipsos interview process

Ipsos candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Conversation

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

  • Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
  • Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
  • Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

  • For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
  • Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
  • For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Technical Evaluation

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

  • Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
  • Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
  • Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

  • Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
  • Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
  • If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Technical and Behavioral Interviews

Rounds of this kind usually include one question about work that did not go well, and it is the part that carries the most information. Anyone can narrate a shipped win. What the interviewer learns from a project that stalled is how you behave without a result to hide behind: whether you noticed the problem yourself, how long it took, and who you told. Answers that route the failure onto a data pipeline or a reorganisation close the topic without answering it, and the follow-up comes back to your own part.

What to demonstrate

  • Whether you found the error yourself or someone else found it, and how long it sat before anyone knew
  • What you changed afterwards, stated as a check you now run rather than a lesson you now believe
  • Whether the mistake you choose has real cost attached, such as a quarter of misdirected roadmap or a metric that was reported upward, instead of one that flatters you

How to prepare

  • Choose a failure you caught yourself and be ready to say what tipped you off. A story where someone else caught it is still usable, but you will be asked why you missed it.
  • Write down the check you added afterwards and where it lives now, so the correction is a concrete artefact rather than a resolution.
  • Rehearse saying the cost out loud. Candidates shrink the number by instinct once the interviewer is in the room.

Where candidates lose points

Watching an experiment daily and stopping when it crosses significance

A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.

Treating last-touch attribution as the causal value of a channel

The attribution label on dim_user is the output of a rule that assigns full credit to whichever touch happened to be recorded last inside a lookback window, and that rule systematically rewards channels that sit close to the conversion, especially branded search and retargeting, which largely intercept demand that already existed. Reallocating spend on those labels moves budget toward the channels that are best at being last, which is why attributed return on ad spend often improves while total signups do not. Nothing in the touchpoint data can settle this, because the counterfactual of not running the channel was never observed. The credible reads are a geo holdout or a scheduled pause, sized in advance on the total-signups metric rather than on the attributed one, and the honest framing in the meantime is that the label describes correlation with conversion and not incremental contribution.

Ignoring interference between units in a marketplace experiment

Ask whether one unit's treatment can change another unit's outcome through shared inventory, a matching pool, a social graph or a common budget. Where it can, randomise at a level that contains the spillover, such as region or time slice, and say explicitly what that costs you in statistical power.

Reading an observational correlation as a causal effect

Name the confounder you are most worried about and the design that would remove it: an experiment, a difference-in-differences with a checked pre-period trend, an instrument, or a regression discontinuity. When none is available, state which direction the bias likely runs and bound the claim accordingly.

Questions & practice

Explain the difference between bagging and boosting, and when you would choose one over the other for market segmentation.

Medium · statistics and probability

Approach

  1. Translate the result into the decision it informs, in one plain sentence.
  2. Quantify uncertainty explicitly rather than reporting a point estimate alone.
  3. Say what the estimate is of, and over what population it generalises.

Follow-up

  • Which assumption here is most likely to be violated in practice?
  • What sample size would you need to detect an effect half this size?

How do you determine the appropriate sample size for a statistically significant consumer survey?

Medium · statistics and probability

Approach

  1. Sanity-check the answer against a simple bound or a simulated case.
  2. Write down the assumption the method needs before you use the method.
  3. Translate the result into the decision it informs, in one plain sentence.

Follow-up

  • Which assumption here is most likely to be violated in practice?
  • What sample size would you need to detect an effect half this size?

How do you deal with highly imbalanced classes in a predictive model (e.g., predicting rare consumer behaviors)?

Medium · machine learning and modelling

Approach

  1. Set a baseline first, so any model has something honest to beat.
  2. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  3. Say how the offline result would be validated online before it is trusted.

Follow-up

  • Where could label leakage enter this setup?
  • How would you choose the decision threshold, and who owns that choice?

What metrics would you use to evaluate the performance of an unsupervised clustering model?

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Check what information would not exist at prediction time, and exclude it.
  3. Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • Where could label leakage enter this setup?

Cluster bootstrap for a per-session rate randomised on users

Hard · cluster bootstrap · randomisation unit · variance

An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.

Approach

  1. Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
  2. Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
  3. Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
  4. Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
  5. Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.

Worked solution (35 min)

  1. per_user = df.groupby(['variant','user_id'])['converted'].agg(['sum','size']); split into two arrays per arm.
  2. point = (t_sum.sum() / t_n.sum()) - (c_sum.sum() / c_n.sum()).
  3. For b in range(B): idx = rng.integers(0, len(t_sum), len(t_sum)); ratio_t = t_sum[idx].sum() / t_n[idx].sum(); same for control; store the difference. Vectorise by drawing a (B, n) index matrix if memory allows.
  4. ci = np.percentile(diffs, [2.5, 97.5]); naive_se = sqrt(p_t*(1-p_t)/n_sessions_t + p_c*(1-p_c)/n_sessions_c); naive_ci = point +/- 1.96*naive_se.
  5. width_ratio = (ci[1]-ci[0]) / (naive_ci[1]-naive_ci[0]).

Expected result: A point estimate identical to the full-data ratio difference, a percentile interval containing that point estimate, and a width ratio greater than 1 whose value approximates sqrt(1 + (m-1)*rho) for the data's mean cluster size m and intraclass correlation rho. A ratio of approximately 1.0 means sessions were resampled instead of users.

Verify it yourself

  • Construct data where every user has exactly one session: the cluster and naive intervals then agree to within Monte Carlo error, since m = 1 makes the design effect 1.
  • Construct data where converted is constant within each user: rho is 1, and the width ratio should approach sqrt(m).
  • The bootstrap distribution's mean sits close to the point estimate; a large offset means the resampled arms have drifted in size.
  • Doubling B narrows the Monte Carlo noise in the percentile endpoints without shifting the interval's centre.

Follow-up

  • Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
  • Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
  • Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?

Explain how you would parse and extract specific entities from an unstructured text dataset using Python.

Medium · sql and data manipulation

Approach

  1. State the window function and its partition and ordering out loud before writing it.
  2. Say which table is the grain you start from, and join outward from it.
  3. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

  • What breaks if events arrive late or out of order?
  • How does the query change if the join becomes one-to-many?

How do you optimize a slow-performing SQL query that involves multiple joins on large consumer databases?

Medium · sql and data manipulation

Approach

  1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  2. Say which table is the grain you start from, and join outward from it.
  3. Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

  • How does the query change if the join becomes one-to-many?
  • What breaks if events arrive late or out of order?

Write a SQL query to join two tables and calculate the rolling average of survey responses over a specific time window.

Medium · sql and data manipulation

Approach

  1. State the window function and its partition and ordering out loud before writing it.
  2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  3. Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

  • How would you verify this result without re-running the same query?
  • What breaks if events arrive late or out of order?

Given a raw dataset in Python, write a script using Pandas to handle missing data, normalize numerical features, and export the cleaned data.

Medium · sql and data manipulation

Approach

  1. State the window function and its partition and ordering out loud before writing it.
  2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  3. Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

  • How does the query change if the join becomes one-to-many?
  • How would you verify this result without re-running the same query?

Weekly visit-to-signup conversion split by acquisition channel

Easy · funnel · conversion · aggregation

From fct_session (session_id, visitor_id, started_at_utc, referrer_channel, is_bot_flagged, consent_state) and fct_event (visitor_id, occurred_at_utc, event_name), compute visit-to-signup conversion for one ISO week, split by channel. session_id is the unique key of fct_session. Denominator: distinct visitor_id with a session starting in the week, is_bot_flagged = FALSE and consent_state <> 'denied'. Numerator: those visitors with a 'signup_completed' event in the same week. Label each visitor with the referrer_channel of their first session in the window. Return channel, visitors, signups and rate, plus one all-channel total row.

Approach

  1. Build a visitor spine that is one row per visitor: filter sessions to the week, drop is_bot_flagged and consent_state = 'denied', then take the first session per visitor with ROW_NUMBER() OVER (PARTITION BY visitor_id ORDER BY started_at_utc, session_id) = 1 to carry the channel label. Collapsing to one row here is what makes the channel buckets mutually exclusive and the totals additive.
  2. session_id is the unique key, so that ordering is total and the label is reproducible. If the table carried no unique key you would have to write an explicit tie rule instead, because two sessions on different channels at the identical timestamp would otherwise label the visitor differently between runs.
  3. Attach the outcome as a semi-join (EXISTS on a signup_completed event for that visitor inside the same week) rather than a join to the event table, so a visitor who fires the event twice does not count twice and inflate the numerator past the denominator.
  4. Aggregate with COUNT() as visitors and COUNT() FILTER (WHERE signed_up) as signups, and compute the rate as signups::numeric / NULLIF(visitors, 0) so an empty channel returns NULL rather than a division error.
  5. Produce the total with GROUP BY GROUPING SETS ((channel), ()), which re-sums numerator and denominator for the total row. Averaging the channel rates gives a different and wrong number whenever channel volumes differ, which they always do.
  6. Verify the spine before trusting the output: COUNT(*) must equal COUNT(DISTINCT visitor_id), and the per-channel visitor counts must sum to the total row.

Worked solution (20 min)

  1. Write the filtered session CTE and check its row count against an unfiltered count, so you know how much volume the bot and consent filters removed.
  2. Add the ROW_NUMBER first-session pick and assert one row per visitor.
  3. Add the EXISTS outcome flag and aggregate with FILTER.
  4. Add GROUPING SETS for the total and format the rate to four decimal places.
  5. Spot-check one channel by hand: pull its visitor list, count signups directly, compare.

Expected result: One row per referrer_channel plus a total row, with visitors, signups and a decimal rate. Channels with zero signups appear with rate 0, not missing. The total rate lies strictly between the minimum and maximum channel rate unless one channel holds all the volume.

Verify it yourself

  • Per-channel visitor counts sum exactly to the total row's visitor count.
  • COUNT(*) equals COUNT(DISTINCT visitor_id) on the spine CTE.
  • Replacing the EXISTS with a plain inner join changes the signup count only if duplicate signup_completed events exist; if it does change, say by how much.
  • Every rate is between 0 and 1.
  • Re-running the first-session pick with session_id removed from the ORDER BY returns the same labels; if it does not, visitors hold same-timestamp sessions on different channels and the tie rule is doing real work.

Follow-up

  • The denominator is distinct visitors. If a browser release shortens cookie lifetime, what happens to this rate, and how would you tell that apart from a genuine drop?
  • A visitor's first session is direct and their signup session is paid search. Your label says direct. When is that the wrong answer for the decision being made?
  • How do you roll four weeks into a month, and why is averaging the four weekly rates wrong?

A surrogate for twelve-month value inside a three-week test

Hard · surrogate metrics · experimentation · long-term value

A monetisation change — an earlier and harder paywall — will be tested for three weeks. The decision quantity is twelve-month cumulative net revenue per acquired account, which cannot be observed in three weeks. Available: fct_event, fct_session, fct_subscription_period (mrr_cents_constant_fx, period_status, change_reason, is_first_paid_period), dim_account, and fct_experiment_exposure (unit_type, unit_id, variant, first_exposed_at_utc, is_in_analysis_population). Construct a surrogate index readable at three weeks, state the assumption that makes it valid, name the mechanism that breaks it, and give the pre-registered rule for refusing to decide. Deliverable: the index, the assumption, and the refusal rule.

Approach

  1. State the surrogacy condition before building anything: a surrogate is valid only if the treatment's entire effect on twelve-month revenue runs through it. Then say where it fails here, because a paywall moves short-run revenue directly and long-run revenue through churn and through who becomes a payer at all.
  2. Fit the index on history rather than on intuition: regress twelve-month cumulative net revenue per account on features observable by day 21 — first-paid flag, activation days, week-3 core-action days, seats billed — using cohorts old enough to have a twelve-month outcome, and hold out a later cohort.
  3. Report the out-of-sample fit as the headline rather than the point estimate: the held-out R-squared and the calibration of predicted against actual deciles are what license any use of the index, and a decile plot catches the mis-calibration an R-squared hides.
  4. Name the bias direction explicitly: the relationship was estimated under the old paywall, so under a harder paywall the marginal payer is a different person and the index over-predicts their value, biasing the treatment arm optimistic.
  5. Surround the index with the direct three-week readouts it cannot contain — revenue per exposed unit, cancel-within-first-period rate, and the free-side signup and activation counts — and pre-register a numeric refusal rule, because the failure mode of a surrogate is being used confidently in exactly the case it was not fit for.

Worked solution (40 min)

  1. Assemble the training set: accounts with first_paid_at_utc at least twelve months old, their realised twelve-month cumulative net revenue, and the day-21 features.
  2. Fit on pre-change cohorts, evaluate on a held-out later cohort, and report both out-of-sample R-squared and predicted-versus-actual decile calibration.
  3. Write the surrogacy assumption as a falsifiable sentence, then write the specific mechanism in this treatment that violates it.
  4. Specify the direct arm readouts on fct_experiment_exposure with is_in_analysis_population = TRUE: revenue per exposed unit, cancel-within-first-period rate, and free-side signup and activation counts.
  5. Write the refusal rule with numbers attached, covering both the fit threshold and the free-side divergence case.

Expected result: A fitted surrogate with a reported out-of-sample R-squared and calibration, a written surrogacy assumption plus the mechanism that violates it, the resulting bias direction, and a numeric refusal rule agreed before launch.

Verify it yourself

  • Standard errors are clustered on fct_experiment_exposure.unit_id, the identifier of the randomisation unit. unit_type is the kind label for that unit — a single value such as 'user' across the whole table — so clustering on it collapses the design to one or two clusters and carries no information about who was randomised together. A per-event or per-session standard error on a user-randomised test is the opposite error: too small, and it will manufacture significance.
  • The analysis population is the exposed set with is_in_analysis_population = TRUE, and a sample-ratio check against the intended split passes before any outcome is read.
  • The index applied to the control arm should reproduce the historical relationship. If control is already mis-calibrated, the treatment estimate is not interpretable at all.

Follow-up

  • What evidence would make you trust a surrogate for this particular change rather than a different one?
  • The index reads plus 8 percent while three-week cancel-within-period is up. Which do you act on, and what do you tell the decision-maker?
  • How would you size this test, on what unit, and what does that do to the three-week horizon?

Randomise a shared workspace feature without contaminating control

Medium · randomisation unit · interference · design effect · clustering

A feature changes a collaborative surface inside a workspace: when one member uses it, other members of the same account see the result in their own view. You have dim_user (user_id, account_id, is_internal), dim_account (account_id, seats_assigned, lifecycle_status) and fct_event. Among active accounts the mean seats_assigned is 6, the coefficient of variation of that count is 1.5, and the intraclass correlation of the weekly core-action rate within an account is 0.10. Choose the randomisation unit, quantify what that choice costs in sample, and specify how you would compute inference.

Approach

  1. State the interference before choosing anything: a treated user changes what an untreated colleague sees, so user-level randomisation puts both arms inside one account and biases the contrast toward zero. Randomise on account_id.
  2. Price the clustering properly. With equal clusters the design effect is 1 + (m - 1) rho = 1 + 5(0.10) = 1.5. Sizes here are far from equal, so use 1 + ((CV^2 + 1) m - 1) rho = 1 + (3.25 x 6 - 1)(0.10) = 2.85. The equal-size shortcut understates the cost by nearly half.
  3. Decide the estimand before the estimator. An account-weighted mean gives every workspace one vote; a user-weighted mean lets the largest workspaces dominate. With this size skew the two can move in opposite directions, so pick the one the decision needs and write it down.
  4. Compute standard errors on the account, not the user: cluster-robust on account_id, or collapse each account to a single number and test those. Below roughly 40 clusters per arm, cluster-robust errors are biased downward, so use a wild cluster bootstrap or randomisation inference over the assignment.
  5. Buy back variance where you can. Stratify assignment by seat band and lifecycle_status before randomising, and decide in advance how the handful of very large accounts are handled, since one enterprise workspace can carry more users than a hundred single-seat ones.

Follow-up

  • Suppose the feature is not workspace-scoped but changes a globally shared ranking model, so no clean cluster exists. What design gets you a causal read, and what does it cost you?
  • You have 900 eligible active accounts in total. Given the design effect, what absolute lift can this test detect, and is the honest answer 'do not run it'?
  • The intraclass correlation is an estimate from last quarter. What happens to your sizing if the true value is 0.25?

Decide whether a one-day core-action drop is real

Easy · seasonality · day-of-week · late-arriving-data

A daily dashboard counts distinct fct_event.user_id with is_core_action = TRUE, filtered on occurred_at_utc, and is read at 09:00 UTC. This morning it shows yesterday down 22% against the day before. fct_event is partitioned on received_at_utc. You have fct_event, fct_session and dim_user with thirteen months of history. Deliver a one-paragraph verdict, escalate or do not escalate, with the evidence that settles it, before anyone proposes a product hypothesis.

Approach

  1. Identify which two weekdays the comparison actually spans, then pull the same weekday-pair transition for the last 52 weeks and place the observed 22% inside that distribution. A day-over-day comparison in a product with a weekday pattern is a comparison of two different populations, so the reference class is the same transition historically, not the prior day.
  2. Measure partition completeness rather than assuming it. For each of the last 30 days compute the share of that day's occurred_at_utc rows that had landed by 09:00 UTC the following morning, split by surface; mobile clients buffer events offline, so the freshest partition is systematically short and the shortfall is not uniform across surfaces.
  3. Recompute the same series keyed on received_at_utc. If the drop survives on both keys it is not a lateness artefact; if it exists only on occurred_at_utc it is the partition filling in.
  4. Check the two exclusion flags before segmenting anything: a change in is_bot_flagged coverage or a batch of is_internal accounts entering or leaving moves a distinct-user count with no user behaviour behind it.
  5. Only if the movement survives all of the above, begin the segment decomposition. Say explicitly in the verdict which of these four checks the movement passed, so the next reader does not repeat them.

Follow-up

  • What publication lag would you set for this dashboard, and how would you derive the number rather than pick it?
  • If you switch the metric to received_at_utc, what does that break for anyone comparing to historical figures?
  • How would you detect the same problem automatically, so a human does not have to notice it each morning?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

  • Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
  • Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
  • Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

  • Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
  • On the third attempt, write the grain of every CTE as a comment before writing its body.
  • Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

  • Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
  • Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
  • Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

  • Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
  • Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
  • Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

  • Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
  • Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
  • Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

  • Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
  • Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
  • Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

  • Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
  • Re-read only your own notes from this week, and open no new material.
  • Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Describe a time when you had to explain a complex statistical model to a non-technical client or stakeholder.

Medium · behavioural and stakeholder questions

Approach

  1. Name the disagreement or constraint, and how you resolved it with evidence.
  2. Quantify the outcome, including what you would not claim credit for.
  3. State the situation in two sentences and spend the rest on your reasoning.

Follow-up

  • How did you know the outcome was caused by your change?
  • What did you decide not to do, and why?

Turn an ambiguous onboarding question into a measurable metric

Easy · scoping · metric definition · stakeholder

Two days before a planning review, a director asks whether onboarding is working. You have dim_user (account_created_at_utc, signup_surface, is_internal), fct_event (is_core_action, flow_id, flow_instance_id, event_name, occurred_at_utc, received_at_utc) and fct_session. No further meeting with the director is possible before you start work. Deliver three clarifying questions you would send in writing, the metric you will compute in the meantime with its numerator, denominator, window and exclusions, and one sentence naming the question you are deliberately not answering.

Approach

  1. Recognise what is being probed: whether you convert a goal into a computable predicate without stalling for requirements or guessing in silence. Listing clarifying questions is the generic answer; shipping a defensible default alongside them is the strong one, because the review is in two days and it will happen with or without you.
  2. Infer the decision behind the request. A question about whether onboarding works, arriving before a planning cycle, usually means whether to staff it next quarter. That points at a rate with visible headroom over several cohorts, not at a descriptive dashboard.
  3. Write the three questions so that each one changes the SQL. Which population, all signups or only self-serve from dim_user.signup_surface. What counts as working, reaching a core action or completing the onboarding flow_id. Against what bar, last quarter's cohorts or a stated target.
  4. Propose the default explicitly: seven-day activation on weekly signup cohorts. Numerator, users with is_core_action = TRUE events on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). Denominator, the signup cohort with is_internal = FALSE. Publish with an eight-day lag, and state that the two-distinct-days threshold is a frozen choice rather than a discovery.
  5. Name the exclusion in the same breath as the number. The series shows whether users activate; it does not establish that onboarding caused the level, which needs a staged rollout or an experiment.

Follow-up

  • The director replies that they meant the onboarding flow specifically, not activation. What changes in the query and in the caveats?
  • Your cohort metric needs an eight-day lag and the review is in two days. What do you present, and how do you label it?
  • Two of your three questions come back unanswered. Which one do you refuse to proceed without?

Quantify your own impact without claiming the topline you touched

Hard · self-assessment · attribution · communication

You are writing the impact section of your own review. Over the year you ran four experiments, one of which shipped and three of which were flat; you corrected the definition of gross monthly revenue churn so that cancellation is recognised at period_end_utc; and you built a self-serve funnel dashboard. Weekly active accounts rose 14% over the same period. Your reviewer knows the data well. Write the three impact claims you would defend, stating for each what you contributed, what evidence supports it, and what portion of the outcome you are not claiming.

Approach

  1. Recognise what is being probed: whether you apply to your own work the causal standard you would apply to somebody else's roadmap claim. Nearly everyone who would reject 'accounts that do Y retain better' will write 'I drove a 14% increase' without noticing it is the same error with a friendlier subject.
  2. Sort the work by the kind of evidence it can carry. The shipped experiment is the only item with a randomised estimate, so it is the only one where an effect size is defensible, and you claim the interval rather than the point estimate.
  3. Claim the three flat experiments as decisions prevented and price them. Features not built, or built differently, on evidence, with the engineering weeks reallocated as the number somebody else can verify. A defensible null is a delivered decision and should be written as one.
  4. Claim the definition fix as correctness, not as improvement. The old figure was overstated by a specific percentage and appeared in a specific set of recurring documents; the impact is the change it produced in the forecast built on top of it, not a change in churn itself.
  5. Claim the dashboard on usage and displacement: distinct weekly users of it, and the ad-hoc request count for six months before against six months after. If the request log does not exist, record the claim as unverified rather than estimating it upward.
  6. Disclaim the 14% explicitly and once. State that it cannot be separated from seasonality, other teams' launches and a pricing change, and bound your own contribution from above using the shipped experiment's interval converted into headline units.

Follow-up

  • Your shipped experiment's interval was +0.2pp to +1.4pp on activation. How much of the 14% can that account for, and how do you say so without undercutting yourself?
  • A peer in the same cycle claims the full 14%. What, if anything, do you do about it?
  • If you could only keep two of your three claims, which do you drop, and why that one?