cswork / libraryOpen practice workspace

Interview guides · Chemours

Chemours Data Scientist Interview Questions & Guide 2026

Chemours data scientist prep: probability, sampling distributions, and stating the assumption behind every test you run.

The 60-second brief

As a Data Scientist at Chemours, you play a crucial role in leveraging data to drive innovation and enhance decision-making across the organization. This position is essential for transforming raw data into actionable insights, which directly impacts product development, operational efficiency, and strategic initiatives. You will collaborate with cross-functional teams to address complex challenges, optimize processes, and enhance product performance, contributing to Chemours’ commitment to sustainability and high-quality solutions.

If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.

Focus areas

  • Turn a vague request into a measurable question
  • Separate novelty effects from durable behaviour change
  • Decompose a metric move by segment and mix

Overview

In this role, you will engage with diverse datasets and employ advanced analytical techniques to develop models that support business objectives. You will be involved in projects that span various domains, from chemical manufacturing optimization to customer experience enhancement, making your contributions vital to the success of both Chemours and its customers. The position is dynamic and intellectually stimulating, offering opportunities to work on significant real-world problems that affect industries worldwide.

The Chemours interview process

Chemours candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

  • Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
  • Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
  • The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

  • Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
  • Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
  • Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Video Interview

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

  • Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
  • Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
  • Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

  • For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
  • Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
  • For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Meeting with Hiring Manager

Underneath the questions about your past work sits a resourcing question. Given four things worth doing and one of you, which gets done and what happens to the rest? Managers ask because that is the daily texture of the job, and because the answer shows whether you rank work by effort or by what it changes. The weak version sorts by personal interest or by whoever asked most insistently. The strong version ties each candidate piece of work to a decision somebody downstream is waiting on, and then names the one you would drop and who you would tell.

What to demonstrate

  • Whether you rank work by the decision it unblocks or by how interesting the method is
  • How you describe a request you declined, and whether you can say who you said it to
  • Whether your sense of how long something takes survives one follow-up question about the messy part
  • How you decide something is good enough to hand over unfinished

How to prepare

  • Write out your current queue and, next to each item, the decision that stays stalled until it lands. Anything with no waiting decision becomes your example of work you would cut
  • Rehearse turning down a plausible stakeholder request out loud, including the smaller alternative you offered instead
  • Have one case where you shipped a rough answer early and one where you refused to, with the reason that separated them

Where candidates lose points

Watching an experiment daily and stopping when it crosses significance

A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.

Counting on an identity key that changes underneath the metric

visitor_id is per browser and per device, and it resets on cookie clearance, private browsing and platform privacy changes, so the distinct-visitor count drifts upward for reasons unrelated to reach. Any rate with visitors in the denominator therefore decays over time even when behaviour is constant, and any rate with visitors in the numerator inflates. The stitching at signup makes it worse in both directions: a user who signed up on mobile and returns on desktop is two visitors and one user, while a shared device is one visitor and several users. Decide which key each metric is counted on, write it into the definition, and when comparing a period before and after a platform privacy change, expect a level shift in every visitor-keyed metric and do not attribute it to the product.

Reading an observational correlation as a causal effect

Name the confounder you are most worried about and the design that would remove it: an experiment, a difference-in-differences with a checked pre-period trend, an instrument, or a regression discontinuity. When none is available, state which direction the bias likely runs and bound the claim accordingly.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Questions & practice

Explain a machine learning project you have worked on. What challenges did you face?

Medium · machine learning and modelling

Approach

  1. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  2. Say how the offline result would be validated online before it is trusted.
  3. Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • How would you choose the decision threshold, and who owns that choice?

Simulate the false positive cost of repeated peeking

Medium · simulation · peeking · type i error

Quantify the cost of peeking. Simulate a two-arm experiment with no true effect: each arm accumulates Bernoulli conversions at a base rate of 0.10 up to 40,000 units per arm. Run a two-sided two-proportion z-test at alpha 0.05 at ten equally spaced interim points, and record whether the test ever crossed. Report the false positive rate over at least 10,000 replications, alongside the rate for a single look at the final sample only. Use a fixed seed and report a Monte Carlo standard error on both figures.

Approach

  1. Generate each replication as two cumulative sums of Bernoulli draws, then read the interim points off the cumulative arrays. Regenerating data at each look would make the looks independent, which destroys exactly the dependence the exercise is about: later looks share data with earlier ones.
  2. Use the pooled-variance two-proportion z: p_pool = (x1+x2)/(n1+n2), z = (p1-p2) / sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2)), reject when |z| > 1.96. State that the normal approximation is fine here because the smallest look has roughly 400 expected conversions per arm.
  3. Vectorise across replications rather than looping: draw a (reps, n) array of uniforms, threshold at 0.10, cumsum along axis 1 and slice the ten look indices. A per-replication loop at 10,000 by 40,000 is unnecessarily slow.
  4. Record the any-cross indicator per replication, take the mean, and compute the Monte Carlo standard error as sqrt(p*(1-p)/reps) so the reported figure comes with its own precision.
  5. Report the single-look rate in the same run as a control. If it does not land near 0.05, the bug is in the test statistic and not in the peeking argument.

Worked solution (30 min)

  1. rng = np.random.default_rng(seed); for memory, batch the replications in chunks and accumulate the any-cross count across chunks.
  2. Per chunk: draw (chunk, 40000) uniforms per arm, x = (u < 0.10).cumsum(axis=1), slice columns at indices 3999, 7999, ..., 39999.
  3. Compute the ten z statistics vectorised over the chunk, take crossed = (np.abs(z) > 1.96).any(axis=1).
  4. Aggregate: peek_rate = total_crossed / reps; single_rate = mean of |z_final| > 1.96; mc_se = sqrt(p*(1-p)/reps) for each.

Expected result: The single-look rate lands at 0.05 within about 0.005 at 10,000 replications. The ten-look rate lands near 0.19, which with 10,000 replications has a Monte Carlo standard error of about 0.004, so anything in roughly 0.18 to 0.20 is consistent and anything near 0.40 indicates independent redraws.

Verify it yourself

  • The single-look control recovers 0.05 within two Monte Carlo standard errors; if it does not, the z statistic is wrong and the headline number is meaningless.
  • The crossing rate is monotone in the number of looks: 1 look near 0.05, 2 near 0.08, 5 near 0.14, 10 near 0.19.
  • Halving the base rate to 0.05 leaves the peeking rate essentially unchanged, since the result is about the test's random walk and not about the conversion level.
  • Re-running with the same seed reproduces the figure exactly.

Follow-up

  • Re-run with 40 looks instead of 10. Why does the curve flatten rather than continue rising linearly?
  • Among the replications that crossed, what is the mean observed lift, and why is it not zero?
  • What does an O'Brien-Fleming boundary or an always-valid confidence sequence change about this simulation, and what does each cost in power?

Permutation test for a difference in conversion rates

Medium · permutation test · resampling · inference

Write a two-sided permutation test from scratch for a difference in conversion rates, using no scipy hypothesis function. Input: a DataFrame with unit_id, variant in {control, treatment} and converted in {0,1}, one row per randomisation unit. Compute the observed difference in proportions, then build the null distribution by reshuffling the variant labels while holding each arm's size fixed. Report the p-value as (1 + the count of permuted statistics at least as extreme in absolute value) / (B + 1) with B at least 10,000, and return the permutation distribution.

Approach

  1. Name the null being tested: the sharp null that each unit's outcome is the same under either label. That is what licenses permuting labels, and it is stronger than the null of equal means, which matters when someone asks whether the test is valid under unequal variances.
  2. Extract converted to a single numpy array of 0s and 1s and record n_treatment. Every permutation is then just a reshuffle of one array, and the treatment mean is the mean of the first n_treatment entries of the shuffled array.
  3. Vectorise the B permutations with rng.permuted on a tiled 2-D array, or with argsort of a (B, n) random matrix. A Python loop calling np.random.shuffle B times is correct but roughly an order of magnitude slower and often runs past the time limit.
  4. Use the +1 correction in both numerator and denominator. Without it a p-value of exactly 0 is reportable, which is false: the observed labelling is itself one of the permutations, so the smallest attainable p-value is 1/(B+1).
  5. Compare the resulting p-value against a two-proportion z-test as a sanity check. At these sample sizes they should agree closely; a large divergence means the statistic or the shuffle is wrong, not that the permutation test found something subtle.

Follow-up

  • The arms are 200 and 20,000 units. Does the permutation test stay valid, and what happens to its resolution at B = 10,000?
  • Give a 95 percent confidence interval for the difference. Can you get it from this permutation distribution, and if not, what would you run instead?
  • The randomisation unit is user_id but the outcome is per session. What breaks, and what is the fix?

Weekly visit-to-signup conversion split by acquisition channel

Easy · funnel · conversion · aggregation

From fct_session (session_id, visitor_id, started_at_utc, referrer_channel, is_bot_flagged, consent_state) and fct_event (visitor_id, occurred_at_utc, event_name), compute visit-to-signup conversion for one ISO week, split by channel. session_id is the unique key of fct_session. Denominator: distinct visitor_id with a session starting in the week, is_bot_flagged = FALSE and consent_state <> 'denied'. Numerator: those visitors with a 'signup_completed' event in the same week. Label each visitor with the referrer_channel of their first session in the window. Return channel, visitors, signups and rate, plus one all-channel total row.

Approach

  1. Build a visitor spine that is one row per visitor: filter sessions to the week, drop is_bot_flagged and consent_state = 'denied', then take the first session per visitor with ROW_NUMBER() OVER (PARTITION BY visitor_id ORDER BY started_at_utc, session_id) = 1 to carry the channel label. Collapsing to one row here is what makes the channel buckets mutually exclusive and the totals additive.
  2. session_id is the unique key, so that ordering is total and the label is reproducible. If the table carried no unique key you would have to write an explicit tie rule instead, because two sessions on different channels at the identical timestamp would otherwise label the visitor differently between runs.
  3. Attach the outcome as a semi-join (EXISTS on a signup_completed event for that visitor inside the same week) rather than a join to the event table, so a visitor who fires the event twice does not count twice and inflate the numerator past the denominator.
  4. Aggregate with COUNT() as visitors and COUNT() FILTER (WHERE signed_up) as signups, and compute the rate as signups::numeric / NULLIF(visitors, 0) so an empty channel returns NULL rather than a division error.
  5. Produce the total with GROUP BY GROUPING SETS ((channel), ()), which re-sums numerator and denominator for the total row. Averaging the channel rates gives a different and wrong number whenever channel volumes differ, which they always do.
  6. Verify the spine before trusting the output: COUNT(*) must equal COUNT(DISTINCT visitor_id), and the per-channel visitor counts must sum to the total row.

Follow-up

  • The denominator is distinct visitors. If a browser release shortens cookie lifetime, what happens to this rate, and how would you tell that apart from a genuine drop?
  • A visitor's first session is direct and their signup session is paid search. Your label says direct. When is that the wrong answer for the decision being made?
  • How do you roll four weeks into a month, and why is averaging the four weekly rates wrong?

Paying accounts with no active seat in 28 days

Medium · anti-join · null-semantics · retention-risk

dim_account holds account_id, account_type, lifecycle_status, seats_licensed. fct_event holds account_id, user_id, occurred_at_utc, is_core_action, and its account_id is NULL for every signed-out and pre-signup event. Find accounts with lifecycle_status = 'active' and account_type <> 'internal' that had no distinct user complete a core action in the trailing 28 days. Return account_id, seats_licensed and days since that account's most recent core action, with NULL where the account has never emitted one. Order by seats_licensed descending.

Approach

  1. Build the recent-activity set first: fct_event rows with is_core_action = TRUE, occurred_at_utc >= now() - interval '28 days', and an explicit account_id IS NOT NULL. Making the NULL exclusion explicit in the CTE is what lets you reason about the anti-join afterwards.
  2. Express the exclusion with NOT EXISTS (correlated on account_id) or a LEFT JOIN with an IS NULL guard. Do not use NOT IN against this column: it is nullable, and SQL's three-valued logic turns the whole predicate UNKNOWN, returning zero rows.
  3. Compute last-seen separately as MAX(occurred_at_utc) per account over all history, LEFT JOINed on, so an account that has never emitted a core action (NULL) is distinguishable from one that went quiet six weeks ago. Those two cases have different causes and different owners.
  4. Rank by seats_licensed, or better by the account's current mrr_cents_constant_fx if you are allowed the subscription table, because a silent fifty-seat account is a renewal conversation and a silent one-seat account is noise.
  5. Before shipping, check whether the never-seen group is a cluster by signup date or surface. A block of accounts with no events at all is usually an instrumentation gap, not a set of customers who stopped using the product.

Worked solution (25 min)

  1. Count NULL account_id rows in fct_event over the window so you know the trap is live in this data rather than hypothetical.
  2. Write the active-account spine and the 28-day activity CTE.
  3. Write the anti-join with NOT EXISTS, then deliberately run the NOT IN version and record that it returns zero rows.
  4. Add the all-time MAX(occurred_at_utc) LEFT JOIN and derive days_since as a date difference.
  5. Split the output into 'quiet' and 'never seen' and eyeball the never-seen group for a shared signup window or surface.

Expected result: One row per silent active account with seats_licensed and days_since_last_core_action, ordered by seats_licensed descending. Accounts that never emitted a core action show NULL and are reported as a separate group rather than as the worst churn risks.

Verify it yourself

  • Silent accounts plus accounts with at least one active user equals the total count of active non-internal accounts.
  • The NOT IN variant returns zero rows, which is the demonstration that a NULL is present.
  • No account_id appears twice in the output.
  • Every days_since_last_core_action is either NULL or greater than 28.

Follow-up

  • How would you distinguish a genuinely idle account from one whose events lost their account_id after an instrumentation change?
  • Would you count on fct_event.account_id or resolve user_id through dim_user instead, and what does each choice miss?
  • Licensed-seat utilisation is the continuous version of this. How would you turn this boolean into that ratio?

How do you prioritize tasks when you have multiple deadlines?

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

What steps would you take to assess the effectiveness of a marketing campaign using data?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

If given a dataset with product sales data, how would you analyze it to uncover insights?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

How would you identify trends in customer feedback?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • Which segment would you cut first, and what would that rule out?

Describe how you would design an experiment to test a new product formulation.

Medium · a/b testing and experimentation

Approach

  1. Name the guardrails that would stop a launch even on a positive primary result.
  2. Name the randomisation unit first; it decides the variance and what the test can detect.
  3. Say whether units interfere with each other, and switch design if they do.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • How would you handle interference between treated and control units?

What statistical techniques do you find most useful in your work?

Medium

Approach

  1. Clarify what is being asked and what a complete answer would contain.
  2. Say what you would check first and why it is the highest-information step.
  3. Work from the decision backwards to the evidence you would need.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Randomise a shared workspace feature without contaminating control

Medium · randomisation unit · interference · design effect · clustering

A feature changes a collaborative surface inside a workspace: when one member uses it, other members of the same account see the result in their own view. You have dim_user (user_id, account_id, is_internal), dim_account (account_id, seats_assigned, lifecycle_status) and fct_event. Among active accounts the mean seats_assigned is 6, the coefficient of variation of that count is 1.5, and the intraclass correlation of the weekly core-action rate within an account is 0.10. Choose the randomisation unit, quantify what that choice costs in sample, and specify how you would compute inference.

Approach

  1. State the interference before choosing anything: a treated user changes what an untreated colleague sees, so user-level randomisation puts both arms inside one account and biases the contrast toward zero. Randomise on account_id.
  2. Price the clustering properly. With equal clusters the design effect is 1 + (m - 1) rho = 1 + 5(0.10) = 1.5. Sizes here are far from equal, so use 1 + ((CV^2 + 1) m - 1) rho = 1 + (3.25 x 6 - 1)(0.10) = 2.85. The equal-size shortcut understates the cost by nearly half.
  3. Decide the estimand before the estimator. An account-weighted mean gives every workspace one vote; a user-weighted mean lets the largest workspaces dominate. With this size skew the two can move in opposite directions, so pick the one the decision needs and write it down.
  4. Compute standard errors on the account, not the user: cluster-robust on account_id, or collapse each account to a single number and test those. Below roughly 40 clusters per arm, cluster-robust errors are biased downward, so use a wild cluster bootstrap or randomisation inference over the assignment.
  5. Buy back variance where you can. Stratify assignment by seat band and lifecycle_status before randomising, and decide in advance how the handful of very large accounts are handled, since one enterprise workspace can carry more users than a hundred single-seat ones.

Worked solution (30 min)

  1. Write the interference down: the outcome for user i depends on the treatment of other users in account(i), so the no-interference assumption fails at the user level and holds at the account level.
  2. Compute both design effects, 1.5 equal-size and 2.85 unequal-size, and use 2.85.
  3. Take the user-level sample requirement from the proportion shortcut, multiply by 2.85, then divide by the mean of 6 users per account to express it in accounts per arm.
  4. Specify the analysis: collapse to one row per account, regress the account-level outcome on variant with stratum fixed effects, and use a wild cluster bootstrap for inference.
  5. State the stopping rule up front: if the required account count exceeds the eligible population, the test is not runnable, and the alternatives are a longer window, a larger target effect, or a non-experimental read.

Expected result: Randomise on account_id. The unequal-size design effect is 2.85, so a user-level requirement of 12,000 per arm becomes 34,200 user-equivalents, which is 5,700 accounts per arm, with inference computed on accounts rather than users.

Verify it yourself

  • The design effect must be at least 1, and must equal 1 when rho = 0 or when mean cluster size is 1. A value below 1 means a term is inverted.
  • Recomputing the unequal-size formula with CV = 0 must return the equal-size value of 1.5.
  • The cluster-robust standard error on account_id should come out about sqrt(2.85) = 1.69 times the naive user-level one. A ratio near 1 means the clustering was not applied.

Follow-up

  • Suppose the feature is not workspace-scoped but changes a globally shared ranking model, so no clean cluster exists. What design gets you a causal read, and what does it cost you?
  • You have 900 eligible active accounts in total. Given the design effect, what absolute lift can this test detect, and is the honest answer 'do not run it'?
  • The intraclass correlation is an estimate from last quarter. What happens to your sizing if the true value is 0.25?

Gross revenue churn doubled with no cancellations behind it

Hard · billing-periods · fx · revenue-churn · decomposition

Gross monthly revenue churn computed from fct_subscription_period doubled from 1.8% to 3.6% in one month. The support queue shows no rise in cancellations and renewals look normal. You have fct_subscription_period with subscription_id, account_id, period_start_utc, period_end_utc, mrr_cents, mrr_cents_constant_fx, seats_billed, period_status, change_reason and canceled_at_utc, plus dim_account. Decompose the 1.8-point rise into named mechanisms, size each in points of the headline, and state the remainder you cannot explain.

Approach

  1. Reconstruct the numerator row by row and group it by change_reason before arguing about causes. A mid-period plan or seat change closes the current period row and opens a new one, so any implementation that treats a closed row as lost revenue books upgrades, downgrades and seat changes as churn; the change_reason breakdown of the numerator makes that visible in one query.
  2. Check the recognition timestamp. Churn belongs at period_end_utc, because revenue continues until the period ends, not at canceled_at_utc when the button was pressed. Then count period_end_utc rows per month across thirteen months: annual cohorts concentrate their period ends in the month twelve months after they were signed, so a spike that repeats in the same month last year is seasonality in the book, not an event this month.
  3. Recompute the whole numerator and denominator on mrr_cents_constant_fx. If the constant-currency figure is materially flatter, the move is an exchange-rate translation and belongs nowhere in a churn narrative.
  4. Check period_status handling. Rows with period_status = 'past_due' are dunning, not cancellation; a status-based rule counts them as loss while a period-end rule does not, and a dunning backlog can move the number by itself.
  5. Express every mechanism in points of the headline, sum them, and print the residual explicitly next to the month-to-month standard deviation of the prior twelve months. A decomposition without a stated remainder is a story, not an accounting.

Follow-up

  • Which of these mechanisms should be fixed in the metric definition and which should be reported as a genuine business fact?
  • How would you present a month whose churn is dominated by an annual cohort anniversary without the audience concluding the business is deteriorating?
  • What would you change so that an upgrade can never enter the churn numerator again, and how would you test that it worked?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

  • For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
  • For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
  • Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

  • Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
  • List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
  • Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

  • Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
  • Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
  • Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

  • Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
  • Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
  • Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

  • Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
  • Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
  • Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

  • Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
  • For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
  • Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

  • Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
  • Listen back and mark every moment you proposed a solution before the success metric existed.
  • Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

Can you describe a situation where you had to work with a difficult team member?

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Pick a story where you drove the decision, not one where you observed it.

Follow-up

  • What would you do differently if you ran that project again?
  • How did you know the outcome was caused by your change?

What motivates you to perform at your best?

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. Pick a story where you drove the decision, not one where you observed it.
  3. State the situation in two sentences and spend the rest on your reasoning.

Follow-up

  • What did you decide not to do, and why?
  • How did you know the outcome was caused by your change?

Quantify your own impact without claiming the topline you touched

Hard · self-assessment · attribution · communication

You are writing the impact section of your own review. Over the year you ran four experiments, one of which shipped and three of which were flat; you corrected the definition of gross monthly revenue churn so that cancellation is recognised at period_end_utc; and you built a self-serve funnel dashboard. Weekly active accounts rose 14% over the same period. Your reviewer knows the data well. Write the three impact claims you would defend, stating for each what you contributed, what evidence supports it, and what portion of the outcome you are not claiming.

Approach

  1. Recognise what is being probed: whether you apply to your own work the causal standard you would apply to somebody else's roadmap claim. Nearly everyone who would reject 'accounts that do Y retain better' will write 'I drove a 14% increase' without noticing it is the same error with a friendlier subject.
  2. Sort the work by the kind of evidence it can carry. The shipped experiment is the only item with a randomised estimate, so it is the only one where an effect size is defensible, and you claim the interval rather than the point estimate.
  3. Claim the three flat experiments as decisions prevented and price them. Features not built, or built differently, on evidence, with the engineering weeks reallocated as the number somebody else can verify. A defensible null is a delivered decision and should be written as one.
  4. Claim the definition fix as correctness, not as improvement. The old figure was overstated by a specific percentage and appeared in a specific set of recurring documents; the impact is the change it produced in the forecast built on top of it, not a change in churn itself.
  5. Claim the dashboard on usage and displacement: distinct weekly users of it, and the ad-hoc request count for six months before against six months after. If the request log does not exist, record the claim as unverified rather than estimating it upward.
  6. Disclaim the 14% explicitly and once. State that it cannot be separated from seasonality, other teams' launches and a pricing change, and bound your own contribution from above using the shipped experiment's interval converted into headline units.

Follow-up

  • Your shipped experiment's interval was +0.2pp to +1.4pp on activation. How much of the 14% can that account for, and how do you say so without undercutting yourself?
  • A peer in the same cycle claims the full 14%. What, if anything, do you do about it?
  • If you could only keep two of your three claims, which do you drop, and why that one?