cswork / libraryOpen practice workspace

Interview guides · Corteva Agriscience

Corteva Agriscience Data Scientist Interview Questions & Guide 2026

Data scientist prep for Corteva Agriscience candidates: drill A/B design, power and minimum detectable effect on original exercises.

The 60-second brief

As a Data Scientist at Corteva Agriscience, you sit at the intersection of advanced mathematics, software engineering, and agricultural innovation. Your work directly impacts how the world grows food, utilizing data to optimize crop protection, improve seed performance, and drive efficiency in complex global supply chains. You are not just building models; you are solving real-world challenges in biology and chemistry that require a deep understanding of experimental design and statistical rigor.

When randomisation is off the table, the skill being checked is naming an identification strategy together with the assumption it rests on: parallel trends for difference-in-differences, relevance and exclusion for an instrument, overlap and conditional ignorability for matching. Say the assumption out loud and say how you would try to break it.

Focus areas

  • Read NRR on a fixed account cohort
  • Power experiments for heavy-tailed account revenue
  • Analyse at the account grain, cluster errors

Overview

This role is critical to the organization’s mission of transforming agriculture. You will collaborate with research specialists, agronomists, and software engineers to turn massive, often messy, datasets into actionable insights. Whether you are working on predictive modeling for crop yields or optimizing chemical research pipelines, your contributions will scale from the laboratory to the field. Expect a fast-paced environment where your ability to communicate complex technical findings to non-technical stakeholders is just as valued as your coding proficiency.

The Corteva Agriscience interview process

Corteva Agriscience candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

  • Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
  • Whether your reason for wanting the role points at the work itself rather than the company's reputation
  • Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

  • Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
  • Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
  • Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Deep Dives

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

  • Whether the query you write matches the plan you just described
  • What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
  • Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

  • Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
  • Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
  • Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Team Interviews

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

  • Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
  • The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
  • Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

  • Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
  • Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
  • Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Presentation Review

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

  • Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
  • Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
  • How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

  • Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
  • For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
  • Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Where candidates lose points

Randomising an experiment at the user level when users share an account

Two problems fire at once. Colleagues in one workspace see each other's work and talk to each other, so a treated user changes the behaviour of a control user in the same account, which violates the no-interference assumption and biases the estimate toward zero. Separately, outcomes within an account are strongly correlated, so the effective sample size is roughly n / (1 + (m - 1) * rho) for m users per account and intra-class correlation rho, not n. With rho around 0.3 and twenty users per account that is a design effect near 6.7, meaning a user-level confidence interval is about two and a half times narrower than it should be and results cross significance thresholds on noise alone. Randomise the account and cluster the standard errors.

Reporting a mean over accounts when account revenue is heavy-tailed

When a small number of accounts hold most of the revenue, the sample mean is dominated by whichever of them happens to be in the sample, and the sample variance keeps growing as more data arrives instead of stabilising. In that regime the usual central-limit-based confidence interval understates uncertainty, and a single renewal or a single large account's batch job can flip the sign of a measured effect. The fixes are to pre-register a winsorisation or capping rule before looking at the outcome, to report account counts crossing a threshold alongside the revenue figure, or to define the estimand on a bounded transform. Choosing the cap after seeing the result is a separate and worse problem, because the cap then encodes the answer.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Questions & practice

How do you differentiate between correlation and causation in complex biological datasets?

Medium · statistics and probability

Approach

  1. Quantify uncertainty explicitly rather than reporting a point estimate alone.
  2. Translate the result into the decision it informs, in one plain sentence.
  3. Write down the assumption the method needs before you use the method.

Follow-up

  • What sample size would you need to detect an effect half this size?
  • How would you explain this result to someone who does not know statistics?

How would you measure the impact of an optimization algorithm on field-testing efficiency?

Medium · machine learning and modelling

Approach

  1. Set a baseline first, so any model has something honest to beat.
  2. Pick an evaluation metric that matches the cost of each error type, not a default.
  3. Say how the offline result would be validated online before it is trusted.

Follow-up

  • Where could label leakage enter this setup?
  • How would you choose the decision threshold, and who owns that choice?

Bootstrap a confidence interval for net revenue retention

Medium · bootstrap · retention · resampling

You have one row per account with arr_start_cents (ARR twelve months ago) and arr_end_cents (ARR today, zero if churned), covering the fixed cohort of accounts that had ARR twelve months ago. Net revenue retention is sum(arr_end_cents) / sum(arr_start_cents). Write a nonparametric bootstrap from scratch, without scipy.stats.bootstrap: resample accounts with replacement, recompute the ratio of sums on each resample, and return the point estimate with a 95% percentile interval from 10,000 resamples. Also report the interval you would get from the mean of per-account ratios, and explain the difference.

Approach

  1. Resample the account, because the account is the unit the estimand is defined over. One bootstrap draw is a vector of account indices and both numerator and denominator are recomputed from that same draw; resampling the two sides independently destroys the within-account correlation that makes a ratio estimator stable.
  2. Vectorise the draws: idx = rng.integers(0, n, size=(B, n)), then end[idx].sum(axis=1) / start[idx].sum(axis=1). A 10,000 by n index matrix is usually far cheaper than a Python loop; if the matrix is too large for memory, chunk over B rather than reverting to a loop.
  3. Take the interval from np.quantile(ratios, [0.025, 0.975]). The percentile interval differs from estimate +/- 1.96 * bootstrap SE whenever the resample distribution is skewed, which it will be here, and the skew is the thing you want represented.
  4. Compute the mean-of-ratios version on the same resamples, and state the exact relationship rather than guessing which of the two is larger. With r_i = arr_end_i / arr_start_i, the ratio of sums is the arr_start-weighted mean of exactly those r_i, so sum(end)/sum(start) - mean(r) = Cov(arr_start, r) / mean(arr_start) using the population covariance. The gap is positive when larger accounts retain and expand better than smaller ones, and negative when they do not; a cohort whose small accounts churn at a higher rate has positive covariance, which puts the mean of per-account ratios BELOW the ratio of sums. Requires arr_start_i > 0 for every account, which the fixed-cohort definition guarantees; r_i is floored at 0 and unbounded above, so a handful of 4x expansions among small accounts can flip the sign. Compute the covariance and report it instead of asserting a direction.
  5. Report the interval width beside the concentration of the cohort. If the largest account is 12% of starting ARR, a narrow interval is evidence that the resampling unit is wrong rather than evidence that the estimate is precise.

Worked solution (30 min)

  1. start = df.arr_start_cents.to_numpy(float); end = df.arr_end_cents.to_numpy(float); n = len(start); point = end.sum() / start.sum()
  2. rng = np.random.default_rng(7); idx = rng.integers(0, n, size=(10_000, n)); ratios = end[idx].sum(1) / start[idx].sum(1)
  3. lo, hi = np.quantile(ratios, [0.025, 0.975]); return point, lo, hi
  4. per_acct = end / start; mean_point = per_acct.mean(); mean_boot = per_acct[idx].mean(1); compare np.quantile(mean_boot, [0.025, 0.975]) against (lo, hi), and report np.cov(start, per_acct, ddof=0)[0,1] / start.mean() as the quantity that accounts for the gap between the two centres.

Expected result: A point estimate equal to sum(end)/sum(start) on the unresampled data — the bootstrap mean is close but must not be substituted for it — and a percentile interval that is asymmetric around the point when expansion is skewed. The mean-of-ratios interval is centred on a different estimand, and which side it falls on is decided by Cov(arr_start, r): it sits below the ratio of sums in the common case where small accounts churn at a higher rate than large ones, and above it when the large accounts are the ones shrinking. Its width is governed by the spread of per-account growth, while the ratio-of-sums width is governed by how concentrated starting ARR is, so neither interval is reliably the wider one.

Verify it yourself

  • point equals sum(end)/sum(start) computed directly on the original arrays, and point - per_acct.mean() equals np.cov(start, per_acct, ddof=0)[0,1] / start.mean() to floating-point tolerance. That is an algebraic identity, not an empirical claim: a mismatch means the per-account ratios or the weights are wrong, and the sign of it tells you which of the two numbers is larger before you look.
  • np.median(ratios) sits within a small fraction of a point of the point estimate; a large gap indicates a wrong resampling unit or a transposed index matrix.
  • Raising B from 10,000 to 20,000 moves the endpoints by far less than the interval width.
  • Setting every arr_end equal to its arr_start gives point == 1.0, per_acct.mean() == 1.0, a zero covariance term and a zero-width interval.

Follow-up

  • The cohort has 800 accounts and the largest is 12% of starting ARR. How much do you trust a percentile interval here?
  • How would you extend this to an interval on the year-over-year change in NRR?
  • Two accounts merged mid-window and one contract was co-termed into the other. How do you keep the cohort fixed?

Write a query using SQL window functions to calculate a rolling average of crop yields over the last three seasons.

Medium · sql and data manipulation

Approach

  1. State the window function and its partition and ordering out loud before writing it.
  2. Check whether any join is one-to-many before aggregating, or the sums inflate.
  3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

  • What breaks if events arrive late or out of order?
  • How would you verify this result without re-running the same query?

Describe your process for joining large datasets from disparate agricultural sources.

Medium · sql and data manipulation

Approach

  1. Say which table is the grain you start from, and join outward from it.
  2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

  • What breaks if events arrive late or out of order?
  • How would you verify this result without re-running the same query?

Seven-day activation rate by signup cohort week

Easy · funnel · cohorts · censoring

dim_account carries account_id, created_at, is_internal and is_current; fct_api_request carries account_id, request_at, http_status, api_key_id and traffic_class. Build a signup cohort by the ISO week of created_at over accounts with is_internal = false. An account counts as activated when it issues a request with http_status < 400, a non-null api_key_id and traffic_class <> 'synthetic_monitor' within 168 hours of its own created_at. Return cohort accounts, activated accounts and the rate per week, and exclude any week that has not yet fully elapsed its 168-hour window.

Approach

  1. Collapse dim_account to one row per account_id before anything else. It is a type 2 dimension, so a plan or status change gives the same account several rows; filtering to is_current = true is the cheapest correct choice here because created_at does not change across versions.
  2. Express the window as interval arithmetic on the timestamptz column: request_at >= created_at AND request_at < created_at + interval '168 hours'. A date-difference of 7 days is a different and wrong condition for accounts created mid-day.
  3. Test activation with EXISTS rather than a join to MIN(request_at). EXISTS short-circuits, keeps the cohort at one row per account, and cannot fan out.
  4. Aggregate by date_trunc('week', created_at AT TIME ZONE 'UTC'), counting accounts and activated accounts, and divide as a ratio of counts.
  5. Drop unreportable weeks: the last account in a cohort week is created just under week_start + 7 days, so the week is only complete once now() >= week_start + interval '14 days'. Without that filter the newest week always looks like a regression.

Worked solution (20 min)

  1. Write the cohort CTE: SELECT account_id, created_at FROM dim_account WHERE is_current AND NOT is_internal AND created_at >= <window start>.
  2. Add the activation predicate as a correlated EXISTS over fct_api_request on account_id with the four conditions: http_status < 400, api_key_id IS NOT NULL, traffic_class <> 'synthetic_monitor', and the 168-hour bracket.
  3. Group by date_trunc('week', created_at AT TIME ZONE 'UTC'); select count() AS cohort_accounts, count() FILTER (WHERE activated) AS activated_accounts, and the ratio cast to numeric.
  4. Add HAVING or an outer WHERE that keeps only weeks where week_start + interval '14 days' <= now().
  5. Spot-check one account that activated on hour 167 and one that activated on hour 169 to confirm the boundary is exclusive at the top.

Expected result: One row per ISO cohort week with cohort_accounts, activated_accounts and activation_rate between 0 and 1; the newest row is at least 14 days old, and accounts with no rows at all in fct_api_request appear in the denominator with activated = 0 rather than disappearing.

Verify it yourself

  • Summing cohort_accounts across weeks equals the count of non-internal current-version accounts created in the same window.
  • Re-running the query a day later leaves every previously reported week byte-identical; any movement means the censoring filter is wrong.
  • activated_accounts is never greater than cohort_accounts, and the count of distinct account_id in the activated set matches a separate COUNT DISTINCT over the fact table.

Follow-up

  • The median time-to-first-successful-call is more informative than a fixed-window rate. Why can you not compute it from this query, and what estimator does it need?
  • How would you separate accounts that never called from accounts that called and got only 4xx responses, and which of those is a product problem?

If you notice a sudden drop in a key product metric, what steps would you take to diagnose the root cause?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

How do you balance long-term research goals with short-term product delivery needs?

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

How would you design a metric to measure the success of a new crop protection product?

Medium · product sense and metrics

Approach

  1. State what result would change your recommendation, so the answer is falsifiable.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • How would you detect that the metric is being gamed rather than genuinely improving?

What are the most common experimentation pitfalls you have encountered in your past projects?

Medium · a/b testing and experimentation

Approach

  1. Name the randomisation unit first; it decides the variance and what the test can detect.
  2. State the primary metric and the minimum effect worth shipping, then size the test.
  3. Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • How would you handle interference between treated and control units?

How do you determine the required sample size for an experiment in a field setting?

Medium · a/b testing and experimentation

Approach

  1. Name the randomisation unit first; it decides the variance and what the test can detect.
  2. Name the guardrails that would stop a launch even on a positive primary result.
  3. Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

  • How would you handle interference between treated and control units?
  • What would you conclude if the result is positive but the test is underpowered?

Size an account-randomised test of a consumption feature

Easy · power · mde · heavy-tailed-outcomes

You have 2,800 non-internal paying accounts. The outcome is the monthly sum of billable_quantity for sku_code = 'compute_hours' from fct_usage_daily, with a mean of 640 and a standard deviation of 2,100 across accounts. Product wants to detect a 5% lift from a scheduling feature, randomised 50/50 at the account level, at 80% power and two-sided alpha 0.05. Give the minimum detectable effect the available sample actually supports, say whether the 5% target is reachable, and state what you would change in the design or the metric before agreeing to run it.

Approach

  1. Write the two-sample formula and plug it before discussing anything else: MDE = (z_0.975 + z_0.80) * sd * sqrt(1/n1 + 1/n2) = 2.80 * 2100 * sqrt(2/1400) = 222 compute-hours, which is 34.7% of the 640 mean. The driver is the coefficient of variation of 3.3, not the account count, so quoting a sample size without it is not a power calculation.
  2. Invert for the stated target. A 5% lift is 32 compute-hours, needing N = 4 * (2.80 * 2100 / 32)^2 = about 135,000 accounts, roughly 48 times the eligible population. Say the test is not powerable on this metric, and do not offer to run it longer: the account population is fixed, and extra weeks add repeated observations on the same clusters rather than new clusters.
  3. Attack the variance instead of the sample. Pre-register winsorisation at the 99th percentile of the pre-period distribution, or move the estimand to a bounded one such as the share of accounts exceeding a usage floor. State explicitly that this changes the estimand rather than cleaning the data, because a cap chosen after seeing the result encodes the answer.
  4. Add CUPED on the account's pre-period compute_hours. At a pre/post correlation of 0.80 the residual variance multiplier is 1 - 0.80^2 = 0.36, the standard error falls to 0.60 of its unadjusted value, and the MDE drops from 222 to 133 compute-hours (20.8% relative). That is still four times the requested 5%.
  5. Fix the randomisation unit and the balance in the same breath: randomise account_id because colleagues share workspaces, and stratify assignment on pre-period usage decile so that a handful of very large accounts cannot land disproportionately in one arm and dominate the difference in means.
  6. Close with an operating decision rather than a number. Run only if the team will act on a roughly 20% MDE after CUPED and winsorisation; otherwise re-specify the primary outcome as a bounded indicator and treat compute-hours as a secondary reported with intervals only.

Worked solution (25 min)

  1. Compute the coefficient of variation, 2100 / 640 = 3.28, and check the top-decile share of total compute_hours before treating the mean as a summary of anything.
  2. Compute MDE = 2.80 * 2100 * sqrt(1/1400 + 1/1400) = 2.80 * 2100 * 0.0378 = 222 compute-hours, 34.7% relative.
  3. Invert to the required N for a 32-hour effect: 4 * (5883 / 32)^2 = about 135,000 accounts, 48x the population.
  4. Recompute under CUPED at rho = 0.80: variance multiplier 0.36, SE multiplier 0.60, MDE 133 compute-hours, 20.8% relative.
  5. Recompute on the binary variant at base rate 0.46 and write the recommendation as one decision with one operating point.

Expected result: MDE is about 222 compute-hours (34.7% relative) as specified and about 133 (20.8%) with CUPED at rho = 0.80. A 5% lift would require roughly 135,000 accounts, about 48 times the eligible population, so the test is declined on this metric and re-specified on a bounded outcome where the MDE falls to about 5.3 percentage points on a 0.46 base.

Verify it yourself

  • sqrt(1/1400 + 1/1400) = 0.0378, so the MDE must land within a few percent of 2.80 * 2100 * 0.0378 = 222. A materially different number means a one-sample or one-sided formula was used.
  • Required N scales with the inverse square of the effect size: halving the target MDE must quadruple N. If it does not, the formula is wrong.
  • The CUPED variance multiplier is 1 - rho^2, never 1 - rho. At rho = 0.80 the standard error falls by 40%, not by 20%.

Follow-up

  • The feature only applies to the 310 accounts on committed_consumption contracts. What is the MDE on that subset, and does it change the recommendation?
  • Suppose the primary outcome becomes a binary indicator, account exceeded 100 compute-hours in the month, at a base rate of 0.46. What is the MDE now, and why did it improve so much?
  • What would make you stop this test early, and how do you write that rule so it is not peeking?

Billable units per account jumped while nothing shipped

Medium · retries · idempotency · guardrail metrics

Billable units per paying account rose 22 percent month over month with no pricing or packaging change. From fct_api_request (account_id, endpoint, http_status, is_retry, idempotency_key, traffic_class, billable_units, request_at) and fct_usage_daily (account_id, sku_code, usage_date, billable_quantity), determine how much of the rise is delivered value and how much is duplicated work. Deliverable: the decomposed figure, the accounts it concentrates in, and a recommendation on whether to report the 22 percent at all.

Approach

  1. Check the guardrail before the headline: compute the customer-visible server error rate per account for both months, numerator http_status >= 500 and denominator excluding synthetic_monitor and load_test. Metered volume rising alongside an error rate is the known failure mode in this domain.
  2. Deduplicate logical work by counting billable_units once per (account_id, idempotency_key) at the first successful request rather than once per row. Rows with a null idempotency_key cannot be deduplicated, so report their share as an explicit uncertainty band instead of assuming they are all unique.
  3. Split by traffic_class before interpreting anything, because one change to a continuous-integration configuration can multiply request volume overnight without a human deciding anything about the product.
  4. Test concentration: compute the per-account distribution of the increase and its top-decile share. A rise carried by a few accounts scaling one batch job is a different finding from a broad shift and gets a different recommendation.
  5. Reconcile against fct_usage_daily for the same accounts and dates, and be ready to explain the expected gap in two sentences: request rows include retries and failures carrying zero billable_units, and the usage table restates after first write.

Follow-up

  • A client retrying a request the server already completed produces duplicate billed work. What protocol or product change removes that, and what would you measure to confirm it worked?
  • If the duplicated volume was genuinely invoiced, what should finance do, and how does that change what belongs in the metric?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

  • For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
  • For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
  • Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

  • Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
  • List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
  • Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

  • Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
  • Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
  • Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

  • Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
  • Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
  • Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

  • Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
  • Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
  • Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

  • Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
  • For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
  • Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

  • Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
  • Listen back and mark every moment you proposed a solution before the success metric existed.
  • Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Describe a time you had to explain a complex technical model to a non-technical stakeholder.

Medium · behavioural and stakeholder questions

Approach

  1. Name the disagreement or constraint, and how you resolved it with evidence.
  2. Quantify the outcome, including what you would not claim credit for.
  3. State the situation in two sentences and spend the rest on your reasoning.

Follow-up

  • What did you decide not to do, and why?
  • What would you do differently if you ran that project again?

Announce a metric fix that cuts the headline number

Medium · metric definitions · stakeholders · communication

Weekly active organisations, the count on the company dashboard, has never excluded rows where dim_account.is_internal is true, and it counts traffic with traffic_class in synthetic_monitor and load_test. Correcting both reduces that count by 11 percent and removes most of the growth reported over two quarters. The figure appears in a board deck and in two teams' quarterly goals, one written on the count and one on the weekly active organisation ratio, whose denominator is accounts whose account_status was in ('trial','free','active_paid') through the week. Decide the order in which you tell people, what the dashboard shows during the transition, and what you propose happens to goals already set against the old definition.

Approach

  1. The interviewer is probing whether you can land a correction as an operational change with a plan attached, rather than as an announcement other people then have to clean up after.
  2. Quantify each exclusion separately before telling anyone: internal accounts, synthetic monitors, load tests. Three known quantities are a discussion; one alarming total is an argument.
  3. Be precise about which side of the metric each exclusion touches, because one team's goal is on a count and the other's is on a ratio. The traffic-class filters remove requests, so they shrink the numerator only. Dropping internal accounts removes them from the ratio's denominator as well, since internal accounts carry ordinary account_status values and therefore sit in that denominator. Internal accounts are active in almost every week while the real base is not, so the numerator loses a larger share than the denominator and the ratio falls by less than the count does. Compute both and say which one the 11 percent is before anybody assumes.
  4. Check whether the trend changes, not only the level. A constant 11 percent shift is a rebasing and nothing more. A shift that widens over time means the reported growth was partly internal or synthetic, which makes the existing goals unachievable as written and changes what you are asking teams to do.
  5. Sequence the disclosure: the metric owner and the two teams whose goals move first and privately, then the board channel with a written bridge, then the dashboard. The dashboard is last because a number that changes without explanation is read as instability rather than as a fix.
  6. Run both series for one reporting period with the bridge visible, restate history rather than letting the series break at a date, and set the date the old series is removed.
  7. Propose the goal treatment yourself: rebase each target by the shift measured on the metric that target is written against, rather than leaving each team to negotiate individually, which is where corrections of this kind usually die.

Follow-up

  • One team's quarterly goal is now unreachable. Rebase the target or let it miss, and what does each choice teach the organisation?
  • How would this have been caught when the metric was first defined?
  • What else on that dashboard shares this failure mode, and how would you find out this week?

Allocate one analyst week across three competing requests

Easy · prioritisation · stakeholders · scoping

Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.

Approach

  1. The interviewer is probing whether you prioritise on decision timing and reversibility or on who asked most forcefully. Sort by the date each decision is actually taken and by what the default outcome is if nothing arrives.
  2. Apply that sort concretely. The Thursday readout has a hard irreversible deadline and no value afterwards. The pricing review has three weeks of slack. The renewal list has a rolling deadline set by term_end_date, so part of it is urgent this week and the rest is not, which means it can be split rather than deferred whole.
  3. Find the cheapest sufficient version of each request rather than the full version. The readout goes in full. The renewal list ships as a filtered query over renewal-eligible accounts ranked by two inspectable signals rather than as a model. The margin work is scoped to the accounts that dominate the pricing decision, since revenue is heavily skewed and the tail will not change the conclusion.
  4. Make the trade visible in one written note to all three at once, with dates. Telling each person separately that they are the priority is how an allocation becomes a credibility problem.
  5. Refuse something explicitly and say why. The model version of the renewal list is the usual candidate, because it cannot be evaluated without a holdout nobody has agreed to yet, and building it this week forecloses that.
  6. Leave slack. A plan with none is a plan to miss the one deadline that cannot move.

Follow-up

  • The sales leader escalates to your manager. What did you already do that makes that a short conversation?
  • Which of the three deadlines would you push back on, and what exactly would you ask for?
  • What would you change about how these requests reach you so next week is not the same?