Interview guides · Greenlight Biosciences

Greenlight Biosciences Data Scientist Interview Questions & Guide 2026

Data scientist prep for Greenlight Biosciences candidates: drill A/B design, power and minimum detectable effect on original exercises.

The 60-second brief

The Data Scientist role at Greenlight Biosciences is pivotal to the company's mission of harnessing RNA technology for sustainable agriculture. You will be at the forefront of analyzing complex biological data to drive decisions that impact product development, user experience, and overall business strategy. This position is vital for translating data into actionable insights that can enhance the efficacy of the company's products and ultimately contribute to greener agricultural practices.

Learn the economics of the product category before the loop. Marketplaces, subscription products and ad-supported products turn on different core quantities (match rate and liquidity, retention and churn, fill rate and yield) and fail in different characteristic ways.

Focus areas

Separate censoring from events in time-to-event models
Build member-month denominators from overlapping coverage spans
Correct for claims runout before reporting recent months

Overview

In this role, you will collaborate closely with cross-functional teams, including product development and engineering, to design experiments, analyze results, and optimize processes. The complexity of biological data and the scale of projects at Greenlight Biosciences ensure that your work will have a significant impact, making this a challenging yet rewarding opportunity for candidates passionate about data-driven solutions and sustainability.

The Greenlight Biosciences interview process

Greenlight Biosciences candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screening Call

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Hiring Manager Interview

Expect a live problem with pieces of it missing, closer to a conversation than an exam. A metric moved, or somebody wants to know whether a change worked, and you are asked how you would find out. The manager is watching the first ninety seconds, specifically whether you establish what decision hangs on the answer before you start proposing methods. Candidates who open with a technique get steered back. Once the decision is clear, describe what the data would look like if the story were true, and say what you would accept as evidence that it is not.

What to demonstrate

Whether you fix the decision the analysis serves before choosing an approach
How you continue when you are told the data you just asked for does not exist
Whether you state what would change your mind, not only what would confirm the hypothesis you started with
How you size an effect before you have measured it

How to prepare

Take a metric you know well and practise explaining in under two minutes the four things that could have moved it and how you would separate them
Pick a recent launch or experiment and write the single number you would ask for first, plus what you would conclude if it came back flat
Practise being interrupted: have someone remove a data source halfway through your answer and carry on without restarting

Panel Interview

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Treating clinical measurements as missing at random

A lab result, a vital sign, or a screening exists because someone ordered it, and ordering tracks suspicion of disease, visit frequency, and site workflow. Imputing the mean or dropping incomplete rows biases the population estimate and can flip the sign of an association, because the untested are systematically healthier or systematically disengaged. The presence indicator is often more predictive than the value, which is a warning sign rather than a feature win: a model that learns test ordering will not transfer to a site with different protocols.

Reading the most recent months of a claims-based series as real

Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.

Reporting a p-value with no effect size or interval

Give the estimated difference with a confidence interval in the units the business cares about, then say whether that whole interval is worth acting on. A p-value only addresses whether you can rule out exactly zero; it says nothing about magnitude.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Questions & practice

Write a function that calculates the correlation coefficient.

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Translate the result into the decision it informs, in one plain sentence.
Say what the estimate is of, and over what population it generalises.

Follow-up

What sample size would you need to detect an effect half this size?
How would you explain this result to someone who does not know statistics?

How do you optimize a machine learning model for better performance?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

Explain the concept of overfitting and how you would address it in your models.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Set a baseline first, so any model has something honest to beat.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Proportion of days covered with shifted, truncated refill intervals

Hard · interval merging · adherence · path dependence

pharmacy_claim has fill_id, member_id, therapeutic_class_code, fill_date, days_supply, reversal_flag, reversed_fill_id. For one therapeutic class and a fixed 12-month window, compute proportion of days covered per member: distinct days on which a dispensed days_supply covers the day, divided by days from the member's first in-window fill through the window end. An early refill shifts coverage forward rather than stacking, and coverage is truncated at the window end. Drop both rows of every reversed pair. Return member_id, pdc, and the share at pdc >= 0.80 among members with at least 2 fills and at least 91 days of follow-up.

Approach

Remove reversals as pairs first. Drop every row with reversal_flag true, and also drop the fill_ids those rows point at through reversed_fill_id. Dropping only the flagged row leaves a dispense that was never collected in the exposure.
Walk fills per member in fill_date order carrying a cursor: start = max(fill_date, previous_end + 1 day), end = start + days_supply - 1. The shift is path dependent, so a plain cumsum over days_supply does not reproduce it. Use itertools.accumulate or a per-member loop over numpy arrays, not a row-wise apply over the whole frame.
Truncate the last interval at the window end before measuring. Without truncation a 90-day fill dispensed on the final day pushes the covered-day count past the denominator and PDC above 1.0.
Use the denominator the definition states: first in-window fill_date through window end, inclusive. Not a flat 365, and not first fill to last fill, which is a different metric that rewards early discontinuation.
Apply the eligibility filter before computing the >= 0.80 share, and report the size of that denominator next to the share. A share without its denominator is not reviewable.

Worked solution (40 min)

Filter to the therapeutic class and the window, then remove reversed pairs by dropping flagged rows and the fill_ids in reversed_fill_id.
Sort by member_id, fill_date. Per member, accumulate start = max(fill_date, prior_end + 1 day) and end = start + days_supply - 1.
Clip every interval's end at the window end and drop intervals whose start is past it.
Covered days per member = sum of (end - start + 1) over the shifted intervals, which are disjoint by construction.
Denominator = (window_end - first_fill_date).days + 1. Divide, then filter to members with >= 2 fills and denominator >= 91 and compute the share at or above 0.80.

Expected result: Per-member PDC in [0, 1], plus one share. Worked case: fills at day 0 (30 days), day 20 (30 days) and day 55 (30 days) in a 180-day denominator give intervals 0-29, 30-59 and 60-89, so 90 covered days and PDC = 0.50. Change the window to 60 days with fills at days 0, 20 and 40 and the shifted intervals truncate to 0-59, giving PDC = 1.00 while a days-supply sum would report 90/60 = 1.50.

Verify it yourself

No member has PDC above 1.0. Any value above 1.0 means intervals stacked instead of shifting, or the final interval was not truncated.
A member with a single 30-day fill at the start of a 180-day denominator scores 30/180 = 0.167 and is excluded from the share by the two-fill rule.
For one member, rebuild covered days as the size of a Python set of date ordinals over the raw fills and confirm it matches the interval arithmetic.
Shifting every fill_date forward by the same number of days changes each denominator but leaves the covered-day counts unchanged except at the truncation boundary.

Follow-up

How does PDC differ from medication possession ratio, and which of the two can exceed 1.0?
A member switches to a different ingredient inside the same therapeutic class mid-window. Should the intervals chain, and what does that do to the class-level number?
Members who die or lose coverage mid-window get a short denominator and often a high PDC. If you then compare mortality by adherence category, what bias have you built in and how do you remove it?

Measure ninety-day primary care contact for an enrollment cohort

Easy · cohort analysis · conversion rate · date windows

member_enrollment gives span_id, member_id, product_type, effective_date and termination_date, which is NULL while a span is active. medical_claim_line gives member_id, service_start_date, procedure_code and claim_status. The cohort is members whose earliest 2025 span has effective_date in January 2025 and who stay covered at least 90 days from it. Return, by product_type, cohort size and the share with at least one paid claim line whose procedure_code is in a supplied list of evaluation-and-management office visit codes and whose service_start_date falls in the half-open window from effective_date to effective_date plus 90 days.

Approach

Build the cohort as one row per member before touching claims. Take the earliest 2025 span per member, require its effective_date in January, and require coverage to effective_date + 90 days, which means termination_date IS NULL OR termination_date >= effective_date + 89.
Use EXISTS for the numerator rather than a join. A member with four office visits must count once; joining and counting rows inflates the numerator and can push the rate above 1.
Keep the window half-open and anchored per member. Every member has a different effective_date, so a single fixed calendar window is a different metric and will not match the cohort definition.
Aggregate numerator and denominator in one pass with a conditional sum grouped by product_type, and carry the raw counts alongside the rate so the reader can see the denominator.
State the reportability rule on the output: the January cohort is not final until 90 days plus professional-claims runout have passed, so label the paid-through date.

Worked solution (15 min)

CTE first_span: per member, the 2025 span with the minimum effective_date, filtered to January effective dates.
Add the continuous-coverage requirement to that CTE using COALESCE(termination_date, DATE '9999-12-31').
Add an EXISTS subquery on medical_claim_line correlated on member_id, restricted to the code list, claim_status = 'paid', and service_start_date >= effective_date AND < effective_date + 90.
Group by product_type: COUNT(*) as cohort_size, SUM(CASE WHEN contacted THEN 1 ELSE 0 END) as contacted, and their quotient.
Print the paid-through date used, and re-run at a 30-day window to confirm the rate falls.

Expected result: One row per product_type with cohort_size, contacted_members and contact_rate = contacted_members / cohort_size, where contacted_members never exceeds cohort_size.

Verify it yourself

contact_rate lies in [0, 1] for every product_type. Above 1 proves the numerator was counted on rows rather than members.
SUM(cohort_size) across product_types equals COUNT(DISTINCT member_id) in the cohort CTE. A larger value means a member with two January spans was counted twice.
A 0-day window returns a rate at or near zero, and a 365-day window returns a rate no lower than the 90-day one.

Follow-up

A member changes plan on day 40, closing one span and opening another. Is she still in the cohort, and which product_type does she report under?
What moves if denied lines are allowed into the numerator, and which definition is right for an access metric?
How would you show this as a cumulative curve by day since effective_date instead of one 90-day point?

Compute allowed PMPM by incurred month without span fan-out

Hard · fan-out · many-to-many · runout

Report allowed PMPM by incurred month and product_type for 2025 from medical_claim_line (member_id, claim_id, claim_version, frequency_code, service_start_date, allowed_amount, claim_status), pharmacy_claim (member_id, fill_date, allowed_amount, reversal_flag, reversed_fill_id), member_enrollment (span_id, member_id, product_type, effective_date, termination_date) and member_risk_score (member_id, score_year, prospective_risk_score), which holds at most one row per member per score year. The numerator is allowed_amount on surviving claim versions plus unreversed fills, attributed to the product_type in force on the service or fill date. The denominator is member-months. Also return a risk-adjusted PMPM for each cell: allowed PMPM divided by the member-month-weighted mean prospective_risk_score for score_year 2025 over the members making up that cell's denominator. A colleague joined claim lines to member_enrollment on member_id alone and the annual total came back 1.4 times the finance figure. Diagnose that, then produce the correct monthly series.

Approach

Name the defect precisely. member_enrollment is one row per span, so joining on member_id alone is one-to-many and duplicates every claim line once per span that member ever held. The inflation factor is the spend-weighted mean span count per member, which is why 1.4 is plausible and why it is not a constant you can divide back out.
Make the join functional: member_id AND service date between effective_date and COALESCE(termination_date, DATE '9999-12-31'). That yields at most one span per line unless spans genuinely overlap, in which case pick a documented precedence rule, most recently effective wins being the usual one, and prove uniqueness with a duplicate check rather than asserting it.
Aggregate numerator and denominator separately and join them on (month, product_type). Computing both sides across one join is the second route to inflation, because the member-month denominator fans out over claim lines just as easily as the dollars fan out over spans.
Clean each numerator source on its own terms: surviving claim versions with voids removed and claim_status 'paid' on the medical side, reversal rows and their originals removed on the pharmacy side.
Apply completion before anyone reads a trend. State the paid-through date and either withhold the three most recent incurred months or apply completion factors, since the lag differs by service type and the incomplete tail reads as a cost improvement and a utilisation drop at once.
Risk-adjust last, and weight the score the way the denominator is weighted. The divisor for a cell is SUM(member_months * prospective_risk_score) / SUM(member_months) over that cell's members, not a plain AVG over distinct members: a member covered two months must not count the same as one covered twelve. member_risk_score is member-level, so join it after the member-month grain is fixed, or it becomes a second fan-out.
State the rule for members with no 2025 member_risk_score row: keep them in member_months, leave them out of the weighted mean, and publish the scored share of member-months beside the adjusted series. A thin score table moves the adjusted number while the raw one sits still, and without that share nobody can tell the two apart.

Follow-up

A member holds two overlapping spans under different product_types on the service date. Which one gets the dollar, and how do you keep the member-month denominator consistent with that choice?
Write the check that proves the span join is one-to-one rather than assuming it.
Finance reports on paid_date and you reported on service_start_date. Reconcile one month between the two bases.

If you were tasked with reducing the time it takes to analyze data, what steps would you take?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize tasks when managing multiple projects?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

Given a dataset, how would you determine the most important features?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you design an experiment to test a new agricultural product?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

Describe a data science project you worked on from start to finish.

Medium

Approach

Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Redesign an adherence metric an outreach team cannot quietly inflate

Medium · gaming · adherence · denominator design · threshold bunching

An outreach team is paid on the share of members reaching proportion of days covered of 0.80 or higher in a therapeutic class, computed from pharmacy_claim (member_id, therapeutic_class_code, fill_date, days_supply, refill_number, reversal_flag, reversed_fill_id). The denominator is members with at least two fills in the class and at least 91 days of follow-up, over a 12-month window. Name three ways the team raises this number without any member taking more medication, then redesign the metric and its guardrails so each lever is closed or visible.

Approach

Attack the denominator first, because it is the cheapest lever. A member who fills once and abandons never enters the denominator at all, so the worst outcome in the class is invisible, and any effort spent on first-fill abandonment lowers the reported share by moving those members in. The incentive runs backwards.
Attack the window arithmetic second. With the denominator running from first fill to window end, a member whose first fill lands late in the year has a short denominator that is trivially easy to cover, so steering new starts to the last quarter raises the metric mechanically. Report by index quarter or fix the window and require the index fill in the first quarter.
Attack the threshold third. Days of supply dispensed late in the window count only up to the window end, but a fill placed 60 days before close, on a member whose last on-hand day has already passed, adds up to 60 covered days, about 0.16 of a 365-day denominator. That alone moves a member from 0.77 to 0.93 with no change in what anyone swallows.
Close the dispensing-side hole: a fill that is billed and reversed is not exposure. Exclude both rows of a reversed pair via reversal_flag and reversed_fill_id, and collapse overlapping fills by pushing each interval start to the later of fill_date and the prior interval's end, so early refills shift coverage forward rather than stacking.
Redesign the scorecard as a pair rather than a single number: the threshold share stays as the headline, and mean PDC plus the share at 0.90 or higher travel with it, because a distribution cannot be bunched at a threshold without showing it.
Add the guardrails that make each lever visible: first-fill abandonment rate (fills with refill_number 0 never followed by a second fill, plus reversed first fills), the share of covered days originating from fills in the final 60 days of the window, and the distribution of PDC in the bands just below and just above 0.80.

Worked solution (30 min)

Write the exposure-interval construction explicitly: drop reversed pairs, sort fills per member and class by fill_date, set each interval start to the greater of fill_date and the prior interval end, and truncate the final interval at the window end.
Write the denominator in two variants, the incumbent at-least-two-fills rule and an index-fill rule requiring the first fill in the first quarter with at least 273 days of follow-up, and state what each includes that the other does not.
Enumerate the three gaming levers with the arithmetic that makes each one work, including the 60-day end-of-window fill computed against a 365-day denominator.
Specify the headline metric plus the two companions (mean PDC, share at 0.90 or higher) and the three guardrails, each with numerator, denominator and window.
Write one sentence stating that dispensing is a proxy for taking and that the proxy overstates adherence in the direction of anyone who fills and does not take.

Expected result: A redesign in which: reversed fills are excluded and overlaps shift forward rather than stack; the denominator is anchored on an index fill in a fixed window so late starters cannot buy a short denominator; the threshold share is reported with mean PDC and the share at 0.90 or higher; and three guardrails ship with it, namely first-fill abandonment rate, the share of covered days contributed by fills in the last 60 days of the window, and the PDC distribution in the bands immediately below and above 0.80.

Verify it yourself

Hand-compute PDC for a member with fills on day 1, day 25 and day 200 at 30 days supply each in a 365-day window. Overlapping the day-25 fill forward gives 90 covered days and PDC of about 0.25; stacking incorrectly gives 90 days too but misplaces them, so check the interval endpoints, not just the total.
Compare the member count under both denominator rules. The index-fill rule should be strictly smaller or equal on early starters and should pull in first-fill abandoners the incumbent rule excluded.
Plot the PDC histogram around 0.80. A smooth distribution is consistent with real behaviour; a spike in the first band above the threshold is not.
Sum covered days sourced from fills in the final 60 days of the window across members who cross the threshold. A large share concentrated there is the end-of-window blitz showing up directly.

Follow-up

The share at 0.80 or higher rose four points and mean PDC rose 0.4 points. What is your reading, and what do you look at next?
Would you pay on 0.80 at all, or on the continuous measure? Say what each choice costs you.
Possession is not ingestion. Name a measurable consequence in another table that would corroborate a real adherence improvement, and state its own weakness.

First-pass denial rate doubled in one received week

Medium · pipeline failure · segment cuts · denominator definition

First-pass denial rate jumped from 6.1 to 11.8 percent in a single received_date week and stayed at the new level. Nothing in the adjudication rules changed that week. You have medical_claim_line: claim_id, claim_version, frequency_code, claim_status, denial_reason_code, member_id, received_date, adjudicated_at, billing_provider_npi. The metric counts lines whose first adjudication returns 'denied', over lines received in the window at claim_version = 1. Find the cause, say whether denial behaviour actually changed, and deliver the corrected series plus the query change that fixes it.

Approach

Cut the excess by denial_reason_code before anything else. A behaviour change spreads across codes; an upstream failure concentrates in one. If a single eligibility-related code carries nearly all of the 5.7 point excess, the claims are fine and the eligibility data was not.
Join the denied lines to member_enrollment and compare the span's created_at to the claim's received_date. Lines denied for members whose coverage span loaded after the claim arrived are a load-ordering failure, not a coding failure.
Check resolution downstream: take those claim_ids and look for a later version with frequency_code 7 that adjudicates to 'paid'. A high resubmit-and-pay share confirms the denial was transient and recoverable.
Verify the denominator is not the mover. Count lines at claim_version = 1 by received_date and check the weekday mix, because submission batching makes received_date strongly day-of-week patterned and a short or holiday week shifts the denominator without changing behaviour.
Confirm runout: the metric should be held until 60 days of adjudication have passed, so check that the comparison weeks are equally mature rather than comparing a settled week to a fresh one.
Restate the series with the affected reason code broken out as its own line, and change the query to carry denial_reason_code into the reported breakdown rather than only the headline rate.

Follow-up

The affected lines all get paid on resubmission. Is the first-pass denial rate then wrong, or right and uninteresting?
How would you monitor for this class of failure without waiting for someone to notice a chart?
What changes if the excess is spread evenly across denial_reason_code instead of concentrated?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Rigour against a deadline

Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.

Tell us about a time you had to convince stakeholders to adopt your recommendations.

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Sequence three urgent requests with one analyst-week available

Medium · prioritisation · tradeoffs · stakeholder communication

Three requests land on Monday and you have one week. An actuarial team needs incurred-claims completion factors restated before a filing deadline on Thursday. A clinical programme owner wants a deterioration model refreshed because its calibration has drifted in one region. A trial operations team wants site enrolment forecasts for a portfolio review in two weeks. Each requester believes theirs is blocking. Deliverable: your sequence with the reasoning, the message you send to whoever is deprioritised, and the smaller artefact you hand each of the two you cannot fully serve.

Approach

The probe is whether you prioritise on consequence and reversibility rather than on who asked loudest or most recently.
Classify each request by what happens if it slips. A regulatory or contractual deadline is irreversible on its date, a drifting model is causing harm every day it keeps running, and a portfolio review can absorb a provisional number. That ordering is defensible to all three requesters because it does not depend on your preferences.
Take the deadline-bound work first, but scope it to the minimum defensible output, because completion factors feeding a filing carry a different error tolerance than a slide.
Do not let the drifting model simply wait. Quantify the harm cheaply by comparing calibration in the affected region against the rest, and if it is materially miscalibrated propose flagging or suppressing its output for that region within the hour rather than at the end of a refresh.
Give each deprioritised requester something real: a provisional forecast with its uncertainty and a refresh date, or a diagnostic that tells them whether their problem is urgent. Say no explicitly with a date rather than going quiet, because silence is what produces escalation.

Follow-up

The programme owner escalates to your manager. What do you want your manager to be able to say?
Midweek the actuarial work needs two more full days than you estimated. What gives?
How does your answer change if the drifting model drives a clinical outreach list rather than a report?

Explain an incomplete cost series to a non-technical executive

Medium · claims runout · uncertainty · executive communication

An executive opens a dashboard showing risk-adjusted allowed PMPM down 6 percent across the last three incurred months and asks whether the cost initiative is working. You know pharmacy adjudicates within days, professional claims within weeks, and inpatient facility claims over months, so those three months are both undercounted and mix-shifted toward cheap services. There is also a genuine movement of roughly 1.5 percent in the older, complete months. Deliverable: a two-minute spoken answer, the change you make to the chart, and what you commit to telling them and when.

Approach

The probe is whether you can be honest about uncertainty without sounding evasive. Executives read hedging as not knowing, so the answer has to end in a commitment.
Separate the two claims immediately. The 6 percent is an artefact of incomplete data, and there is a smaller real movement in the months that are complete. Delivering only the debunk leaves them holding nothing.
Explain the lag in their terms: the bills for the most expensive care arrive last, so an incomplete month always looks cheap and always looks like it is improving. Keep the phrase completion factor out of the first pass.
Fix the chart instead of explaining around it. Shade or omit the incomplete months, print the paid-through date on the axis, and overlay the same series as it looked at equivalent maturity a year earlier so the shape is comparable rather than merely lower.
Close with a date and a magnitude. Commit to a restated figure once runout matures, say roughly how far you expect the 6 percent to shrink, and name what you would need to see to call the initiative working.

Follow-up

They ask for your best guess today, knowing it is provisional. What do you say?
The restated number comes back at 1 percent. How do you handle having flagged the 6 percent at all?
How do you stop this dashboard producing the same conversation next quarter?