Interview guides · Exact Sciences

Exact Sciences Data Scientist Interview Questions & Guide 2026

Data scientist interview prep for Exact Sciences: original practice on experiment design, SQL and metric reasoning, with worked solutions.

The 60-second brief

As a Data Scientist at Exact Sciences, you will operate at the critical intersection of advanced analytics and life-saving diagnostics. You are not just building models; you are translating complex biological and clinical data into actionable insights that directly impact patient outcomes. Your work supports the development and refinement of non-invasive screening and diagnostic tests, requiring a sophisticated blend of statistical rigor, machine learning expertise, and an appreciation for the complexities of clinical research.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

Build member-month denominators from overlapping coverage spans
Minimise protected health information in every extract
Separate censoring from events in time-to-event models

Overview

This role is pivotal in driving the innovation pipeline for Exact Sciences. You will collaborate with cross-functional teams—including clinicians, software engineers, and product managers—to address high-stakes problems, such as optimizing test sensitivity, analyzing population health data, and automating diagnostic workflows. You will navigate high-dimensional data sets in a fast-paced environment, where your ability to communicate technical findings to non-technical stakeholders will be as important as the code you write.

Be prepared to discuss how your data science work translates into tangible business or clinical value. At Exact Sciences, the 'why' behind the model is as significant as the architecture itself.

Where to focus your preparation

Treat the sections below as preparation areas and confirm the format with your recruiter.

Preparation focus

No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.

What to demonstrate

Breadth across SQL, experimentation and product reasoning
Ability to state assumptions before choosing a method

How to prepare

Drill the practice exercises below and time yourself
Prepare three quantified stories about decisions you drove

Where candidates lose points

Reading the most recent months of a claims-based series as real

Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.

Pre-post evaluation on a high-cost or high-risk cohort

Cohorts selected on an extreme value of the outcome regress toward the mean on their own. Members identified as the top 1 percent of spend in one year spend far less the next, whether or not anyone intervenes, because the selecting year captured both chronic severity and one-off events. A pre-post design on such a cohort will report savings every time. A concurrent comparison group selected by the same rule in the same period, or a regression discontinuity at the selection threshold, is the minimum credible design.

Ending an analysis without a recommendation or next step

Close with what you would do and what would change your mind, stated as a condition you can check later. If the evidence is genuinely inconclusive, recommend the specific next measurement and say what it costs in time or exposure.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Questions & practice

Simulate how ignoring clustering inflates the false positive rate

Medium · simulation · design effect · type i error

Simulate a provider-level comparison with no true effect. 40 providers, 60 members each, a continuous outcome with total variance 1 and intraclass correlation 0.02. Randomise providers 20 to 20, not members. Over 5,000 replications, report the share of replications where a two-sample t-test run on all 2,400 individual observations, ignoring provider, returns p below 0.05. Report the same share when the unit of analysis is the 40 provider means. Then state how the first number follows from the design effect 1 + (m - 1) x ICC.

Approach

Decompose the variance explicitly rather than tuning it: provider variance = ICC = 0.02, residual variance = 1 - ICC = 0.98. Draw u_j once per provider and e_ij per member, so the outcome is u_j + e_ij and the correlation between two members of one provider is 0.02 by construction.
Randomise at the provider level. That is the entire mechanism: the treatment indicator is constant within a provider, so in any single replicate the provider intercepts are confounded with the arm, and the naive test reads that confounding as signal.
Vectorise over replications with a (reps, providers, members) array. 5,000 x 2,400 normals is trivial in numpy, and a Python loop is what tempts people to cut replications to 500 and report a number with Monte Carlo noise of plus or minus 0.017.
Predict the answer before running it: design effect = 1 + (60 - 1) x 0.02 = 2.18, naive standard errors are too small by sqrt(2.18) = 1.476, so the rejection rate at nominal 0.05 is about 2 x (1 - Phi(1.96 / 1.476)) = 0.18. Agreement between prediction and simulation is what makes the result a demonstration rather than an anecdote.
Run the provider-mean analysis as the control. Recovering 0.05 there proves the generator is correct and isolates the failure to the analysis unit.

Follow-up

How many providers would you need for 80 percent power on a 0.2 SD difference with 60 members each and this ICC?
Cluster sizes are unequal in reality. The design effect approximation becomes 1 + ((1 + CV^2) x mbar - 1) x ICC. Which direction does that move your sample size and why?
The intervention cannot be withheld from any provider. Sketch a design that still yields a defensible estimate.

Four data-quality rules over corrected and cancelled lab results

Easy · data quality · record supersession · unit drift

lab_result has result_id, order_id, patient_id, loinc_code, value_numeric, value_text, units, reference_low, reference_high, specimen_collected_ts, resulted_ts, result_status (preliminary, final, corrected, cancelled), supersedes_result_id. Return one row per rule with a failing count, a failing share, and one example result_id, for these rules: more than one surviving row per (order_id, loinc_code) after resolving corrections and dropping cancelled; resulted_ts earlier than specimen_collected_ts; a numeric result whose unit is not the modal unit for its loinc_code; value_numeric and value_text both null on a row with result_status 'final'.

Approach

Resolve the correction chain before anything else. Drop result_status 'cancelled', then within each (order_id, loinc_code) keep the row whose result_id appears in no other row's supersedes_result_id. That terminal node is the surviving value and the rule holds even if a correction is loaded with a backdated timestamp.
Build each rule as a boolean mask over a named frame rather than as a filtered copy, so the failing share has an explicit, per-rule denominator. Rule three is eligible only on rows with a non-null value_numeric; rule four only on final rows.
For the unit rule, derive the modal unit per loinc_code from the data instead of hardcoding an expected unit. The failure being hunted is one analyte reported in two units whose values differ by a fixed factor, which no range check catches because both sets of numbers look plausible.
Emit a tidy frame with columns rule, eligible_rows, failing_rows, failing_share, example_result_id. A printed report cannot be diffed between runs; a frame can be stored and alerted on.
Order the output by failing_share descending so the run has a lead finding rather than four equal-weight lines.

Follow-up

The modal-unit rule fires on 8 percent of one analyte. How do you decide whether that is a unit-conversion bug or a second legitimate assay?
What would you add to catch a value that is inside its reference range but physiologically impossible?
Which of these four rules should block a downstream pipeline and which should only alert, and why?

Build a claims lag triangle and complete recent months

Medium · chain ladder · runout · pivot

medical_claim_line has claim_id, claim_version, frequency_code, service_start_date, paid_date, allowed_amount, claim_status. Using 24 fully mature incurred months, build a development triangle of allowed dollars by incurred month and payment lag in whole months, derive cumulative completion factors by chain ladder, then estimate ultimate allowed for the three most recent incurred months at a stated paid-through date. Deduplicate to surviving claim versions before building the triangle. Return incurred_month, paid_to_date, completion_factor, estimated_ultimate.

Approach

Deduplicate to surviving claim versions and keep paid lines first. A triangle built on all versions develops on adjustment churn rather than on payment timing, and replacements arrive late, so the distortion concentrates in exactly the tail you are trying to estimate.
Compute lag as a whole-month difference between incurred month and paid month, not as day difference divided by 30. Ragged month lengths otherwise shuffle identical claims between lag buckets depending on which month they fall in.
Pivot to incurred_month by lag, cumulate along lag, then form age-to-age factors as the ratio of the column L+1 total to the column L total, summed over incurred months mature enough to have both. Volume weighting is the chain ladder; a simple mean of per-month ratios lets one low-volume month dominate the factor.
Chain the age-to-age factors from each lag to ultimate and invert: completion factor at lag L is the reciprocal of the product of factors from L onward. Apply as estimated_ultimate = paid_to_date / completion_factor.
State the assumption you have just made, which is that the development pattern is stable. It is not, after a claims-system migration, a network change or a processing backlog, and the most recent month's factor is the least reliable because it rests on the fewest observations while carrying the largest adjustment.

Worked solution (35 min)

Deduplicate to surviving versions, drop voids, keep claim_status 'paid', and derive incurred_month from service_start_date and paid_month from paid_date.
lag = (paid_month.year - incurred_month.year) x 12 + (paid_month.month - incurred_month.month); drop negative lags and investigate them separately.
Pivot to a triangle of summed allowed_amount, cumulate along the lag axis, and mask cells past the paid-through date so no cell contains a future payment.
Compute volume-weighted age-to-age factors per lag over the mature rows, then chain them into cumulative completion factors.
Apply the factor for each recent month's current maturity to its paid_to_date and return the four columns.

Expected result: Three rows. Completion is well below 1 at short lag and approaches 1 in the tail; for a mixed medical book a one-month-old incurred month commonly sits far from complete, while a twelve-month-old one is close. Exact values are book-specific, so the reviewable property is monotonicity and the holdout test below, not the numbers themselves.

Verify it yourself

Cumulative completion factors are non-decreasing in lag and converge toward 1 at the tail.
Hold out one fully mature incurred month, apply the factors at each of its intermediate maturities, and confirm each estimate lands within a few percent of its known total.
estimated_ultimate is greater than or equal to paid_to_date for every row.
A pharmacy-only triangle completes far faster than an inpatient-facility-only triangle, which confirms the lag computation is reading real payment timing.

Follow-up

A processing backlog means the last two months developed slower than history. What does your estimate do, and how would you detect that before reporting?
Pharmacy, professional and inpatient facility develop on different clocks. How do you split the triangle, and what breaks if the service mix shifts?
What happens to a PMPM series if someone reports an unadjusted recent month, and in which direction does the error point?

Turn overlapping coverage spans into fractional member-month denominators

Medium · intervals · gaps and islands · denominators

member_enrollment holds span_id, member_id, plan_id, product_type, effective_date and termination_date, which is NULL while the span is active. A member can hold several spans in a year, and back-dated plan changes make spans overlap. Return one row per calendar month of 2025 with total member-months, where a member contributes covered days in the month divided by that month's length, and a day covered by two overlapping spans counts once. Both span endpoints are inclusive. Collapse overlaps with window functions before joining a month calendar, and do not assume one span per member.

Approach

Normalise the open end first. COALESCE(termination_date, DATE '9999-12-31') or clip to the reporting end, because a NULL termination_date makes every BETWEEN comparison unknown and drops exactly the members who are still covered.
Merge overlapping spans per member before the calendar join. Order by effective_date, carry a running MAX of prior termination_date with a frame ending one row before, and open a new island where effective_date exceeds that running max plus one day, which keeps adjacent spans contiguous rather than splitting them.
Clip each merged island to the reporting year, then cross join to a twelve-row month calendar and keep pairs that intersect.
Compute covered days as LEAST(island_end, month_end) minus GREATEST(island_start, month_start) plus one, and divide by the actual length of that month rather than a flat 30, so February and the 31-day months are not distorted.
Sum the fractions by month. Merging after the calendar join also works but forces a DISTINCT over member-days, which stops scaling at population size.

Follow-up

Where does a continuous-enrolment filter with an allowable 45-day gap belong, and how does that change the island boundary rule?
One human appears under two member_ids after a master person index merge. What does that do to member-months, and how would you detect it?
How do you treat a span that terminates on the day it starts, and what does that row usually represent?

Report site activation to first randomization without hiding non-enrollers

Medium · grain collapse · left join · censoring

study_site holds one row per (study_id, site_id): site_activation_date, which is NULL for a site selected but never activated, and site_status. study_subject_visit holds subject_visit_id, study_id, site_id, subject_id, randomization_ts (NULL for screen failures), visit_name, visit_number, actual_visit_date and visit_status, with one row per subject per protocol visit, so randomization_ts repeats across every visit of a randomized subject and a site that has screened nobody has no rows there at all. For one study_id and a stated data cut date, return per activated site the activation date, the first randomization_ts and the days between them, with activated sites that have randomized nobody still present. Add a study-level summary giving median days to first randomization and the share of activated sites with no randomization by day 90.

Approach

Take the site universe from study_site, never from study_subject_visit. The visit table can only contain sites that have screened at least one subject, so a universe built from DISTINCT site_id there omits exactly the sites this metric exists to expose. Filter study_site to the study and to site_activation_date IS NOT NULL, then LEFT JOIN the per-site first randomization.
Collapse the grain before aggregating anything on the visit side. The table repeats randomization_ts across every visit of a randomized subject, so any count taken at row grain is inflated by visits per subject. MIN(randomization_ts) per site is safe, COUNT is not.
State the date boundary rule. randomization_ts is a timestamp and site_activation_date is a date, so a randomization at 23:40 local time can land on either side of a day boundary depending on the zone the timestamp is stored in. Cast once, deliberately, and say which zone.
Use PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days) for the median and be explicit that it ignores NULLs, so it is a median over enrolling sites only. That is exactly why the non-enroller share is reported beside it rather than instead of it.
Censor the day-90 denominator against the data cut. A site activated 40 days before the cut has not had 90 days to randomize, so it cannot be scored on that bar and belongs outside that denominator.

Worked solution (25 min)

CTE sites: site_id and site_activation_date from study_site for the study where site_activation_date IS NOT NULL. The (study_id, site_id) key already gives one activation date per site, so assert that key rather than de-duplicating defensively.
CTE first_rand: site_id and MIN(randomization_ts) over study_subject_visit rows for the study where randomization_ts IS NOT NULL, grouped by site_id.
LEFT JOIN sites to first_rand and compute days_to_first_randomization with an explicit cast of the timestamp to a date.
Summary: PERCENTILE_CONT(0.5) over non-null days, plus the count and share of sites with no randomization by day 90, restricted to sites activated at least 90 days before the data cut.
Compare the site-level row count with COUNT(DISTINCT site_id) in study_subject_visit for the study; the difference is the activated sites that have screened nobody, which is the population the metric is for.

Expected result: One row per activated site with site_id, site_activation_date, first_randomization_ts and days_to_first_randomization, NULL in both randomization columns for sites that have randomized nobody, including sites with no study_subject_visit rows at all; plus one summary row with median_days_over_enrolling_sites, sites_activated_90d_or_more and share_with_no_randomization_by_day_90.

Verify it yourself

Site-level row count equals the number of study_site rows for the study with a non-null site_activation_date, and is greater than or equal to COUNT(DISTINCT site_id) in study_subject_visit for that study. Below it means either an activated site is missing from study_site or a visit row carries a site_id whose activation date is NULL, both data-quality findings.
No days_to_first_randomization is negative. A negative value is a data-quality finding to raise, not a number to clamp at zero.
The day-90 denominator excludes sites activated within 90 days of the data cut. Including them can only add sites that have not had a full 90 days, and any of those without a randomization would be miscounted as a failure.

Follow-up

The same site_id appears under two studies. What breaks, and what is the correct grain for this metric?
Would you present this as a median, as a Kaplan-Meier curve of time to first randomization, or both, and what does the curve show that the median cannot?
A site randomized one subject who withdrew at visit 2. Does that site count as enrolling here, and how does that differ from a randomized-and-retained measure?

Explain the trade-offs between different classification metrics (e.g., sensitivity vs. specificity) in the context of diagnostic testing.

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you validate your models to ensure they generalize well to new, unseen patient populations?

Medium

Approach

Say what you would check first and why it is the highest-information step.
Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Size a clinic-randomised trial when the intracluster correlation binds

Medium · cluster randomisation · design effect · stepped wedge

A care-coordination change is delivered at the clinic level and cannot be withheld from some members of a participating clinic. Forty-eight clinics are available for 1:1 randomisation. The primary outcome is the 90-day primary care contact rate among newly enrolled members, baseline 0.62, with a target of plus 5 percentage points. Each clinic contributes about 60 eligible members a year, and the intracluster correlation on this outcome is 0.02. Two-sided alpha 0.05, 80 percent power. Give clinics required per arm, say whether 48 suffices, and what you would do.

Approach

Compute the individually randomised sample size first, then inflate it by the design effect 1 + (m - 1) x ICC, where m is the average cluster size. The design effect is the price of correlation within clinics and is the single largest term in this calculation.
Convert the inflated individual N into clusters by dividing by m, and compare with the 24 clinics available per arm.
Check the ceiling before proposing a longer accrual window. As m grows, effective members per cluster approaches 1 / ICC, here 50, so 24 clinics can never contribute more than 1,200 effective members per arm no matter how many members each one enrols. If the required effective N exceeds that, no amount of accrual rescues the parallel design.
Propose the designs that actually change the arithmetic: a stepped wedge in which every clinic crosses over so each contributes a within-cluster comparison, or a smaller randomisation unit (provider pod) if the intervention can be delivered at that level without contamination.
Add the two refinements a reviewer will ask for: use a t-distribution with roughly 2k - 2 degrees of freedom rather than z when k is small (here about 46 df, which raises N by about 4 percent), and use covariate-constrained randomisation on baseline contact rate so 48 clinics do not land badly imbalanced by chance.

Worked solution (30 min)

Individual n per arm = 7.849 x (0.62 x 0.38 + 0.67 x 0.33) / 0.05^2 = 7.849 x 0.4567 / 0.0025 = 1,434.
Design effect = 1 + (60 - 1) x 0.02 = 2.18.
Inflated n per arm = 1,434 x 2.18 = 3,126 members, so clusters per arm = 3,126 / 60 = 52.1, round up to 53.
Compare with 24 clinics available per arm: the study is short by more than a factor of two.
Compute the ceiling: 1 / 0.02 = 50 effective members per clinic, so 24 clinics give at most 1,200 effective members per arm against the 1,434 required.

Expected result: 53 clinics per arm (106 total) against 48 available. The effective-sample ceiling of 1,200 per arm is below the 1,434 needed, so the parallel cluster design is infeasible at any panel size; the answer is a stepped wedge, a smaller randomisation unit, or a pre-registered larger MDE.

Verify it yourself

Verify the design effect directly: 1 + 59 x 0.02 = 2.18.
Verify the diminishing return: doubling m from 60 to 120 raises effective members per cluster only from 27.5 to 35.5, not from 60 to 120.
Recompute with t-based quantiles at 46 df and confirm the requirement moves by a few percent, not by a factor.

Follow-up

Sketch the stepped-wedge analysis model. What does a secular trend do to the estimate if you leave period effects out?
The ICC was estimated from a prior year at 0.02 with a wide interval. How do you size under ICC uncertainty?
Six clinics withdraw after randomisation. What does that do to power compared with losing the same number of members?

Avoidable admissions fell the quarter a programme launched

Hard · mix shift · regression to the mean · reclassification · seasonality

Ambulatory care sensitive admissions per 1,000 member-years fell 18 percent in the quarter a care-management programme launched, enrolling the top 2 percent of members by prior-year spend. You have encounter, member_enrollment including a product line added mid-quarter, and medical_claim_line. Emergency visits per 1,000 rose 9 percent over the same quarter and observation stays rose as well. Produce a decomposition that states how much of the 18 percent is attributable to the programme, and state the design you would need before claiming any of it.

Approach

Rebuild the denominator from coverage spans as member-years, counting partial coverage fractionally, and recompute the rate with and without the product line that entered mid-quarter. A newly added, structurally healthier population inflates the denominator and deflates every rate built on it without a single admission being avoided.
Freeze the numerator's code list to one version. The ambulatory care sensitive list is defined on ICD-10-CM principal diagnosis codes, and in the United States that code set is updated effective October 1, so a quarter spanning that boundary can gain or lose qualifying codes. Recompute both quarters against a single code set version.
Decompose the remaining change with a Kitagawa split across product_type and prior-year risk decile: the mix term is the population getting younger or healthier, the within term is the only place a programme effect can live.
Test reclassification directly. An inpatient stay recorded as an observation stay is not an avoided admission. Compute total acute contacts per 1,000 member-years as inpatient plus observation plus emergency, and check whether that total moved at all. If it is flat while inpatient falls, care was relabelled and deflected, not prevented.
Compare to the same quarter of the prior year rather than the adjacent quarter. Several ambulatory care sensitive conditions, including heart failure decompensation and bacterial pneumonia, are strongly seasonal, so a quarter-over-quarter comparison confounds the programme with the calendar.
Address selection last and explicitly. The enrolled cohort was chosen on an extreme of prior-year spend, so it regresses toward the mean with no intervention at all, because the selecting year captured both chronic severity and one-off events. A pre-post design on this cohort will report savings every time. The minimum credible design is a concurrent comparison group selected by the same rule in the same period, or a regression discontinuity at the 2 percent threshold, with a parallel-trends check on at least four pre-periods.

Follow-up

The parallel-trends check fails on two of the four pre-periods. What do you do, without abandoning the question?
Express the effect you can defend as an absolute risk difference and a number needed to treat, with the baseline rate stated. Why is that more useful here than a relative reduction?
The programme cannot be withheld from anyone. Sketch a rollout design that still identifies an effect, and say what secular trend does to the estimate.

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Explaining a method to someone who will not follow the maths

Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.

How do you handle missing or noisy data in a clinical dataset?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Describe a time you had to select a model architecture for a specific predictive task. What were the key considerations?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Scope a one-line request for a readmission rate

Easy · scoping · measure specification · stakeholder communication

A clinical operations lead messages you: 'What is our readmission rate? The committee meets Friday.' You have encounter (encounter_id, admit_ts, discharge_ts, encounter_type, admission_type, discharge_disposition, drg_code, index_encounter_id) and member_enrollment coverage spans. Do not open a query editor yet. Deliverable: the three to five questions you send back, ranked by how much the answer moves the number; the default specification you will build if nobody replies by Wednesday; and the one line you will place under the figure so the committee does not read it against a published benchmark.

Approach

Read the probe: this tests whether you convert ambiguity into a specification without either stalling or guessing silently. Both failure modes are common, and the silent guess is worse because nobody can see it.
Rank your questions by leverage on the number rather than by curiosity. Whether the numerator is all-cause or unplanned, and whether the clock starts at discharge_ts, move the result far more than a tie-break rule on same-day returns.
Ask about the comparison first. A number going to a committee will be compared to something, and whether that something is last quarter, another panel, or a national figure decides whether you owe them a raw rate, a risk-adjusted rate, or an observed-over-expected ratio.
Commit to a default in writing so silence does not block you: unplanned acute inpatient readmission within 30 days of discharge_ts, index stays excluding planned admissions, acute-to-acute transfers, discharges against medical advice, and dispositions of expired or hospice, denominator of eligible index discharges, paid-through date stated.
Write the caveat as a restriction on use rather than a hedge: name the one comparison the figure supports and the one it does not.

Follow-up

They reply that they want it 'the way the benchmark does it'. What do you ask next?
The committee wants it split by attending provider. What changes in the specification, and what do you refuse to show?
How does your answer change if the meeting is tomorrow rather than Friday?