Interview guides · Grail

Grail Data Scientist Interview Questions & Guide 2026

Modeling drills for a Grail data scientist interview: baselines, regression interpretation, class imbalance, model defense.

The 60-second brief

A Data Scientist at Grail operates at the absolute frontier of healthcare technology and data science. The company's mission—to detect cancer early when it can be cured—relies entirely on the ability to find incredibly faint biological signals within massive, highly complex genomic datasets. In this role, you are not just building standard recommendation engines or churn models; you are developing machine learning algorithms and statistical frameworks that directly impact clinical outcomes and patient lives.

In modelling rounds the live question is usually why this model class for this problem, and how you would know six weeks after launch that it is still working. Deriving gradients by hand is rarely what is being probed.

Focus areas

Minimise protected health information in every extract
Separate censoring from events in time-to-event models
Reconstruct drug exposure intervals from dispensing records

Overview

This position is highly interdisciplinary, requiring you to bridge the gap between complex software engineering, advanced machine learning, and biological data pipelines. Because Grail deals with high-dimensional genomic sequencing data, the algorithms you design must be exceptionally optimized, highly scalable, and mathematically rigorous. You will collaborate closely with clinical researchers, software engineers, and product teams to translate raw sequencing inputs into highly accurate, life-saving diagnostic insights.

The work is both intellectually challenging and deeply meaningful. As a, you will face unique technical challenges, such as handling sparse matrices, optimizing deep learning architectures, and building robust system engineering pipelines that can process petabytes of clinical data. If you are motivated by high-stakes problem-solving and want to apply your technical expertise to one of the most critical challenges in modern medicine, this role offers an unparalleled platform for impact.

The Grail interview process

Grail candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Technical Screening

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Coding Assessment

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Virtual Onsite Loop

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Behavioral Conversations

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Treating clinical measurements as missing at random

A lab result, a vital sign, or a screening exists because someone ordered it, and ordering tracks suspicion of disease, visit frequency, and site workflow. Imputing the mean or dropping incomplete rows biases the population estimate and can flip the sign of an association, because the untested are systematically healthier or systematically disengaged. The presence indicator is often more predictive than the value, which is a warning sign rather than a feature win: a model that learns test ordering will not transfer to a site with different protocols.

Immortal time in adherence, treatment, and enrolment definitions

Classifying members as adherent, treated, or programme-enrolled requires them to survive and stay covered long enough to accumulate the defining events. That guaranteed event-free interval is assigned to the exposed group, so the exposure looks protective for reasons that have nothing to do with the treatment. Adherence studies are the classic case: measuring 12-month proportion of days covered and then comparing mortality builds survival into the exposure definition. Use time-varying exposure or a landmark analysis with the classification window excluded from follow-up.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Questions & practice

Describe how you would validate a model to prevent data leakage, especially when dealing with longitudinal or patient-grouped data.

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Describe how you would monitor a deployed machine learning model for data drift and performance degradation in a production environment.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

How would you handle extreme class imbalance when training a deep learning model for rare event detection?

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Bootstrap a confidence interval for a ratio-of-sums PMPM

Medium · bootstrap · ratio estimator · resampling unit

You have one row per member for a fully run-out incurred year: member_id, member_months, allowed_amount. Report allowed PMPM, defined as sum(allowed_amount) divided by sum(member_months), with a 95 percent confidence interval from a bootstrap you write yourself. No library resampler. Resample members with replacement, recompute the ratio on each of 2,000 replicates, take a percentile interval, and seed the generator so the result reproduces. In two sentences, state why resampling claim lines instead of members would be wrong.

Approach

Name the estimator precisely: PMPM is a ratio of sums, not the mean of per-member ratios. The two differ whenever exposure correlates with spend, and it does, because members who churn out mid-year are systematically different from members covered all twelve months.
Resample the unit of independence, which is the member, and recompute numerator and denominator together on the same draw. Recomputing only the numerator against a fixed denominator understates variance and is a common slip.
Pull member_months and allowed_amount into numpy arrays once, then draw indices with rng.integers(0, n, size=n) per replicate. Index draws over arrays run one to two orders of magnitude faster than DataFrame.sample, which is the difference between 2,000 replicates and 200.
Take the 2.5th and 97.5th percentiles of the replicate ratios. Expect an interval that is asymmetric around the point estimate, because healthcare spend is right skewed. That asymmetry is the method working, not a bug to centre away.
Answer the line-resampling question concretely: the many lines of one high-cost member are not independent draws, so line resampling shrinks the interval toward zero width, and member_months has no line-level analogue, so the denominator becomes undefined.

Worked solution (25 min)

Compute the point estimate as allowed.sum() / months.sum() on the full sample.
Extract both columns to float numpy arrays and create rng = np.random.default_rng(seed).
Loop 2,000 times: idx = rng.integers(0, n, n); ratio = allowed[idx].sum() / months[idx].sum(); store it.
Take np.percentile(replicates, [2.5, 97.5]) and report the point estimate with that interval and the seed.

Expected result: A point estimate plus a 95 percent percentile interval that is wider on the upper side than the lower. With tens of thousands of members and a typical commercial cost distribution the relative half-width lands on the order of a few percent; an interval a small fraction of a percent wide is the signature of having resampled rows rather than members.

Verify it yourself

Re-running with the same seed reproduces both bounds exactly.
Raising replicates from 2,000 to 8,000 moves each bound by less than its third significant figure.
The mean of the replicate ratios sits close to the point estimate; a large gap indicates a bug in the resampler rather than skew.
Replicate denominators cluster tightly around the observed total member-months, which confirms the draw size is n rather than a fixed constant.

Follow-up

Half a percent of members have a transplant or a long NICU stay. Does a percentile interval still have nominal coverage, and what would you do about the tail?
The question changes to comparing PMPM across provider panels. What becomes the resampling unit and why?
How would you use the same replicates to test whether this year's PMPM differs from last year's, given the same members appear in both?

Explain how you would represent a highly sparse dataset in memory to minimize space while maintaining fast lookup times.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Write a function to reverse a linked list or manipulate complex nested data structures.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Solve a complex optimization problem using dynamic programming, explaining the time and space complexity of your approach.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Report site activation to first randomization without hiding non-enrollers

Medium · grain collapse · left join · censoring

study_site holds one row per (study_id, site_id): site_activation_date, which is NULL for a site selected but never activated, and site_status. study_subject_visit holds subject_visit_id, study_id, site_id, subject_id, randomization_ts (NULL for screen failures), visit_name, visit_number, actual_visit_date and visit_status, with one row per subject per protocol visit, so randomization_ts repeats across every visit of a randomized subject and a site that has screened nobody has no rows there at all. For one study_id and a stated data cut date, return per activated site the activation date, the first randomization_ts and the days between them, with activated sites that have randomized nobody still present. Add a study-level summary giving median days to first randomization and the share of activated sites with no randomization by day 90.

Approach

Take the site universe from study_site, never from study_subject_visit. The visit table can only contain sites that have screened at least one subject, so a universe built from DISTINCT site_id there omits exactly the sites this metric exists to expose. Filter study_site to the study and to site_activation_date IS NOT NULL, then LEFT JOIN the per-site first randomization.
Collapse the grain before aggregating anything on the visit side. The table repeats randomization_ts across every visit of a randomized subject, so any count taken at row grain is inflated by visits per subject. MIN(randomization_ts) per site is safe, COUNT is not.
State the date boundary rule. randomization_ts is a timestamp and site_activation_date is a date, so a randomization at 23:40 local time can land on either side of a day boundary depending on the zone the timestamp is stored in. Cast once, deliberately, and say which zone.
Use PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days) for the median and be explicit that it ignores NULLs, so it is a median over enrolling sites only. That is exactly why the non-enroller share is reported beside it rather than instead of it.
Censor the day-90 denominator against the data cut. A site activated 40 days before the cut has not had 90 days to randomize, so it cannot be scored on that bar and belongs outside that denominator.

Worked solution (25 min)

CTE sites: site_id and site_activation_date from study_site for the study where site_activation_date IS NOT NULL. The (study_id, site_id) key already gives one activation date per site, so assert that key rather than de-duplicating defensively.
CTE first_rand: site_id and MIN(randomization_ts) over study_subject_visit rows for the study where randomization_ts IS NOT NULL, grouped by site_id.
LEFT JOIN sites to first_rand and compute days_to_first_randomization with an explicit cast of the timestamp to a date.
Summary: PERCENTILE_CONT(0.5) over non-null days, plus the count and share of sites with no randomization by day 90, restricted to sites activated at least 90 days before the data cut.
Compare the site-level row count with COUNT(DISTINCT site_id) in study_subject_visit for the study; the difference is the activated sites that have screened nobody, which is the population the metric is for.

Expected result: One row per activated site with site_id, site_activation_date, first_randomization_ts and days_to_first_randomization, NULL in both randomization columns for sites that have randomized nobody, including sites with no study_subject_visit rows at all; plus one summary row with median_days_over_enrolling_sites, sites_activated_90d_or_more and share_with_no_randomization_by_day_90.

Verify it yourself

Site-level row count equals the number of study_site rows for the study with a non-null site_activation_date, and is greater than or equal to COUNT(DISTINCT site_id) in study_subject_visit for that study. Below it means either an activated site is missing from study_site or a visit row carries a site_id whose activation date is NULL, both data-quality findings.
No days_to_first_randomization is negative. A negative value is a data-quality finding to raise, not a number to clamp at zero.
The day-90 denominator excludes sites activated within 90 days of the data cut. Including them can only add sites that have not had a full 90 days, and any of those without a randomization would be miscounted as a failure.

Follow-up

The same site_id appears under two studies. What breaks, and what is the correct grain for this metric?
Would you present this as a median, as a Kaplan-Meier curve of time to first randomization, or both, and what does the curve show that the median cannot?
A site randomized one subject who withdrew at visit 2. Does that site count as enrolling here, and how does that differ from a randomized-and-retained measure?

Write a comprehensive suite of unit tests for a data processing script, explaining your choice of test cases and mock data.

Medium

Approach

Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Explain how Python manages memory, specifically addressing how it handles large integers and why integer overflow behaves differently in Python compared to other languages.

Medium

Approach

State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Size a clinic-randomised trial when the intracluster correlation binds

Medium · cluster randomisation · design effect · stepped wedge

A care-coordination change is delivered at the clinic level and cannot be withheld from some members of a participating clinic. Forty-eight clinics are available for 1:1 randomisation. The primary outcome is the 90-day primary care contact rate among newly enrolled members, baseline 0.62, with a target of plus 5 percentage points. Each clinic contributes about 60 eligible members a year, and the intracluster correlation on this outcome is 0.02. Two-sided alpha 0.05, 80 percent power. Give clinics required per arm, say whether 48 suffices, and what you would do.

Approach

Compute the individually randomised sample size first, then inflate it by the design effect 1 + (m - 1) x ICC, where m is the average cluster size. The design effect is the price of correlation within clinics and is the single largest term in this calculation.
Convert the inflated individual N into clusters by dividing by m, and compare with the 24 clinics available per arm.
Check the ceiling before proposing a longer accrual window. As m grows, effective members per cluster approaches 1 / ICC, here 50, so 24 clinics can never contribute more than 1,200 effective members per arm no matter how many members each one enrols. If the required effective N exceeds that, no amount of accrual rescues the parallel design.
Propose the designs that actually change the arithmetic: a stepped wedge in which every clinic crosses over so each contributes a within-cluster comparison, or a smaller randomisation unit (provider pod) if the intervention can be delivered at that level without contamination.
Add the two refinements a reviewer will ask for: use a t-distribution with roughly 2k - 2 degrees of freedom rather than z when k is small (here about 46 df, which raises N by about 4 percent), and use covariate-constrained randomisation on baseline contact rate so 48 clinics do not land badly imbalanced by chance.

Worked solution (30 min)

Individual n per arm = 7.849 x (0.62 x 0.38 + 0.67 x 0.33) / 0.05^2 = 7.849 x 0.4567 / 0.0025 = 1,434.
Design effect = 1 + (60 - 1) x 0.02 = 2.18.
Inflated n per arm = 1,434 x 2.18 = 3,126 members, so clusters per arm = 3,126 / 60 = 52.1, round up to 53.
Compare with 24 clinics available per arm: the study is short by more than a factor of two.
Compute the ceiling: 1 / 0.02 = 50 effective members per clinic, so 24 clinics give at most 1,200 effective members per arm against the 1,434 required.

Expected result: 53 clinics per arm (106 total) against 48 available. The effective-sample ceiling of 1,200 per arm is below the 1,434 needed, so the parallel cluster design is infeasible at any panel size; the answer is a stepped wedge, a smaller randomisation unit, or a pre-registered larger MDE.

Verify it yourself

Verify the design effect directly: 1 + 59 x 0.02 = 2.18.
Verify the diminishing return: doubling m from 60 to 120 raises effective members per cluster only from 27.5 to 35.5, not from 60 to 120.
Recompute with t-based quantiles at 46 df and confirm the requirement moves by a few percent, not by a factor.

Follow-up

Sketch the stepped-wedge analysis model. What does a secular trend do to the estimate if you leave period effects out?
The ICC was estimated from a prior year at 0.02 with a wide interval. How do you size under ICC uncertainty?
Six clinics withdraw after randomisation. What does that do to power compared with losing the same number of members?

First-pass denial rate doubled in one received week

Medium · pipeline failure · segment cuts · denominator definition

First-pass denial rate jumped from 6.1 to 11.8 percent in a single received_date week and stayed at the new level. Nothing in the adjudication rules changed that week. You have medical_claim_line: claim_id, claim_version, frequency_code, claim_status, denial_reason_code, member_id, received_date, adjudicated_at, billing_provider_npi. The metric counts lines whose first adjudication returns 'denied', over lines received in the window at claim_version = 1. Find the cause, say whether denial behaviour actually changed, and deliver the corrected series plus the query change that fixes it.

Approach

Cut the excess by denial_reason_code before anything else. A behaviour change spreads across codes; an upstream failure concentrates in one. If a single eligibility-related code carries nearly all of the 5.7 point excess, the claims are fine and the eligibility data was not.
Join the denied lines to member_enrollment and compare the span's created_at to the claim's received_date. Lines denied for members whose coverage span loaded after the claim arrived are a load-ordering failure, not a coding failure.
Check resolution downstream: take those claim_ids and look for a later version with frequency_code 7 that adjudicates to 'paid'. A high resubmit-and-pay share confirms the denial was transient and recoverable.
Verify the denominator is not the mover. Count lines at claim_version = 1 by received_date and check the weekday mix, because submission batching makes received_date strongly day-of-week patterned and a short or holiday week shifts the denominator without changing behaviour.
Confirm runout: the metric should be held until 60 days of adjudication have passed, so check that the comparison weeks are equally mature rather than comparing a settled week to a fresh one.
Restate the series with the affected reason code broken out as its own line, and change the query to carry denial_reason_code into the reported breakdown rather than only the headline rate.

Follow-up

The affected lines all get paid on resubmission. Is the first-pass denial rate then wrong, or right and uninteresting?
How would you monitor for this class of failure without waiting for someone to notice a chart?
What changes if the excess is spread evenly across denial_reason_code instead of concentrated?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Explaining a method to someone who will not follow the maths

Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.

Disagree with a product manager about an adherence nudge

Medium · immortal time · confounding · disagreement

A product manager proposes shipping a refill-reminder feature to the whole member base, citing an internal analysis: members with 12-month proportion of days covered at or above 0.80 had 31 percent fewer admissions than members below it. The roadmap date is three weeks out and engineering is scoped. You believe the comparison builds survival and continuous coverage into the exposure definition, and is separately confounded by the kind of member who refills on time. Deliverable: what you say in the room, what you send afterwards, and the smallest study you would accept as sufficient to ship.

Approach

The probe is whether you can be specific and non-territorial at once. A disagreement that arrives as 'the analysis is flawed' costs you the room; one that names a mechanism and offers a cheaper path keeps it.
Name the two defects separately, because they have different fixes. Classifying a member as adherent over 12 months requires them to stay alive and stay covered for those 12 months, and that guaranteed event-free interval is assigned to the adherent group. Separately, members who refill on schedule differ from those who do not in ways claims never record.
Show rather than assert. Rerun the same comparison as a landmark analysis: classify adherence over days 1 to 90, start follow-up at day 91 for everyone still enrolled and event-free, and report how much of the 31 percent survives.
Separate the association claim from the intervention claim out loud. Nothing in the analysis speaks to whether a reminder changes refill behaviour, which is the actual product question and is separately testable.
Offer the shippable path: a randomised holdout on a subset with proportion of days covered as the primary endpoint, powered on the plausible effect of a reminder rather than on the 31 percent, with a stated read date.

Follow-up

The landmark rerun still shows a 20 percent difference. Do you ship?
The product manager says a holdout delays launch. How do you cost that argument?
What primary endpoint do you choose, and why not admissions?

Explain an incomplete cost series to a non-technical executive

Medium · claims runout · uncertainty · executive communication

An executive opens a dashboard showing risk-adjusted allowed PMPM down 6 percent across the last three incurred months and asks whether the cost initiative is working. You know pharmacy adjudicates within days, professional claims within weeks, and inpatient facility claims over months, so those three months are both undercounted and mix-shifted toward cheap services. There is also a genuine movement of roughly 1.5 percent in the older, complete months. Deliverable: a two-minute spoken answer, the change you make to the chart, and what you commit to telling them and when.

Approach

The probe is whether you can be honest about uncertainty without sounding evasive. Executives read hedging as not knowing, so the answer has to end in a commitment.
Separate the two claims immediately. The 6 percent is an artefact of incomplete data, and there is a smaller real movement in the months that are complete. Delivering only the debunk leaves them holding nothing.
Explain the lag in their terms: the bills for the most expensive care arrive last, so an incomplete month always looks cheap and always looks like it is improving. Keep the phrase completion factor out of the first pass.
Fix the chart instead of explaining around it. Shade or omit the incomplete months, print the paid-through date on the axis, and overlay the same series as it looked at equivalent maturity a year earlier so the shape is comparable rather than merely lower.
Close with a date and a magnitude. Commit to a restated figure once runout matures, say roughly how far you expect the 6 percent to shrink, and name what you would need to see to call the initiative working.

Follow-up

They ask for your best guess today, knowing it is provisional. What do you say?
The restated number comes back at 1 percent. How do you handle having flagged the 6 percent at all?
How do you stop this dashboard producing the same conversation next quarter?

Make an honest case for your own analytical impact

Hard · impact attribution · counterfactual · self-assessment

You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.

Approach

The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.

Follow-up

Two other people also claim the contract measure. How do you split it?
Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
What is the strongest argument that your year had less impact than you just described?