Interview guides · Elevance Health

Elevance Health Data Scientist Interview Questions & Guide 2026

Data scientist interview prep for Elevance Health: original practice on experiment design, SQL and metric reasoning, with worked solutions.

The 60-second brief

The Data Scientist role at Elevance Health is pivotal in shaping the future of healthcare analytics and decision-making. As a Data Scientist, you will harness the power of data to derive insights that directly influence the health outcomes of millions. Your work will span multiple domains, including predictive modeling, machine learning, and data visualization, driving initiatives that improve our services and enhance user experiences.

If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.

Focus areas

Risk-adjust outcomes before comparing providers or panels
Separate censoring from events in time-to-event models
Build member-month denominators from overlapping coverage spans

Overview

In this role, you will collaborate with cross-functional teams, including product development, operations, and business intelligence, to tackle complex challenges. The impact of your analyses will not only enhance our product offerings but also support strategic initiatives that streamline operations and optimize patient care. You will engage with real-world problems, making your contributions both meaningful and essential to the mission of Elevance Health.

This is an exciting opportunity for candidates who are passionate about data and its transformative potential in the healthcare sector. You will be at the forefront of developing innovative solutions that address the intricate needs of our users while contributing to a culture of data-driven decision-making.

The Elevance Health interview process

Elevance Health candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening Call

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Interviews

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Behavioral Interviews

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Immortal time in adherence, treatment, and enrolment definitions

Classifying members as adherent, treated, or programme-enrolled requires them to survive and stay covered long enough to accumulate the defining events. That guaranteed event-free interval is assigned to the exposed group, so the exposure looks protective for reasons that have nothing to do with the treatment. Adherence studies are the classic case: measuring 12-month proportion of days covered and then comparing mortality builds survival into the exposure definition. Use time-varying exposure or a landmark analysis with the classification window excluded from follow-up.

Reading the most recent months of a claims-based series as real

Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.

Reporting a p-value with no effect size or interval

Give the estimated difference with a confidence interval in the units the business cares about, then say whether that whole interval is worth acting on. A p-value only addresses whether you can rule out exactly zero; it says nothing about magnitude.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Questions & practice

How would you implement a decision tree algorithm from scratch?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Can you explain a time you optimized a piece of code or algorithm?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.
Check what information would not exist at prediction time, and exclude it.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Collapse claim versions before summing allowed amounts by member

Easy · claim versioning · groupby · deduplication

You are given medical_claim_line as a pandas DataFrame: claim_line_id, claim_id, claim_version, frequency_code (1 original, 7 replacement, 8 void), member_id, service_start_date, procedure_code, allowed_amount, claim_status. Return total allowed_amount per member for service dates in one stated calendar month, counting each claim once at its surviving version and excluding any claim whose surviving version is a void. Multi-line claims must keep every line of that surviving version. Output columns: member_id, allowed_amount, sorted descending. Do not deduplicate on claim_id alone.

Approach

Separate the two grains out loud before writing code: versions live at the claim level, dollars live at the line level. Every bug in this exercise comes from mixing them.
Compute the surviving version per claim with a groupby transform of max over claim_version, then keep the lines whose claim_version equals that value. A transform keeps the frame at line grain, which a sort plus head(1) does not.
Drop whole claims whose surviving version carries frequency_code 8. A void is not a zero-dollar line, it retracts the claim, so filtering rows rather than claims leaves the prior version's lines behind.
Filter to claim_status 'paid' and state the assumption: denied, pended and reversed lines carry an allowed_amount that never became a liability, so including them overstates cost.
Restrict service_start_date to the month boundaries, then groupby member_id and sum. Sort descending and return.

Worked solution (20 min)

Build a max-version lookup per claim_id with groupby('claim_id')['claim_version'].transform('max') and keep rows where claim_version equals it.
From the surviving rows, collect the claim_ids whose frequency_code is 8 and drop those claim_ids entirely.
Filter claim_status to 'paid' and service_start_date into the month window.
Group by member_id, sum allowed_amount, sort descending, reset the index.

Expected result: A two-column frame member_id, allowed_amount. On a claim with versions 1, 7, 7 only the third version's lines contribute; a claim whose latest version is frequency_code 8 appears nowhere. Total allowed is strictly below a naive sum over all rows whenever any claim in the sample was adjusted, and equal to it when none was.

Verify it yourself

The distinct claim_id count in your filtered frame equals the count of distinct claim_ids whose surviving version is not a void.
The line count you kept for each surviving claim equals that claim's line count at its max version, not 1.
A naive sum over every row is greater than or equal to your total, with equality only when no claim has claim_version above 1.

Follow-up

A claim has versions 1, 7, 7 where the second replacement has fewer lines than the first. What does your code return and is that correct?
How would you handle a claim whose lines span two calendar months at the boundary of the reporting window?
The same logic has to run over 400 million lines in a warehouse. What changes, and what stays the same?

Given a list of numbers, write a function to find the maximum product of two integers.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Discuss the time complexity of your solution to a problem you've recently solved.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Reconstruct drug coverage intervals and compute proportion of days covered

Hard · gaps and islands · window functions · adherence

pharmacy_claim holds fill_id, member_id, ndc_code, therapeutic_class_code, fill_date, days_supply, reversal_flag and reversed_fill_id. Early refills overlap, so days_supply must be laid end to end: each fill's coverage starts at the later of its fill_date and the previous interval's end. Both rows of a reversed pair are excluded. For one therapeutic_class_code over 2025-01-01 to 2025-06-30, return per member the proportion of days covered, defined as covered days divided by days from first fill_date to window end, plus the share of members at or above 0.80. Require at least two fills and 91 days of follow-up.

Approach

Clean the fills first. Drop rows with reversal_flag TRUE and also the fills they point at through reversed_fill_id. Dropping only the reversal row leaves an original fill counted as exposure that never reached the member. Use NOT EXISTS for that second step, since reversed_fill_id is nullable and NOT IN would return nothing.
Shift, never stack. The shifted end is covered_end_i = GREATEST(fill_date_i, covered_end_{i-1}) + days_supply_i, which looks recursive but has a closed form: with cum_i the running sum of days_supply through fill i, covered_end_i = cum_i + MAX over j <= i of (fill_date_j - cum_{j-1}). That is SUM(days_supply) OVER (ORDER BY fill_date ROWS UNBOUNDED PRECEDING) plus a running MAX of an anchor column, so it is two window functions and no recursive CTE.
Derive covered_start_i as GREATEST(fill_date_i, LAG(covered_end)). The intervals are disjoint by construction, so covered days is a plain sum of lengths and never needs a distinct day grid.
Clip intervals to the window end before summing, and decide explicitly whether to clip at disenrollment as well. Supply that runs past the window must not inflate the numerator.
Use the denominator the spec names, first fill_date to window end, and hold it constant across classes. A fixed-window denominator yields a different and usually lower number, and mixing the two across classes makes the comparison meaningless.
Apply the inclusion rules at member level after the intervals are built, then compute the share at or above 0.80 with the qualifying member count beside it.

Worked solution (40 min)

Filter to the therapeutic_class_code and window, then remove reversal rows and their originals with NOT EXISTS.
Per member ordered by fill_date, compute cum_supply and anchor = fill_date - (cum_supply - days_supply), then covered_end = cum_supply + MAX(anchor) OVER (ORDER BY fill_date ROWS UNBOUNDED PRECEDING).
Compute covered_start = GREATEST(fill_date, LAG(covered_end) OVER (ORDER BY fill_date)), clip both ends to the window, and drop intervals that collapse.
Sum interval lengths as covered_days, compute follow_up_days from first fill_date to window end, and divide.
Apply the two-fill and 91-day rules, then aggregate the share at or above 0.80.

Expected result: One row per qualifying member with covered_days, follow_up_days and pdc in [0, 1], plus a summary row with qualifying members and the share at or above 0.80. A member with 30-day fills on 2025-01-01, 2025-01-20 and 2025-04-01 is covered 2025-01-01 to 2025-03-01 and 2025-04-01 to 2025-04-30, giving 60 + 30 = 90 covered days over 181 follow-up days and a PDC of 0.497.

Verify it yourself

No pdc exceeds 1.0 before any clipping. A value above 1 proves supplies were stacked instead of shifted.
On a twenty-member sample, the summed interval lengths equal the distinct covered day count from a slow day-grid version.
A member with one fill is excluded by the two-fill rule rather than appearing with a PDC of 1.0.
Re-running with reversals left in raises covered days for exactly the members who have a reversed_fill_id, and for no one else.

Follow-up

A member is hospitalised for twelve days mid-window, when the facility supplies medication. Should those days count as covered, and what do published adherence measures do about them?
Someone wants to classify adherence over twelve months and then compare mortality from month zero. What is wrong with that design and what fixes it?
Two different ndc_codes in the same therapeutic class overlap. Is that stacking or switching, and how does your interval logic treat each case?

Explain how you would analyze patient data to identify potential areas for improving health outcomes.

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

Imagine you have to present your findings to a non-technical audience; how would you approach this?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

Given a dataset, how would you identify trends and anomalies?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you approach designing a data pipeline for a new healthcare application?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Can you explain how you would handle missing data in a dataset?

Medium

Approach

Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Specify a readmission rate two parties will settle on

Easy · measure specification · exclusions · observed over expected · runout

A payer and a hospital group are writing a shared-savings term that settles on 30-day unplanned readmissions. You have encounter (encounter_id, patient_id, member_id, facility_id, encounter_type, admit_ts, discharge_ts, admission_type, discharge_disposition, is_planned_admission, index_encounter_id, drg_code, principal_diagnosis_code) and member_enrollment coverage spans. Produce the measure specification both sides sign: numerator, denominator, exclusion list, observation window, and the paid-through date at which a discharge month becomes reportable. State whether settlement is on the raw rate or an observed-over-expected ratio, and defend the choice.

Approach

Fix the unit of analysis first: the denominator is eligible index stays, not members and not patients. Say out loud that one member can contribute several index stays in a period, and that a readmission is itself eligible to become an index stay, because leaving that unstated is where two implementations diverge.
Write the exclusion list as a waterfall with a count at each step rather than a sentence: planned admissions (is_planned_admission TRUE), discharge_disposition in expired, hospice and ama, and stays whose discharge_disposition is transfer_acute, which are the first leg of one episode rather than an index stay that ended.
Add the exposure condition the encounter table cannot express on its own: an index stay only counts if the member has coverage in member_enrollment for the full 30 days after discharge_ts, with the allowable gap written in days. Without it, a disenrollment reads as a clean outcome.
Define the numerator join precisely: the earliest later inpatient encounter with is_planned_admission FALSE and admit_ts within 30 days of the index discharge_ts, at any facility_id, capped at one readmission per index stay so a single unstable member cannot dominate a facility's rate.
Set the maturity rule. Inpatient facility claims adjudicate slowest of any service type, so name a paid-through lag, put the paid-through date on the report header, and state that a discharge month is restated until it matures.
Settle on observed-over-expected and say why: raw rates confound case mix with care, and the expected model is fitted on age, sex and comorbidities drawn from the 12 months of claims before admit_ts, excluding diagnoses first coded during the index stay so complications of that stay do not inflate the facility's own expectation.

Worked solution (25 min)

Write the eligible-index waterfall as a numbered list before touching any logic, with a placeholder count beside each line so the reviewer can see how much of the raw discharge volume each exclusion removes.
Write the numerator rule in one sentence including the tie-break for two admissions on the same day, and the one-readmission-per-index cap.
State the coverage requirement as a concrete filter: covered on every day from discharge_ts to discharge_ts + 30, allowable gap 0 days, evaluated against member_enrollment spans rather than a single active flag.
Name the paid-through lag and write the restatement policy: a discharge month is provisional until it matures and is republished at every refresh until then.
Write two sentences on why settlement is on O/E, including what risk adjustment does not fix, namely unrecorded severity and differences in social support at discharge.

Expected result: A one-page specification containing: an exclusion waterfall with a count at every step; a numerator counting at most one unplanned acute readmission per index stay within 30 days of discharge_ts at any facility; a 30-day continuous-coverage requirement with the allowable gap stated in days; a named paid-through lag with a restatement policy; and settlement on the observed-over-expected ratio with a confidence interval, on the stated ground that raw rates are not comparable across facilities with different case mix.

Verify it yourself

Recompute the raw rate with and without the transfer_acute exclusion. A facility that receives transfers should move materially; if nothing moves, the transfer legs are not being detected at all.
Compare the numerator count to the count of distinct encounters serving as a readmission. They should be equal; a larger numerator means one readmission is being credited to two index stays.
Re-run the same discharge month at two paid-through dates 60 days apart. A stable rate means the lag is long enough; a rate that climbs means it is not.
Confirm every excluded stay lands in exactly one exclusion bucket so the waterfall sums back to the raw discharge count.

Follow-up

A member transfers from one acute facility to another on day two of the stay. One episode or two, and which facility owns the outcome?
The hospital starts admitting borderline patients to observation status instead of inpatient. What happens to your numerator, your denominator and the rate, and which companion metric would show it?
Why would you refuse to report this rate for a facility with 40 eligible index stays in a quarter?

Protocol deviations fell as new sites came online

Hard · reporting lag · denominator mix · clustering

Protocol deviations per 100 completed visits across a multi-site study fell from 4.6 to 2.9 over two months while randomisation accelerated. study_subject_visit carries study_id, site_id, subject_id, visit_number, visit_name, planned_visit_date, actual_visit_date, visit_window_days, visit_status, protocol_deviation_flag, deviation_severity, data_entry_ts and site_activation_date. Operations wants to credit a retraining rollout that began in that window. Establish whether deviation behaviour changed, and deliver a version of the series that is safe to review monthly.

Approach

Treat deviation flags as late-arriving data. Build a lag triangle of data_entry_ts minus actual_visit_date, because deviations are largely identified during monitoring review weeks after the visit, so the two most recent months are structurally undercounted in exactly the way an improvement looks.
Hold back the months that are not developed, or apply development factors fitted on fully developed months, and restate. Do this before any explanation involving training.
Attack the denominator. It counts visits with visit_status 'completed', while out_of_window and missed visits are excluded even though out-of-window attendance is itself a deviation in most protocols, so a site that pushes visits out of window lowers the rate twice over.
Standardise the visit mix. Newly activated sites contribute mostly screening and baseline visits, which carry fewer procedures and fewer opportunities to deviate than later treatment visits, so compute the rate within visit_number strata and reweight to a fixed mix before comparing months.
Split major from minor deviation_severity and report them separately. A single major deviation can remove a subject from the per-protocol analysis set while ten minor ones may not, so a pooled rate can fall while the consequential series rises.
Before attributing anything to training, model at the right level: subjects are nested in sites, so use a random intercept per site or cluster-robust standard errors. With average cluster size m and intracluster correlation rho the variance is understated by the design effect 1 + (m - 1) times rho, which at m = 60 and rho = 0.02 is about 2.2, so naive intervals are roughly 1.5 times too narrow.

Follow-up

Design a defensible evaluation of the retraining rollout, given it was deployed site by site and cannot be withheld.
Your reweighted series is flat but major deviations rose. What do you escalate and to whom?
How would you decide the holdback period, and how would you communicate a provisional number without it being quoted as final?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Explaining a method to someone who will not follow the maths

Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.

Describe a project where you faced significant challenges and how you overcame them.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.
Close with what you would do differently, concretely.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Make an honest case for your own analytical impact

Hard · impact attribution · counterfactual · self-assessment

You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.

Approach

The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.

Follow-up

Two other people also claim the contract measure. How do you split it?
Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
What is the strongest argument that your year had less impact than you just described?

Disagree with a product manager about an adherence nudge

Medium · immortal time · confounding · disagreement

A product manager proposes shipping a refill-reminder feature to the whole member base, citing an internal analysis: members with 12-month proportion of days covered at or above 0.80 had 31 percent fewer admissions than members below it. The roadmap date is three weeks out and engineering is scoped. You believe the comparison builds survival and continuous coverage into the exposure definition, and is separately confounded by the kind of member who refills on time. Deliverable: what you say in the room, what you send afterwards, and the smallest study you would accept as sufficient to ship.

Approach

The probe is whether you can be specific and non-territorial at once. A disagreement that arrives as 'the analysis is flawed' costs you the room; one that names a mechanism and offers a cheaper path keeps it.
Name the two defects separately, because they have different fixes. Classifying a member as adherent over 12 months requires them to stay alive and stay covered for those 12 months, and that guaranteed event-free interval is assigned to the adherent group. Separately, members who refill on schedule differ from those who do not in ways claims never record.
Show rather than assert. Rerun the same comparison as a landmark analysis: classify adherence over days 1 to 90, start follow-up at day 91 for everyone still enrolled and event-free, and report how much of the 31 percent survives.
Separate the association claim from the intervention claim out loud. Nothing in the analysis speaks to whether a reminder changes refill behaviour, which is the actual product question and is separately testable.
Offer the shippable path: a randomised holdout on a subset with proportion of days covered as the primary endpoint, powered on the plausible effect of a reminder rather than on the 31 percent, with a stated read date.

Follow-up

The landmark rerun still shows a 20 percent difference. Do you ship?
The product manager says a holdout delays launch. How do you cost that argument?
What primary endpoint do you choose, and why not admissions?