Interview guides · Dana-Farber Cancer Institute

Dana-Farber Cancer Institute Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Dana-Farber Cancer Institute: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

As a Data Scientist at Dana-Farber Cancer Institute, you play a vital role at the intersection of advanced computational research, clinical trials, and breakthrough cancer care. Operating within world-class departments dedicated to transforming data into life-saving insights, you collaborate directly with basic biologists, clinicians, and principal investigators. Your work directly enables teams to parse complex genomic datasets, optimize treatment pipelines, and accelerate discoveries that reduce the burden of cancer globally.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

Separate censoring from events in time-to-event models
Correct for claims runout before reporting recent months
Reconstruct drug exposure intervals from dispensing records

Overview

The problems you tackle are both intellectually demanding and deeply meaningful. You might build scalable statistical pipelines in R or Python, evaluate emerging machine learning methodologies for cancer classification, or design robust analytical frameworks for clinical research. Because Dana-Farber Cancer Institute fosters an exceptionally collaborative environment, you serve as both a technical expert and a cross-functional consultant, translating ambiguous biological hypotheses into rigorous, reproducible data science solutions.

This role requires a rare blend of scientific rigor, engineering capability, and product-sense thinking. You will manage diverse datasets, benchmark new analysis tools, and contribute to long-term computational architecture that empowers researchers across the institute. Expect an environment where your analytical output directly shapes clinical understanding and where rigorous data methodology is paramount to driving public health impact.

The Dana-Farber Cancer Institute interview process

Dana-Farber Cancer Institute candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Phone Screen

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Screen

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Take-Home Data Challenge

Before any modelling, the dataset is itself the first test. Take-home data usually carries something broken: rows duplicated at an unexpected grain, a join that silently drops part of the population, timestamps stored in more than one timezone, or missingness correlated with the outcome. An hour spent profiling row counts, key uniqueness and date ranges is not overhead, because it decides whether every number after it is real. What separates submissions is whether you report the defects you found and adapt the analysis to them, rather than modelling over them quietly and hoping the aggregate absorbs it.

What to demonstrate

Whether you established the grain of each table and checked row counts after every join, and said so in the writeup
Whether data defects you found are surfaced with their effect on the conclusion, instead of being dropped without comment
Whether filters and exclusions are reproducible from the submitted code, with the size of the excluded population quantified

How to prepare

Write a short profiling script you can point at any unfamiliar table: row count, distinct key count, null rate per column, and the min and max of every date field, then run it before anything else
Write the funnel or the join chain as one query and check the row count at each grain, so a silent fan-out shows up as a number rather than as a wrong answer later
On a past dataset, list every exclusion you applied and how many rows each one removed, then draft the single sentence about it you would put in a report

Final Panel Presentation

Where a loop ends with a senior leader, that conversation is rarely another skills test. The technical signal already exists by then, so the questions tend to open up: what you would look at first, where a metric you have heard about could mislead, what you would push back on. The decision being made is scope, which in practice means level and how much you would be trusted to own unsupervised. Treating it as a formality is the usual mistake. An open question late in the day is still being scored, and a vague answer reads as someone who has not run anything themselves.

What to demonstrate

Whether your view of the business has anything specific behind it, given that you are working only from what is public and are expected to say so
Whether the scope of work you describe owning matches the scope of the role, instead of sitting a level below it
Whether you can disagree with something concrete and stay useful about it, rather than agreeing with everything said in the room
Whether your questions are ones only this person could answer, as opposed to ones the recruiter already covered

How to prepare

Build one view you could defend for two minutes using only public information: what the funnel probably looks like, which metric likely drives decisions, and where that metric could mislead. Being wrong for a stated reason survives this round; having no view does not
Write down the largest piece of work you have owned from question to decision, who else touched it, and what you decided alone, then check that it reads at the level you are interviewing for
Prepare one thing you would want changed if you joined and phrase it as a question rather than a verdict, so it opens a conversation instead of closing one

Technical and Behavioral Interviews

Rounds of this kind usually include one question about work that did not go well, and it is the part that carries the most information. Anyone can narrate a shipped win. What the interviewer learns from a project that stalled is how you behave without a result to hide behind: whether you noticed the problem yourself, how long it took, and who you told. Answers that route the failure onto a data pipeline or a reorganisation close the topic without answering it, and the follow-up comes back to your own part.

What to demonstrate

Whether you found the error yourself or someone else found it, and how long it sat before anyone knew
What you changed afterwards, stated as a check you now run rather than a lesson you now believe
Whether the mistake you choose has real cost attached, such as a quarter of misdirected roadmap or a metric that was reported upward, instead of one that flatters you

How to prepare

Choose a failure you caught yourself and be ready to say what tipped you off. A story where someone else caught it is still usable, but you will be asked why you missed it.
Write down the check you added afterwards and where it lives now, so the correction is a concrete artefact rather than a resolution.
Rehearse saying the cost out loud. Candidates shrink the number by instinct once the interviewer is in the room.

Where candidates lose points

Rates built on member counts rather than exposure

Members join and leave mid-period, so dividing events by distinct members mixes a person covered for 30 days with one covered for 365. New joiners also have artificially low observed utilisation because their claims have not arrived yet and because care takes time to initiate. Denominators must be member-months or member-years, and comparative quality measures usually need a continuous-enrolment requirement with an explicit allowable gap, stated in days.

Pre-post evaluation on a high-cost or high-risk cohort

Cohorts selected on an extreme value of the outcome regress toward the mean on their own. Members identified as the top 1 percent of spend in one year spend far less the next, whether or not anyone intervenes, because the selecting year captured both chronic severity and one-off events. A pre-post design on such a cohort will report savings every time. A concurrent comparison group selected by the same rule in the same period, or a regression discontinuity at the selection threshold, is the minimum credible design.

Sizing estimates built on unnamed, unrevisable assumptions

Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.

Reading experiment results before checking the arm split

Compare observed arm counts against the intended allocation ratio, not an assumed even split, and set the alarm far below the conventional 0.05: at 0.05 roughly one healthy experiment in twenty trips it, which is why sample-ratio checks usually run at p < 0.001 or stricter. The test's power scales with sample size, so it misses a real diversion on a small experiment and fires on an imbalance too small to move the estimate on a very large one. A flag means go find the assignment or logging fault before reading any outcome, not report a mismatch.

Questions & practice

How do you decide between frequentist and Bayesian methods when analyzing a dataset with a small sample size?

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Translate the result into the decision it informs, in one plain sentence.
Write down the assumption the method needs before you use the method.

Follow-up

How would you explain this result to someone who does not know statistics?
Which assumption here is most likely to be violated in practice?

How do you prevent data leakage when engineering features for a predictive model built on longitudinal patient data?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Set a baseline first, so any model has something honest to beat.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Proportion of days covered with shifted, truncated refill intervals

Hard · interval merging · adherence · path dependence

pharmacy_claim has fill_id, member_id, therapeutic_class_code, fill_date, days_supply, reversal_flag, reversed_fill_id. For one therapeutic class and a fixed 12-month window, compute proportion of days covered per member: distinct days on which a dispensed days_supply covers the day, divided by days from the member's first in-window fill through the window end. An early refill shifts coverage forward rather than stacking, and coverage is truncated at the window end. Drop both rows of every reversed pair. Return member_id, pdc, and the share at pdc >= 0.80 among members with at least 2 fills and at least 91 days of follow-up.

Approach

Remove reversals as pairs first. Drop every row with reversal_flag true, and also drop the fill_ids those rows point at through reversed_fill_id. Dropping only the flagged row leaves a dispense that was never collected in the exposure.
Walk fills per member in fill_date order carrying a cursor: start = max(fill_date, previous_end + 1 day), end = start + days_supply - 1. The shift is path dependent, so a plain cumsum over days_supply does not reproduce it. Use itertools.accumulate or a per-member loop over numpy arrays, not a row-wise apply over the whole frame.
Truncate the last interval at the window end before measuring. Without truncation a 90-day fill dispensed on the final day pushes the covered-day count past the denominator and PDC above 1.0.
Use the denominator the definition states: first in-window fill_date through window end, inclusive. Not a flat 365, and not first fill to last fill, which is a different metric that rewards early discontinuation.
Apply the eligibility filter before computing the >= 0.80 share, and report the size of that denominator next to the share. A share without its denominator is not reviewable.

Worked solution (40 min)

Filter to the therapeutic class and the window, then remove reversed pairs by dropping flagged rows and the fill_ids in reversed_fill_id.
Sort by member_id, fill_date. Per member, accumulate start = max(fill_date, prior_end + 1 day) and end = start + days_supply - 1.
Clip every interval's end at the window end and drop intervals whose start is past it.
Covered days per member = sum of (end - start + 1) over the shifted intervals, which are disjoint by construction.
Denominator = (window_end - first_fill_date).days + 1. Divide, then filter to members with >= 2 fills and denominator >= 91 and compute the share at or above 0.80.

Expected result: Per-member PDC in [0, 1], plus one share. Worked case: fills at day 0 (30 days), day 20 (30 days) and day 55 (30 days) in a 180-day denominator give intervals 0-29, 30-59 and 60-89, so 90 covered days and PDC = 0.50. Change the window to 60 days with fills at days 0, 20 and 40 and the shifted intervals truncate to 0-59, giving PDC = 1.00 while a days-supply sum would report 90/60 = 1.50.

Verify it yourself

No member has PDC above 1.0. Any value above 1.0 means intervals stacked instead of shifting, or the final interval was not truncated.
A member with a single 30-day fill at the start of a 180-day denominator scores 30/180 = 0.167 and is excluded from the share by the two-fill rule.
For one member, rebuild covered days as the size of a Python set of date ordinals over the raw fills and confirm it matches the interval arithmetic.
Shifting every fill_date forward by the same number of days changes each denominator but leaves the covered-day counts unchanged except at the truncation boundary.

Follow-up

How does PDC differ from medication possession ratio, and which of the two can exceed 1.0?
A member switches to a different ingredient inside the same therapeutic class mid-window. Should the intervals chain, and what does that do to the class-level number?
Members who die or lose coverage mid-window get a short denominator and often a high PDC. If you then compare mortality by adherence category, what bias have you built in and how do you remove it?

Write a SQL query using window functions to calculate the rolling three-month average of patient enrollment across multiple clinical trial sites.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Given a table of experimental runs, write a query to rank performance scores within each department using dense ranking functions.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Compute allowed PMPM by incurred month without span fan-out

Hard · fan-out · many-to-many · runout

Report allowed PMPM by incurred month and product_type for 2025 from medical_claim_line (member_id, claim_id, claim_version, frequency_code, service_start_date, allowed_amount, claim_status), pharmacy_claim (member_id, fill_date, allowed_amount, reversal_flag, reversed_fill_id), member_enrollment (span_id, member_id, product_type, effective_date, termination_date) and member_risk_score (member_id, score_year, prospective_risk_score), which holds at most one row per member per score year. The numerator is allowed_amount on surviving claim versions plus unreversed fills, attributed to the product_type in force on the service or fill date. The denominator is member-months. Also return a risk-adjusted PMPM for each cell: allowed PMPM divided by the member-month-weighted mean prospective_risk_score for score_year 2025 over the members making up that cell's denominator. A colleague joined claim lines to member_enrollment on member_id alone and the annual total came back 1.4 times the finance figure. Diagnose that, then produce the correct monthly series.

Approach

Name the defect precisely. member_enrollment is one row per span, so joining on member_id alone is one-to-many and duplicates every claim line once per span that member ever held. The inflation factor is the spend-weighted mean span count per member, which is why 1.4 is plausible and why it is not a constant you can divide back out.
Make the join functional: member_id AND service date between effective_date and COALESCE(termination_date, DATE '9999-12-31'). That yields at most one span per line unless spans genuinely overlap, in which case pick a documented precedence rule, most recently effective wins being the usual one, and prove uniqueness with a duplicate check rather than asserting it.
Aggregate numerator and denominator separately and join them on (month, product_type). Computing both sides across one join is the second route to inflation, because the member-month denominator fans out over claim lines just as easily as the dollars fan out over spans.
Clean each numerator source on its own terms: surviving claim versions with voids removed and claim_status 'paid' on the medical side, reversal rows and their originals removed on the pharmacy side.
Apply completion before anyone reads a trend. State the paid-through date and either withhold the three most recent incurred months or apply completion factors, since the lag differs by service type and the incomplete tail reads as a cost improvement and a utilisation drop at once.
Risk-adjust last, and weight the score the way the denominator is weighted. The divisor for a cell is SUM(member_months * prospective_risk_score) / SUM(member_months) over that cell's members, not a plain AVG over distinct members: a member covered two months must not count the same as one covered twelve. member_risk_score is member-level, so join it after the member-month grain is fixed, or it becomes a second fan-out.
State the rule for members with no 2025 member_risk_score row: keep them in member_months, leave them out of the weighted mean, and publish the scored share of member-months beside the adjusted series. A thin score table moves the adjusted number while the raw one sits still, and without that share nobody can tell the two apart.

Worked solution (45 min)

CTE med: surviving claim version per claim_id, voids dropped, claim_status 'paid', service_start_date in 2025.
CTE rx: fills with reversal_flag FALSE and not referenced by any reversed_fill_id, via NOT EXISTS, fill_date in 2025.
Attribute each numerator row to one span with the date-bounded join, after asserting that row counts before and after the join are equal.
CTE denom: member-months by month, product_type and member_id from spans alone, with fractional partial months and overlapping spans collapsed first.
Join numerator to denominator aggregated on (incurred_month, product_type) and divide for allowed_pmpm.
Join denom to member_risk_score on member_id for score_year 2025, compute the cell divisor as SUM(member_months * prospective_risk_score) / SUM(member_months) over scored members, carry the scored share of member-months, and divide allowed_pmpm by the divisor for risk_adjusted_pmpm.
Withhold the three most recent incurred months and label the paid-through date on every row.

Expected result: One row per (incurred_month, product_type) with allowed_medical, allowed_pharmacy, member_months, allowed_pmpm, mean_risk_score, scored_member_month_share and risk_adjusted_pmpm, covering January through September when the paid-through date is year end and three months are withheld. Row count of the claim-line set is identical before and after the enrollment join.

Verify it yourself

Claim-line row count before and after the span join is unchanged. An increase is fan-out; a decrease is a service date with no covering span, which is a real finding about eligibility gaps.
SUM(allowed_amount) across product_types for a month equals the ungrouped monthly total computed without touching member_enrollment at all.
member_months for a month never exceeds the count of distinct members covered at any point in that month.
mean_risk_score for a cell lies between the minimum and maximum prospective_risk_score among its scored members, and risk_adjusted_pmpm equals allowed_pmpm exactly when that mean is 1.0. If the weighted mean equals a plain AVG over distinct scored members in every cell, the member-month weights were never applied.
A downward step in the last reported months means the runout holdback was not applied; the withheld tail should not be visible in the series at all.

Follow-up

A member holds two overlapping spans under different product_types on the service date. Which one gets the dollar, and how do you keep the member-month denominator consistent with that choice?
Write the check that proves the span join is one-to-one rather than assuming it.
Finance reports on paid_date and you reported on service_start_date. Reconcile one month between the two bases.

Imagine we are launching an interactive web tool for researchers. What user-centric metrics should we track to evaluate its adoption and utility?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you design a core metric to measure the success of a newly implemented clinical data pipeline?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

A key research dashboard is showing an unexpected drop in user engagement among clinical collaborators. How would you diagnose the root cause?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Describe a project where you faced shifting priorities or ambiguous requirements. How did you realign your team?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

What are some common experimentation pitfalls you watch out for when running concurrent tests on overlapping user cohorts?

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
Name the guardrails that would stop a launch even on a positive primary result.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

Walk through how you would design an A/B test for a new web-based user interface designed for clinical researchers.

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Decide the analysis before seeing data, including how long it runs and when you look.
Say whether units interfere with each other, and switch design if they do.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

Score trial sites without rewarding enrollment that does not stick

Medium · site scorecards · shrinkage · guardrails · gaming

A clinical operations group ranks sites on subjects randomized per activated site-month, using study_subject_visit (study_id, site_id, subject_id, arm_code, randomization_ts, site_activation_date, visit_name, visit_number, visit_status, primary_endpoint_value, protocol_deviation_flag, deviation_severity). Bottom-ranked sites lose allocation on the next study. Define the primary metric, two guardrails, the handling of sites activated too recently to rank, and the estimator you rank on. Then state what a site under allocation pressure does to the number, and which guardrail sees it.

Approach

Move the numerator from randomized to randomized and retained through the protocol-defined primary endpoint visit. A subject who randomizes and withdraws consumes screening, drug supply and monitoring while contributing nothing to the analysis population, so counting them rewards the wrong behaviour precisely when a site is under pressure.
Build the denominator as exposure, not sites: site-months from site_activation_date to the earlier of period end and database lock, per study. Ranking on a raw count instead of a rate confuses a large site with a fast one.
Handle young sites by reporting rather than dropping. Sites with fewer than a stated minimum exposure are shown in an insufficient-exposure band, because excluding them is how a non-enrolling tail disappears from a scorecard, and the non-enrolling tail is usually the larger operational problem.
Rank on a shrunken estimate, not the raw ratio. With a few site-months of exposure the ratio has enormous variance, and a site with two site-months and one randomization will top an unshrunken table. Partial pooling toward the study mean, or a simple empirical-Bayes posterior mean, prevents rank being an artefact of small exposure.
Pick guardrails that read the two pressure responses. A site that loosens eligibility judgement shows an implausibly low screen-failure rate, computed as subjects with randomization_ts NULL and visit_status screen_failed over subjects screened. A site that pushes visits to hit counts shows up in major protocol deviations per 100 completed visits, reported separately from minor ones because a single major deviation can remove a subject from the per-protocol set.
Report the companion the median hides: the share of activated sites with no randomization by day 90. A median computed only over sites that eventually randomized is silent about the sites that never did.

Worked solution (30 min)

Define the numerator as subjects with non-null randomization_ts whose primary endpoint visit reaches visit_status completed with a non-null primary_endpoint_value, per study and site.
Compute site-months of exposure from site_activation_date to period end or database lock, and set the minimum exposure below which a site is banded rather than ranked.
Compute the raw rate, then the shrunken estimate by partial pooling toward the study mean, and print both columns side by side so the shrinkage is visible rather than hidden.
Specify the two guardrails with numerator, denominator and window, keeping major and minor deviations as separate series.
Add the no-randomization-by-day-90 share across all activated sites, so the non-enrolling tail appears on the same page as the ranking.

Expected result: A scorecard whose primary metric is randomized-and-retained subjects per activated site-month per study, ranked on a shrunken estimate with the raw rate shown beside it; sites below a stated minimum exposure reported in an insufficient-exposure band rather than excluded; guardrails of screen-failure rate and major protocol deviations per 100 completed visits reported separately from minor ones; and a companion share of activated sites with no randomization by day 90.

Verify it yourself

Compare raw and shrunken rankings. The largest rank movements should be concentrated among the lowest-exposure sites; if a high-exposure site moves far, the pooling is misconfigured.
Confirm the denominator's site-months sum matches an independent count of months between activation and period end across all sites.
Check that screen failures are being counted at all: a site reporting a zero screen-failure rate across dozens of screened subjects is a data-capture problem or an eligibility-judgement problem, and both need an answer.
Verify that excluding the insufficient-exposure band does not change the study-level mean materially; if it does, those sites carry more of the study than the banding implies.

Follow-up

A site has the best randomized-and-retained rate and the worst major deviation rate. What do you recommend, and what would change your answer?
Your shrunken ranking moves a site from 3rd to 14th. How do you explain shrinkage to the site's operations lead in two sentences?
Primary endpoint retention takes 18 months to observe. What do you rank on in the meantime, and what bias does that interim choice introduce?

Readmission rate rose while every facility improved

Medium · simpsons paradox · mix shift · risk adjustment

Quarter over quarter the 30-day unplanned readmission rate for an attributed population rose from 14.2 to 15.1 percent, yet all eleven facilities in encounter show a flat or falling rate over the same period. You have encounter (facility_id, admit_ts, discharge_ts, encounter_type, admission_type, discharge_disposition, drg_code, is_planned_admission, index_encounter_id) and twelve months of prior medical_claim_line for comorbidities. Explain the direction reversal numerically, and state what the population-level number should be reported as instead.

Approach

Confirm the reversal is arithmetic, not a definition difference, by recomputing both the facility rates and the pooled rate from the same query with the same index-stay exclusions.
Decompose the pooled change exactly with a Kitagawa split. With facility weights w and within-facility rates r, the change equals the sum over facilities of (w_1 - w_0) times (r_0 + r_1)/2, the mix term, plus the sum of (w_0 + w_1)/2 times (r_1 - r_0), the within term. If the mix term exceeds the total change and the within term is negative, the reversal is volume shifting toward facilities with higher baseline rates.
Ask why the weights moved. Check index-stay volume by facility, and check whether an exclusion changed behaviour: acute-to-acute transfers (discharge_disposition 'transfer_acute') must not open an index stay, and if one facility began transferring more, its denominator shrinks toward its sickest retained patients.
Check case mix inside the facilities too, using drg_code and comorbidities drawn from the 12 months of claims before admit_ts, excluding complications of the index stay itself. A facility rate can be flat while its case mix worsens.
Replace the raw pooled rate with an observed over expected ratio, where expected is the sum of predicted probabilities from a model fitted on age, sex and prior comorbidities. Raw rates are not comparable across a population whose facility and case mix is moving.

Follow-up

Your O/E ratio is 1.04 with a 95 percent interval of 0.96 to 1.13. What does that let you say to a facility, and what does it not?
If one facility contributes 40 percent of index stays, how does that change how you present the decomposition?
How would you handle a member with two index stays in the quarter, where the second is also a readmission from the first?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

How do you approach mentoring junior analysts or educating collaborators on best programming practices?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Own an analysis that shipped wrong and was acted on

Easy · retrospective · ownership · data quality

Describe an analysis you delivered that was wrong, where someone acted on it before the error surfaced. Pick something with a mechanism you can name, not a typo. Cover what the number claimed, what was actually true, how much time passed before it was caught, who caught it, and what you told the people who had already acted. Deliverable: a five-minute account that ends with the specific control you added afterwards, plus one occasion since when that control has fired.

Approach

Read the probe: the interviewer is testing whether you detect your own errors and whether the correction was structural. Pick an error with a nameable mechanism (a denominator built from distinct members instead of member-months, a series read before claims runout completed, a cohort selected on the outcome) so the story has a diagnosis rather than a mood.
Open with the decision that was made on your number, not with the bug. The severity of an analytic error is measured in the action it caused: a rate filing, a staffing plan, a programme expansion.
State the counterfactual number. 'The true figure was 14.1 admissions per 1,000 member-years, not 9.8' is a fact; 'it was off by a lot' is not.
Describe the detection path honestly, including whether you found it or somebody else did. Say how long it took, because time to detection is the part you actually control next time.
Close on the control, and make it something that protects a reader who has never met you: a unit test on the denominator, a paid-through date stamped on every chart, a required exclusions block in the spec template. Name one later date it caught something.

Follow-up

What would have had to be true for you to catch this before delivery rather than after?
Did the stakeholder change how much they trust your work, and did you want them to?
Give an example of a control you added that turned out to be the wrong control.

Explain an incomplete cost series to a non-technical executive

Medium · claims runout · uncertainty · executive communication

An executive opens a dashboard showing risk-adjusted allowed PMPM down 6 percent across the last three incurred months and asks whether the cost initiative is working. You know pharmacy adjudicates within days, professional claims within weeks, and inpatient facility claims over months, so those three months are both undercounted and mix-shifted toward cheap services. There is also a genuine movement of roughly 1.5 percent in the older, complete months. Deliverable: a two-minute spoken answer, the change you make to the chart, and what you commit to telling them and when.

Approach

The probe is whether you can be honest about uncertainty without sounding evasive. Executives read hedging as not knowing, so the answer has to end in a commitment.
Separate the two claims immediately. The 6 percent is an artefact of incomplete data, and there is a smaller real movement in the months that are complete. Delivering only the debunk leaves them holding nothing.
Explain the lag in their terms: the bills for the most expensive care arrive last, so an incomplete month always looks cheap and always looks like it is improving. Keep the phrase completion factor out of the first pass.
Fix the chart instead of explaining around it. Shade or omit the incomplete months, print the paid-through date on the axis, and overlay the same series as it looked at equivalent maturity a year earlier so the shape is comparable rather than merely lower.
Close with a date and a magnitude. Commit to a restated figure once runout matures, say roughly how far you expect the 6 percent to shrink, and name what you would need to see to call the initiative working.

Follow-up

They ask for your best guess today, knowing it is provisional. What do you say?
The restated number comes back at 1 percent. How do you handle having flagged the 6 percent at all?
How do you stop this dashboard producing the same conversation next quarter?