Interview guides · Johnson & Johnson

Johnson & Johnson Data Scientist Interview Questions & Guide 2026

Practice explaining analysis before a Johnson & Johnson data scientist interview: the recommendation, the uncertainty, the follow-ups.

The 60-second brief

This guide covers what a Data Scientist at Johnson & Johnson is expected to do and how to prepare for the interview.

The boundary with engineering varies enough to be worth asking about directly. Some seats end at the analysis and the writeup; others carry feature pipelines, scheduling and alerting, which decides whether production data plumbing belongs in your prep at all.

Focus areas

Risk-adjust outcomes before comparing providers or panels
Build member-month denominators from overlapping coverage spans
Minimise protected health information in every extract

The Johnson & Johnson interview process

Johnson & Johnson candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screening

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Technical Screening

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Onsite Interview

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Ignoring clustering by provider, facility, or site

Patients within a provider panel, and subjects within a trial site, are correlated on both case mix and practice pattern. Standard errors computed as if observations were independent are too narrow by roughly the design effect, 1 + (m - 1) * ICC, where m is the average cluster size. With 50 patients per provider and an intracluster correlation of only 0.02, variance is understated by about a factor of two, which manufactures significant provider differences out of noise. Cluster-robust errors or a random intercept per provider, and cluster-level randomisation when you design the test, are the corrections.

Reading the most recent months of a claims-based series as real

Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.

Crediting a treatment for regression to the mean

Selecting a group because it is extreme (lowest-engagement users, accounts having their worst month, the bottom decile of a score) moves that group's expected next-period value back toward the average even under no treatment, by exactly as much as the selecting measure is imperfectly correlated with its own later value. Compare against units that met the same selection rule and went untreated, or use two pre-periods so the bounce-back is visible before the intervention starts. A pre-post number on a group chosen for being extreme measures the selection rule, not the treatment.

Comparing periods without accounting for seasonality or day-of-week

Compare whole weeks against whole weeks and check whether the same swing appeared in prior cycles or prior years before attributing it to anything you changed. Weekday and weekend populations often differ enough that a Tuesday-to-Saturday comparison is meaningless.

Questions & practice

Explain the underlying mechanics of Bayesian updating and how you apply it to updating clinical trial response priors.

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Say what the estimate is of, and over what population it generalises.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

How do you define success for a machine learning model deployed to assist in early-stage target identification?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.
Set a baseline first, so any model has something honest to beat.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Four data-quality rules over corrected and cancelled lab results

Easy · data quality · record supersession · unit drift

lab_result has result_id, order_id, patient_id, loinc_code, value_numeric, value_text, units, reference_low, reference_high, specimen_collected_ts, resulted_ts, result_status (preliminary, final, corrected, cancelled), supersedes_result_id. Return one row per rule with a failing count, a failing share, and one example result_id, for these rules: more than one surviving row per (order_id, loinc_code) after resolving corrections and dropping cancelled; resulted_ts earlier than specimen_collected_ts; a numeric result whose unit is not the modal unit for its loinc_code; value_numeric and value_text both null on a row with result_status 'final'.

Approach

Resolve the correction chain before anything else. Drop result_status 'cancelled', then within each (order_id, loinc_code) keep the row whose result_id appears in no other row's supersedes_result_id. That terminal node is the surviving value and the rule holds even if a correction is loaded with a backdated timestamp.
Build each rule as a boolean mask over a named frame rather than as a filtered copy, so the failing share has an explicit, per-rule denominator. Rule three is eligible only on rows with a non-null value_numeric; rule four only on final rows.
For the unit rule, derive the modal unit per loinc_code from the data instead of hardcoding an expected unit. The failure being hunted is one analyte reported in two units whose values differ by a fixed factor, which no range check catches because both sets of numbers look plausible.
Emit a tidy frame with columns rule, eligible_rows, failing_rows, failing_share, example_result_id. A printed report cannot be diffed between runs; a frame can be stored and alerted on.
Order the output by failing_share descending so the run has a lead finding rather than four equal-weight lines.

Worked solution (25 min)

Drop cancelled rows, then compute the set of result_ids referenced by supersedes_result_id and keep rows not in that set.
Rule one: group the survivors by (order_id, loinc_code) and count rows above 1.
Rule two: mask on resulted_ts < specimen_collected_ts over rows where both timestamps are present.
Rule three: compute the mode of units per loinc_code over rows with non-null value_numeric, then mask rows whose unit differs from it.
Rule four: mask rows with result_status 'final' and both value columns null. Assemble the four results into one frame with counts, shares and an example id.

Expected result: A four-row frame. After the resolver, rule one should report zero failing groups on a well-formed extract; a non-zero count there means the supersession chain branches or a duplicate load occurred, and that is the finding to lead with rather than a percentage.

Verify it yourself

Running the resolver a second time on its own output is a no-op, and exactly one row survives per (order_id, loinc_code).
No result_id appearing in any supersedes_result_id survives the resolver.
Seed one row with a converted unit for an analyte otherwise reported in a single unit and confirm rule three catches exactly that row and nothing else.
Each rule's failing_share uses its own eligible_rows, so a rule with no eligible rows returns share 0 rather than a division error.

Follow-up

The modal-unit rule fires on 8 percent of one analyte. How do you decide whether that is a unit-conversion bug or a second legitimate assay?
What would you add to catch a value that is inside its reference range but physiologically impossible?
Which of these four rules should block a downstream pipeline and which should only alert, and why?

Explain how you would structure an ETL pipeline query to aggregate daily drug discovery assay results for downstream modeling ingestion.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

How would you handle missing values and duplicate records when joining disparate electronic medical record tables in a data warehouse?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Deduplicate claim versions before totalling allowed amounts

Easy · dedup · window functions · claim versions

medical_claim_line holds one row per service line per claim version: claim_line_id, claim_id, claim_version, frequency_code (1 original, 7 replacement, 8 void), member_id, service_start_date, procedure_code, allowed_amount, claim_status and adjudicated_at. Adjusted claims appear more than once, so summing every row double counts them. Return total allowed_amount by procedure_code for service_start_date in Q1 2025, counting only lines with claim_status 'paid' on the surviving version of each claim, which is the highest claim_version per claim_id, and dropping the claim entirely when that surviving version carries frequency_code 8. Resolve survivorship with a window function, not a self-join.

Approach

Resolve survivorship at claim_id, not claim_line_id. Carry MAX(claim_version) OVER (PARTITION BY claim_id) alongside every line and keep lines where claim_version equals it, because a replacement version can contain a different number of lines than the original.
Read the frequency_code of the surviving version only. A frequency_code 8 removes the claim outright; it does not revert payment to the prior version, so a void must delete the claim rather than promote version n-1.
Apply claim_status and the date filter after survivorship is resolved. Filtering paid lines first can strip the surviving version and silently elect a superseded one.
Group by procedure_code and sum allowed_amount, then state the paid-through date on the output because Q1 amounts keep moving until adjudication runout completes.
Sanity check the shrinkage: report how many claim_ids and how many dollars the dedup removed, so the reviewer can see the step did something.

Worked solution (20 min)

Build a CTE that adds surviving_version = MAX(claim_version) OVER (PARTITION BY claim_id) and surviving_freq = the frequency_code of the row at that version, to every line.
Filter to claim_version = surviving_version so all lines of the surviving version survive together.
Drop claims whose surviving_freq is 8, then apply claim_status = 'paid' and service_start_date between 2025-01-01 and 2025-03-31.
Group by procedure_code, sum allowed_amount, and emit the count of distinct claim_ids contributing.
Run the naive SUM over all rows beside it and report the difference as the double count that was removed.

Expected result: One row per procedure_code with summed allowed_amount over surviving, non-voided, paid Q1 2025 lines. A claim with an original and one replacement contributes only the replacement's lines; a claim whose surviving version is a void contributes nothing.

Verify it yourself

Deduped total is strictly below the naive paid-and-Q1 SUM taken across all versions whenever at least one claim carries paid Q1 lines on more than one version. If the two totals match on such a claim, the dedup did nothing.
For one claim_id known to have three versions, the line count it contributes equals the number of its paid Q1 2025 lines at its highest claim_version, not 1.
Let B be the set of claim_ids whose surviving version has at least one paid line with a Q1 2025 service_start_date. COUNT(DISTINCT claim_id) in the output equals the size of B minus the number of claims in B whose surviving version carries frequency_code 8. Comparing instead against COUNT(DISTINCT claim_id) over the whole raw table is not a void test: that gap also absorbs every claim dropped by claim_status or by service date.

Follow-up

Two rows share a claim_id and claim_version but differ on adjudicated_at. What do you do, and what does that imply about the extract?
How does the query change if the request is allowed_amount by paid_date rather than service_start_date, and which basis does a finance reconciliation want?
A void arrives for a claim already published in a closed month. How do you restate without rewriting history in the warehouse?

If we launch a new digital health companion feature, what core metrics will tell us if it is improving patient retention versus adding noise?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Imagine a key clinical engagement metric drops by fifteen percent week-over-week. Walk me through your diagnostic workflow to isolate the root cause.

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you design a product metric framework to measure the adoption and clinical utility of a new real-world evidence dashboard?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you balance competing product metrics when optimizing a predictive sales forecasting tool used by global finance leadership?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

What steps do you take when an experiment shows statistically significant results that contradict domain intuition?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Say whether units interfere with each other, and switch design if they do.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

How do you determine sample size and power when dealing with rare disease patient cohorts with inherently low sample volumes?

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
Say whether units interfere with each other, and switch design if they do.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

Reduce variance on a skewed cost outcome with CUPED

Medium · cuped · variance reduction · stratification · skewed outcomes

A member-randomised digital care programme will be read on 12-month allowed PMPM, built from surviving medical_claim_line versions plus unreversed pharmacy_claim rows over member-months from member_enrollment. The outcome is heavily right-skewed. Eighty percent of members have 12 months of prior continuous coverage and therefore a usable pre-period PMPM; among those, the correlation between pre-period and post-period member PMPM is 0.45. Propose a variance reduction plan, quantify the expected gain, and say how you handle the 20 percent with no pre-period.

Approach

Apply CUPED with the pre-period PMPM as the covariate: theta = Cov(Y, X) / Var(X), and Y_adj = Y - theta x (X - mean(X)). Because X is measured strictly before randomisation it cannot be affected by treatment, so the adjusted difference in means stays unbiased while its variance falls by the factor 1 - rho^2.
Quantify honestly: rho = 0.45 gives 1 - 0.2025 = 0.80, a 20 percent variance reduction inside the covered stratum, which is a real but modest gain. Do not quote consumer-experimentation figures of 40 to 50 percent, which come from outcomes far more autocorrelated than health spend.
Handle the skew before handling the covariate. Pre-specify a truncation point (a percentile of the prior year, or a fixed stop-loss amount) applied identically to both arms and fixed before unblinding, and state plainly that you are now estimating an effect on truncated cost. Report the untruncated result as a secondary so nobody can claim the cap created the finding.
Do not impute a mean pre-period for the 20 percent with none. Stratify: run CUPED inside the covered stratum, take the unadjusted difference in the no-pre-period stratum, and combine with fixed weights; equivalently, regress with a missingness indicator and an indicator-by-covariate interaction. Both keep the estimator unbiased because the stratum is defined pre-randomisation.
Get some of the same gain for free at assignment time by stratifying or covariate-constrained randomisation on prior-year cost decile, which removes the post-hoc analytic choices that a sceptical reviewer will otherwise interrogate.

Worked solution (30 min)

Estimate theta on the full randomised sample as Cov(Y, X) / Var(X), using both arms because X is pre-treatment.
Form Y_adj = Y - theta x (X - mean(X)) inside the covered stratum and compute the variance ratio 1 - rho^2 = 1 - 0.45^2 = 0.80.
Combine strata: assuming equal per-member outcome variance, the overall variance of the treatment-effect estimate falls by about 0.8 x 20 percent = 16 percent.
Translate to MDE: sqrt(0.84) = 0.916, so the MDE falls about 8 percent; equivalently you save about 16 percent of the sample at fixed MDE.
Validate on a historical A/A split: CUPED should return an effect indistinguishable from zero with the claimed variance reduction.

Expected result: theta = Cov(Y, X) / Var(X); variance falls to 0.80 of baseline inside the 80 percent with a pre-period, about 16 percent overall under equal per-member variance, which shrinks the MDE by roughly 8 percent. The no-pre-period members enter as their own stratum, never as imputed means.

Verify it yourself

theta should be close to rho x sd(Y) / sd(X); a large gap points at an outlier driving the covariance.
The CUPED and unadjusted treatment-effect point estimates should be close; a large divergence means X is not pre-treatment or the cap is doing the work.
Re-estimate rho after truncation, since capping both tails usually moves it.
Run the whole procedure on a historical A/A period and confirm a null effect with narrower intervals.

Follow-up

rho drops to 0.20 after truncation. What do you do with the CUPED plan?
Members with no pre-period are new enrollees. What does that do to your assumption of equal per-member variance across strata?
Would you switch the estimand from mean PMPM to a rank-based or log-scale comparison, and what would that cost you with a finance audience?

Allowed PMPM fell fourteen percent in three months

Easy · claims runout · completion factors · incurred vs paid

Allowed PMPM in the monthly series fell from $412 to $354 over the three most recent incurred months, and leadership wants to announce the saving. You have medical_claim_line (allowed_amount, service_start_date, received_date, paid_date, claim_id, claim_version, frequency_code, place_of_service_code), pharmacy_claim (allowed_amount, fill_date, reversal_flag) and member_enrollment coverage spans. The chart carries no paid-through date. Work out how much of the fall survives completion, and hand back a restated series with an explicit paid-through date and the completion factors you applied.

Approach

Get the paid-through date first and put it on the chart. Without it the series is uninterpretable, because every incurred month is a different age.
Build a lag triangle: for each incurred month, cumulative allowed_amount by the number of months between service_start_date and paid_date, on surviving claim versions only (drop frequency_code 8 voids and keep the highest claim_version per claim_id).
Estimate development factors separately for pharmacy fills, professional lines and inpatient facility lines. Pharmacy adjudicates within days, professional within weeks, inpatient facility over months, so one blended factor understates the correction on the newest month and overstates it on the oldest.
Gross each of the last three incurred months up by the reciprocal of its cumulative development factor, using factors fitted on months that have already fully run out.
Rebuild the denominator independently as member-months from coverage spans. Member-months are complete on day one, so if the denominator also shows a lag pattern the enrollment file is late, which is a different bug with a different fix.
Restate the series, mark the last three months as estimated, and report the residual movement after completion as the only part worth investigating.

Follow-up

The completed series still shows a three percent fall. What do you look at next, and in what order?
How would you detect that the fall is a mix shift toward cheaper services rather than lower volume?
A contract settles on this number at a fixed paid-through date. What do you owe the other party about the estimate you just made?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Rigour against a deadline

Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.

Tell me about a time you had to push back on a stakeholder who wanted to deploy a machine learning model that you knew was not ready for production.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Make an honest case for your own analytical impact

Hard · impact attribution · counterfactual · self-assessment

You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.

Approach

The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.

Follow-up

Two other people also claim the contract measure. How do you split it?
Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
What is the strongest argument that your year had less impact than you just described?

Defend a null result against a programme sponsor

Medium · regression to the mean · difference-in-differences · stakeholder management

A care management programme enrolled the top 1 percent of members by prior-year allowed spend. The sponsor's deck shows allowed PMPM for enrollees falling 34 percent from the year before enrollment to the year after, and asks for budget to triple the programme. You rebuild the evaluation with a concurrent comparison group selected by the identical spend rule in the same period. The difference-in-differences estimate is a 3 percent reduction with a confidence interval spanning zero. Deliverable: how you present this, to whom, in what order, and what you propose next.

Approach

Name the probe: whether you can deliver a finding that costs somebody their programme without softening it into uselessness or creating an adversary who routes around you next time.
Lead with the mechanism, not the verdict. Show the comparison group's own unadjusted drop, which will be large, because a cohort selected on an extreme of the outcome regresses toward the mean whether or not anyone intervenes. The sponsor's 34 percent is mostly that, and it is a property of the selection rule rather than a criticism of their clinicians.
Give the sponsor the finding privately before it appears in any deck their leadership sees. Being surprised in a room is what turns a methods disagreement into a political one.
State the estimate with its interval and say what it rules out as well as what it fails to establish. A 3 percent point estimate whose interval crosses zero is not evidence of no effect; it is insufficient power to separate a modest effect from none, and those are different claims.
Arrive with a design rather than only an objection. Propose a regression discontinuity at the enrollment threshold if the rule is applied sharply, or a randomised rollout across the next wave of eligible members, and state the sample size needed to detect the effect size the sponsor believes in.

Follow-up

The sponsor says withholding the programme from a comparison group is unethical. What do you propose instead?
Leadership wants one number for the board next week. What do you give them?
What result would change your mind and make you believe the 34 percent?