Interview guides · IQVIA

IQVIA Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for IQVIA: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

As a Data Scientist at IQVIA, you sit at the critical intersection of advanced analytics, healthcare, and human life. Your work is fundamental to the IQVIA mission: to provide solutions that enable healthcare companies to innovate with confidence, maximize opportunities, and ultimately drive healthcare outcomes forward. You are not just building models; you are transforming massive, complex, and often sensitive clinical and real-world data into actionable insights that shape global health strategies.

Product-sense cases reward reasoning from a mechanism to a testable prediction. Reciting every metric you can name reads as pattern matching; naming the single quantity that would move if your explanation were true reads as thinking.

Focus areas

Reconstruct drug exposure intervals from dispensing records
Risk-adjust outcomes before comparing providers or panels
Minimise protected health information in every extract

Overview

The role is inherently multidisciplinary, requiring you to bridge the gap between technical rigor and clinical relevance. You will likely contribute to projects involving predictive modeling, clinical trial optimization, or patient journey analysis. Because IQVIA operates at a massive scale, your ability to handle high-dimensional data and translate complex statistical results into clear, business-driven narratives is what defines your success and impact within the organization.

The IQVIA interview process

IQVIA candidates report 6 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Automated Assessment

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Technical Discussions

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Project Deep Dive

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Interviews with Team

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Final Decision

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Treating clinical measurements as missing at random

A lab result, a vital sign, or a screening exists because someone ordered it, and ordering tracks suspicion of disease, visit frequency, and site workflow. Imputing the mean or dropping incomplete rows biases the population estimate and can flip the sign of an association, because the untested are systematically healthier or systematically disengaged. The presence indicator is often more predictive than the value, which is a warning sign rather than a feature win: a model that learns test ordering will not transfer to a site with different protocols.

Immortal time in adherence, treatment, and enrolment definitions

Classifying members as adherent, treated, or programme-enrolled requires them to survive and stay covered long enough to accumulate the defining events. That guaranteed event-free interval is assigned to the exposed group, so the exposure looks protective for reasons that have nothing to do with the treatment. Adherence studies are the classic case: measuring 12-month proportion of days covered and then comparing mortality builds survival into the exposure definition. Use time-varying exposure or a landmark analysis with the classification window excluded from follow-up.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Averaging per-user rates to produce a population rate

Decide which quantity you want: the mean of per-user ratios and the ratio of summed numerator to summed denominator are different estimands, and heavy users dominate one but not the other. For a ratio metric, aggregate numerator and denominator separately and use the delta method for its variance.

Questions & practice

How would you approach feature selection when working with high-dimensional genomic or patient data?

Medium · statistics and probability

Approach

Translate the result into the decision it informs, in one plain sentence.
Write down the assumption the method needs before you use the method.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

Can you explain the difference between correlation and causation in the context of observational studies?

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Translate the result into the decision it informs, in one plain sentence.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

What are the assumptions of a linear regression model, and how do you validate them?

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Describe your process for debugging a model that is producing unexpected results.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Bootstrap a confidence interval for a ratio-of-sums PMPM

Medium · bootstrap · ratio estimator · resampling unit

You have one row per member for a fully run-out incurred year: member_id, member_months, allowed_amount. Report allowed PMPM, defined as sum(allowed_amount) divided by sum(member_months), with a 95 percent confidence interval from a bootstrap you write yourself. No library resampler. Resample members with replacement, recompute the ratio on each of 2,000 replicates, take a percentile interval, and seed the generator so the result reproduces. In two sentences, state why resampling claim lines instead of members would be wrong.

Approach

Name the estimator precisely: PMPM is a ratio of sums, not the mean of per-member ratios. The two differ whenever exposure correlates with spend, and it does, because members who churn out mid-year are systematically different from members covered all twelve months.
Resample the unit of independence, which is the member, and recompute numerator and denominator together on the same draw. Recomputing only the numerator against a fixed denominator understates variance and is a common slip.
Pull member_months and allowed_amount into numpy arrays once, then draw indices with rng.integers(0, n, size=n) per replicate. Index draws over arrays run one to two orders of magnitude faster than DataFrame.sample, which is the difference between 2,000 replicates and 200.
Take the 2.5th and 97.5th percentiles of the replicate ratios. Expect an interval that is asymmetric around the point estimate, because healthcare spend is right skewed. That asymmetry is the method working, not a bug to centre away.
Answer the line-resampling question concretely: the many lines of one high-cost member are not independent draws, so line resampling shrinks the interval toward zero width, and member_months has no line-level analogue, so the denominator becomes undefined.

Worked solution (25 min)

Compute the point estimate as allowed.sum() / months.sum() on the full sample.
Extract both columns to float numpy arrays and create rng = np.random.default_rng(seed).
Loop 2,000 times: idx = rng.integers(0, n, n); ratio = allowed[idx].sum() / months[idx].sum(); store it.
Take np.percentile(replicates, [2.5, 97.5]) and report the point estimate with that interval and the seed.

Expected result: A point estimate plus a 95 percent percentile interval that is wider on the upper side than the lower. With tens of thousands of members and a typical commercial cost distribution the relative half-width lands on the order of a few percent; an interval a small fraction of a percent wide is the signature of having resampled rows rather than members.

Verify it yourself

Re-running with the same seed reproduces both bounds exactly.
Raising replicates from 2,000 to 8,000 moves each bound by less than its third significant figure.
The mean of the replicate ratios sits close to the point estimate; a large gap indicates a bug in the resampler rather than skew.
Replicate denominators cluster tightly around the observed total member-months, which confirms the draw size is n rather than a fixed constant.

Follow-up

Half a percent of members have a transplant or a long NICU stay. Does a percentile interval still have nominal coverage, and what would you do about the tail?
The question changes to comparing PMPM across provider panels. What becomes the resampling unit and why?
How would you use the same replicates to test whether this year's PMPM differs from last year's, given the same members appear in both?

Given a list of patient records, how would you group and aggregate them based on specific clinical criteria?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Write a function to identify patterns or outliers in a given dataset.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Reconstruct drug coverage intervals and compute proportion of days covered

Hard · gaps and islands · window functions · adherence

pharmacy_claim holds fill_id, member_id, ndc_code, therapeutic_class_code, fill_date, days_supply, reversal_flag and reversed_fill_id. Early refills overlap, so days_supply must be laid end to end: each fill's coverage starts at the later of its fill_date and the previous interval's end. Both rows of a reversed pair are excluded. For one therapeutic_class_code over 2025-01-01 to 2025-06-30, return per member the proportion of days covered, defined as covered days divided by days from first fill_date to window end, plus the share of members at or above 0.80. Require at least two fills and 91 days of follow-up.

Approach

Clean the fills first. Drop rows with reversal_flag TRUE and also the fills they point at through reversed_fill_id. Dropping only the reversal row leaves an original fill counted as exposure that never reached the member. Use NOT EXISTS for that second step, since reversed_fill_id is nullable and NOT IN would return nothing.
Shift, never stack. The shifted end is covered_end_i = GREATEST(fill_date_i, covered_end_{i-1}) + days_supply_i, which looks recursive but has a closed form: with cum_i the running sum of days_supply through fill i, covered_end_i = cum_i + MAX over j <= i of (fill_date_j - cum_{j-1}). That is SUM(days_supply) OVER (ORDER BY fill_date ROWS UNBOUNDED PRECEDING) plus a running MAX of an anchor column, so it is two window functions and no recursive CTE.
Derive covered_start_i as GREATEST(fill_date_i, LAG(covered_end)). The intervals are disjoint by construction, so covered days is a plain sum of lengths and never needs a distinct day grid.
Clip intervals to the window end before summing, and decide explicitly whether to clip at disenrollment as well. Supply that runs past the window must not inflate the numerator.
Use the denominator the spec names, first fill_date to window end, and hold it constant across classes. A fixed-window denominator yields a different and usually lower number, and mixing the two across classes makes the comparison meaningless.
Apply the inclusion rules at member level after the intervals are built, then compute the share at or above 0.80 with the qualifying member count beside it.

Worked solution (40 min)

Filter to the therapeutic_class_code and window, then remove reversal rows and their originals with NOT EXISTS.
Per member ordered by fill_date, compute cum_supply and anchor = fill_date - (cum_supply - days_supply), then covered_end = cum_supply + MAX(anchor) OVER (ORDER BY fill_date ROWS UNBOUNDED PRECEDING).
Compute covered_start = GREATEST(fill_date, LAG(covered_end) OVER (ORDER BY fill_date)), clip both ends to the window, and drop intervals that collapse.
Sum interval lengths as covered_days, compute follow_up_days from first fill_date to window end, and divide.
Apply the two-fill and 91-day rules, then aggregate the share at or above 0.80.

Expected result: One row per qualifying member with covered_days, follow_up_days and pdc in [0, 1], plus a summary row with qualifying members and the share at or above 0.80. A member with 30-day fills on 2025-01-01, 2025-01-20 and 2025-04-01 is covered 2025-01-01 to 2025-03-01 and 2025-04-01 to 2025-04-30, giving 60 + 30 = 90 covered days over 181 follow-up days and a PDC of 0.497.

Verify it yourself

No pdc exceeds 1.0 before any clipping. A value above 1 proves supplies were stacked instead of shifted.
On a twenty-member sample, the summed interval lengths equal the distinct covered day count from a slow day-grid version.
A member with one fill is excluded by the two-fill rule rather than appearing with a PDC of 1.0.
Re-running with reversals left in raises covered days for exactly the members who have a reversed_fill_id, and for no one else.

Follow-up

A member is hospitalised for twelve days mid-window, when the facility supplies medication. Should those days count as covered, and what do published adherence measures do about them?
Someone wants to classify adherence over twelve months and then compare mortality from month zero. What is wrong with that design and what fixes it?
Two different ndc_codes in the same therapeutic class overlap. Is that stacking or switching, and how does your interval logic treat each case?

How do you prioritize tasks when working on multiple projects with competing deadlines?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you optimize a script that is running too slowly on a large volume of healthcare records?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

Plan interim looks and multiplicity control for a year-long trial

Hard · sequential testing · alpha spending · multiple comparisons · novelty effects

A 12-month cluster-randomised care-management trial has one primary endpoint (ambulatory care sensitive admissions per 1,000 member-years), three secondary endpoints, and five pre-specified subgroups. The steering group wants a read every month. At month 2 the primary is nominally significant at p = 0.03 with an effect roughly three times the powered assumption. Write the monitoring and analysis plan that should have been in place, and say precisely what you tell the steering group about month 2.

Approach

Quantify the peeking cost rather than asserting it. Testing accumulating data repeatedly at a fixed nominal 0.05 raises the overall type I error to roughly 0.14 at five equally spaced looks and roughly 0.19 at ten; a monthly read for a year is worse than either.
Replace fixed-threshold looks with a Lan-DeMets alpha-spending plan on a pre-specified number of looks. An O'Brien-Fleming spending function is right here because it spends almost nothing early (a first look needs a nominal p on the order of 1e-5) and preserves close to 0.05 for the final analysis; Pocock's constant boundary (about 0.0158 nominal at each of five looks) buys early stopping power at the cost of a stricter final look. Monitor the safety guardrail on a separate, non-binding harm boundary, because stopping for harm is a different decision from stopping for efficacy.
Control the endpoint family with a gatekeeping structure: test the primary at full alpha, and only on success test the three secondaries in a pre-specified fixed sequence or with Holm. Declare the five subgroups exploratory, test them as treatment-by-subgroup interactions rather than as within-subgroup significance, and report them with FDR control and that label attached.
Separate the three distinct reasons the month-2 effect is inflated. Alpha inflation from the unplanned look is one. Early-exposure inflation is another, since the first clusters to go live are the enthusiastic ones and their staff are running a new process attentively, which decays as it becomes routine. Claims incompleteness is the third and is specific to this outcome: inpatient facility claims run out over months, so month 2 admission counts are undercounted, and the undercount does not have to be equal across arms.
Tell the steering group that month 2 is not a result: p = 0.03 is nowhere near any defensible first-look boundary, the recent incurred months are incomplete at the stated paid-through date, and estimates taken at a crossing are conditionally biased upward, so even a real stop would require a bias-adjusted effect estimate rather than the raw one.

Worked solution (40 min)

State the fixed number of interim analyses and the information fractions at which they occur, then pick the spending function and derive nominal boundaries so the total spend equals 0.05.
Write the gatekeeping rule for the three secondaries and the exploratory label plus FDR rule for the five subgroups.
Specify that the primary is computed only at a stated paid-through date, with the three most recent incurred months either held back or completion-factored.
Add an exposure-time analysis: index every cluster on its own go-live date and report the effect by cluster-months since activation, so habituation and staggered rollout are visible rather than pooled away.
Write the month-2 statement: the boundary was not crossed, the data are incomplete, and no effect estimate will be circulated.

Expected result: A Lan-DeMets alpha-spending plan with a pre-specified look schedule and an O'Brien-Fleming shape for the primary, a separate non-binding harm boundary for the guardrail, fixed-sequence or Holm gatekeeping across the three secondaries, and the five subgroups as FDR-controlled exploratory interaction tests. Month 2 is reported as not a result on three independent grounds: boundary not crossed, claims incomplete, and early estimates conditionally inflated.

Verify it yourself

Confirm the naive repeated-testing error rate: about 0.14 at five looks and about 0.19 at ten, against a nominal 0.05.
Confirm the subgroup family-wise rate without correction: 1 - 0.95^5 = 0.23 under independence.
Confirm the chosen boundaries' cumulative spend equals the nominal 0.05.
Confirm every reported figure carries a paid-through date and that the last three incurred months are held back or completion-factored.

Follow-up

The steering group insists on monthly visibility. What can you show them every month that does not spend alpha?
How would you tell a genuine effect apart from an exposure-time artefact when clusters went live on a staggered schedule?
If the trial does cross an efficacy boundary at month 8, what effect estimate do you publish and why is it not the raw one?

Glycemic control share jumped when a feed connected

Medium · informative missingness · unit mismatch · metric specification

The share of members with a controlled glycemic result moved from 0.62 to 0.81 the month a second result feed was connected. lab_result carries result_id, order_id, patient_id, encounter_id, loinc_code, analyte_name, value_numeric, units, reference_low, reference_high, abnormal_flag, result_status and supersedes_result_id. The metric is members whose latest result in the year is below the control threshold, divided by members with any result in the year. Decide whether control actually improved, and specify the metric so results from both feeds can be pooled.

Approach

Split numerator and denominator. The denominator is members with any result, so a feed that tests more people changes the metric without changing anyone's health. Count denominator members before and after and see how much of the 19 point move is denominator growth.
Recompute on a fixed cohort: members who had a qualifying result in both the before and after periods. If the share is flat on the fixed cohort, the movement is who got tested, not what their results were.
Deduplicate to the latest non-cancelled row per (order_id, loinc_code), respecting supersedes_result_id, because a corrected result arrives as a new row and counting both gives one person two values.
Audit units and loinc_code mix by feed. Two unit conventions coexist for this analyte, percent and mmol/mol, related by mmol/mol equal to 10.929 times (percent minus 2.15), so 7.0 percent is 53 mmol/mol. Comparing mmol/mol values to a percent threshold would classify them all as uncontrolled and depress the share, so a rise cannot be explained by that mismatch, which rules it out rather than in.
State plainly that the tested population is not the covered population. Testing is ordered when someone suspects a problem or when a site runs standing orders, so the mean among tested members is not the population mean and cannot be made into one by imputation.
Respecify: numerator and denominator both restricted to members with a stated continuous-enrolment requirement, one deduplication rule, one unit after conversion, one threshold, and a separately reported testing rate so the denominator's movement is visible.

Follow-up

Someone proposes imputing the mean result for untested members. What happens to the estimate and why?
A model you are building gains AUC from a feature that marks whether the test exists. Why is that a warning rather than a win?
How would you report this metric so a clinical committee can see improvement separately from coverage of testing?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Getting an analysis acted on

Work that nobody used is a common and unflattering pattern in data careers, and interviewers probe for it. Have a story about an analysis that changed a decision, and be specific about how you got it in front of the person who could act. Also have one about work that went nowhere, with your reading of why.

How do you handle missing data in longitudinal clinical studies?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Make an honest case for your own analytical impact

Hard · impact attribution · counterfactual · self-assessment

You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.

Approach

The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.

Follow-up

Two other people also claim the contract measure. How do you split it?
Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
What is the strongest argument that your year had less impact than you just described?

Push back on an over-broad extract without blocking the work

Hard · minimum necessary · de-identification · governance

A researcher asks for a one-off extract covering 12,000 members: member_id, date of birth, five-digit residential postal code, full lab_result rows including value_text, and clinical note text, held on their laptop for a retrospective analysis of one lab analyte over time. They describe it as de-identified because names are excluded. A flat refusal is not an acceptable answer. Deliverable: the column list you will actually provide with a justification per retained field, the sentence explaining why free text defeats an enumerated-identifier approach, your small-cell rule for the published stratification, and what you deliver this week.

Approach

The probe is whether privacy is a design skill for you or a compliance reflex. Flat refusal and unquestioned fulfilment both fail, for opposite reasons.
Reconstruct the analysis from the research question rather than from the request. A longitudinal trend in one analyte needs a stable pseudonymous subject key, the analyte's loinc_code, value_numeric, units, the reference bounds, and a date. It does not need names, notes, or a precise birth date.
Replace each identifier with the least precise version that still answers the question: a salted per-study pseudonym instead of member_id, age in years or an age band instead of date of birth, a consistent per-subject date offset if only intervals matter, and a coarser geography than five-digit postal code.
Explain the free-text problem in one sentence. Removing an enumerated list of identifier fields does not de-identify a note, because notes carry names, relationships, employers, dates and rare clinical detail in prose, so only a reviewed extraction into structured fields or a formal statistical determination addresses it.
Set the publication rule before the analysis runs. Suppress cells below a stated threshold and suppress complements too, because a reader holding a row total and all but one cell recovers the suppressed count by subtraction.
Keep them moving. Deliver the reduced extract into a controlled environment this week rather than onto a laptop, and offer to derive any note-based variable yourself as a structured flag so they never need the prose.

Follow-up

They say the age band loses resolution their model needs. What do you do?
How do you set the suppression threshold, and who signs off on it?
The analysis later needs linking to a second data source. What changes?