Interview guides · ICON

ICON Data Scientist Interview Questions & Guide 2026

ICON data scientist prep: probability, sampling distributions, and stating the assumption behind every test you run.

The 60-second brief

A Data Scientist at ICON operates at the intersection of advanced analytics and clinical research. You are not merely building models in a vacuum; you are leveraging data to drive decision-making in clinical trials, real-world evidence (RWE) generation, and patient care optimization. Your work directly impacts the efficiency of drug development and the quality of life for patients globally.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

Correct for claims runout before reporting recent months
Separate censoring from events in time-to-event models
Reconstruct drug exposure intervals from dispensing records

Overview

This role is intellectually demanding, requiring a bridge between complex statistical rigor and the practical constraints of the pharmaceutical and healthcare industries. You will be expected to collaborate with diverse teams, including clinical operations, medical experts, and external stakeholders. Success in this role requires the ability to translate technical findings into actionable insights that satisfy both scientific standards and business requirements.

Be prepared to discuss your work in the context of clinical outcomes, as the hiring team often prioritizes domain expertise and the ability to apply data science to patient-centric problems over pure algorithmic theory.

The ICON interview process

ICON candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Interviews

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Behavioral Interviews

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Stakeholder Engagement

An extra round usually exists because something is still open after the standard loop: a skill the earlier interviews did not sample, a level decision, or two interviewers who disagreed. It is rarely a rerun of what you already did well. Ask the recruiter who you are meeting, what function they sit in, and how long the session runs. That is an ordinary scheduling question, and the answer changes what you should prepare. What separates a strong candidate here is treating the round as a fresh evaluation with its own bar, rather than assuming earlier performance carries you through or sinks you.

What to demonstrate

Whether you can answer well on ground the earlier rounds did not cover, without leaning on what you already said to someone else
Consistency of the facts in your stories: the same sample size, timeframe, team size and scope of your own role as in earlier conversations
How you handle an unfamiliar format live, including whether you ask what kind of answer is wanted before producing one

How to prepare

Ask the recruiter for the interviewer's function, the length, and whether to expect a coding surface, a discussion, or a presentation. Preparing for a 30 minute conversation with a partner team is not the same work as preparing for a 60 minute technical block.
Write out what each earlier round actually covered, then list the two or three areas nobody probed. That gap is the most likely subject of the extra round.
Re-read the numbers in the project stories you have already told, so a second telling does not quietly contradict the first.

Where candidates lose points

Ignoring clustering by provider, facility, or site

Patients within a provider panel, and subjects within a trial site, are correlated on both case mix and practice pattern. Standard errors computed as if observations were independent are too narrow by roughly the design effect, 1 + (m - 1) * ICC, where m is the average cluster size. With 50 patients per provider and an intracluster correlation of only 0.02, variance is understated by about a factor of two, which manufactures significant provider differences out of noise. Cluster-robust errors or a random intercept per provider, and cluster-level randomisation when you design the test, are the corrections.

Pre-post evaluation on a high-cost or high-risk cohort

Cohorts selected on an extreme value of the outcome regress toward the mean on their own. Members identified as the top 1 percent of spend in one year spend far less the next, whether or not anyone intervenes, because the selecting year captured both chronic severity and one-off events. A pre-post design on such a cohort will report savings every time. A concurrent comparison group selected by the same rule in the same period, or a regression discontinuity at the selection threshold, is the minimum credible design.

Reading an observational correlation as a causal effect

Name the confounder you are most worried about and the design that would remove it: an experiment, a difference-in-differences with a checked pre-period trend, an instrument, or a regression discontinuity. When none is available, state which direction the bias likely runs and bound the claim accordingly.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Questions & practice

Simulate how ignoring clustering inflates the false positive rate

Medium · simulation · design effect · type i error

Simulate a provider-level comparison with no true effect. 40 providers, 60 members each, a continuous outcome with total variance 1 and intraclass correlation 0.02. Randomise providers 20 to 20, not members. Over 5,000 replications, report the share of replications where a two-sample t-test run on all 2,400 individual observations, ignoring provider, returns p below 0.05. Report the same share when the unit of analysis is the 40 provider means. Then state how the first number follows from the design effect 1 + (m - 1) x ICC.

Approach

Decompose the variance explicitly rather than tuning it: provider variance = ICC = 0.02, residual variance = 1 - ICC = 0.98. Draw u_j once per provider and e_ij per member, so the outcome is u_j + e_ij and the correlation between two members of one provider is 0.02 by construction.
Randomise at the provider level. That is the entire mechanism: the treatment indicator is constant within a provider, so in any single replicate the provider intercepts are confounded with the arm, and the naive test reads that confounding as signal.
Vectorise over replications with a (reps, providers, members) array. 5,000 x 2,400 normals is trivial in numpy, and a Python loop is what tempts people to cut replications to 500 and report a number with Monte Carlo noise of plus or minus 0.017.
Predict the answer before running it: design effect = 1 + (60 - 1) x 0.02 = 2.18, naive standard errors are too small by sqrt(2.18) = 1.476, so the rejection rate at nominal 0.05 is about 2 x (1 - Phi(1.96 / 1.476)) = 0.18. Agreement between prediction and simulation is what makes the result a demonstration rather than an anecdote.
Run the provider-mean analysis as the control. Recovering 0.05 there proves the generator is correct and isolates the failure to the analysis unit.

Follow-up

How many providers would you need for 80 percent power on a 0.2 SD difference with 60 members each and this ICC?
Cluster sizes are unequal in reality. The design effect approximation becomes 1 + ((1 + CV^2) x mbar - 1) x ICC. Which direction does that move your sample size and why?
The intervention cannot be withheld from any provider. Sketch a design that still yields a defensible estimate.

Collapse claim versions before summing allowed amounts by member

Easy · claim versioning · groupby · deduplication

You are given medical_claim_line as a pandas DataFrame: claim_line_id, claim_id, claim_version, frequency_code (1 original, 7 replacement, 8 void), member_id, service_start_date, procedure_code, allowed_amount, claim_status. Return total allowed_amount per member for service dates in one stated calendar month, counting each claim once at its surviving version and excluding any claim whose surviving version is a void. Multi-line claims must keep every line of that surviving version. Output columns: member_id, allowed_amount, sorted descending. Do not deduplicate on claim_id alone.

Approach

Separate the two grains out loud before writing code: versions live at the claim level, dollars live at the line level. Every bug in this exercise comes from mixing them.
Compute the surviving version per claim with a groupby transform of max over claim_version, then keep the lines whose claim_version equals that value. A transform keeps the frame at line grain, which a sort plus head(1) does not.
Drop whole claims whose surviving version carries frequency_code 8. A void is not a zero-dollar line, it retracts the claim, so filtering rows rather than claims leaves the prior version's lines behind.
Filter to claim_status 'paid' and state the assumption: denied, pended and reversed lines carry an allowed_amount that never became a liability, so including them overstates cost.
Restrict service_start_date to the month boundaries, then groupby member_id and sum. Sort descending and return.

Worked solution (20 min)

Build a max-version lookup per claim_id with groupby('claim_id')['claim_version'].transform('max') and keep rows where claim_version equals it.
From the surviving rows, collect the claim_ids whose frequency_code is 8 and drop those claim_ids entirely.
Filter claim_status to 'paid' and service_start_date into the month window.
Group by member_id, sum allowed_amount, sort descending, reset the index.

Expected result: A two-column frame member_id, allowed_amount. On a claim with versions 1, 7, 7 only the third version's lines contribute; a claim whose latest version is frequency_code 8 appears nowhere. Total allowed is strictly below a naive sum over all rows whenever any claim in the sample was adjusted, and equal to it when none was.

Verify it yourself

The distinct claim_id count in your filtered frame equals the count of distinct claim_ids whose surviving version is not a void.
The line count you kept for each surviving claim equals that claim's line count at its max version, not 1.
A naive sum over every row is greater than or equal to your total, with equality only when no claim has claim_version above 1.

Follow-up

A claim has versions 1, 7, 7 where the second replacement has fewer lines than the first. What does your code return and is that correct?
How would you handle a claim whose lines span two calendar months at the boundary of the reporting window?
The same logic has to run over 400 million lines in a warehouse. What changes, and what stays the same?

Expand overlapping coverage spans into fractional member-months

Medium · interval expansion · denominators · vectorisation

member_enrollment has span_id, member_id, plan_id, product_type, effective_date, termination_date (NULL while active). Both dates are inclusive. Write a function returning one row per calendar month of a stated year with total member-months, where a member contributes covered days in the month divided by days in that month, capped at one month per member even when two spans overlap after a plan change or a retroactive span. Do not expand to one row per member-day: the sample has 400,000 members. Output: month, member_months.

Approach

Resolve NULL termination_date to the reporting end date and say so in a comment. An open span is not an infinite span, and clipping it at the year end is what keeps the denominator finite and auditable.
Merge overlapping and adjacent spans per member before touching months. Sort by member_id and effective_date, carry a running maximum of the end date, and open a new merged group when effective_date exceeds that running max plus one day. Adjacent means a gap of zero days, which a plan change produces constantly.
Cross join the merged spans to the 12 month boundaries rather than to days. Overlap days = (min(span_end, month_end) - max(span_start, month_start)).days + 1, clipped below at 0. That is at most 12 rows per span instead of 365.
Divide overlap days by the number of days in that month, so February and July are weighted correctly, then sum by month.
Validate on a constructed member before trusting the aggregate: a single span covering the whole year must sum to exactly 12.0.

Follow-up

Product type changes mid-year. The metric must be reported by product_type. Where does the cap now apply and what breaks?
An eligibility file arrives with a retroactive termination that shortens a span you already reported on. How do you restate?
Why member-months rather than distinct members, in one sentence, for a director who wants the simpler number?

Find inpatient encounters with no resulted lab during the stay

Medium · anti-join · null semantics · event matching

encounter holds encounter_id, patient_id, encounter_type, admit_ts, discharge_ts and facility_id. lab_result holds result_id, order_id, patient_id, encounter_id (NULL for outpatient standing orders), loinc_code, specimen_collected_ts, resulted_ts and result_status. Return every inpatient encounter discharged in 2025 with no final or corrected lab_result linked to it. A colleague wrote WHERE encounter_id NOT IN (SELECT encounter_id FROM lab_result) and got zero rows back. Say why, write the correct query, then say what changes when labs must instead be matched on patient_id with the specimen collected between admit_ts and discharge_ts.

Approach

Name the mechanism rather than the symptom. x NOT IN (subquery) expands to x <> y1 AND x <> y2 AND ..., and a single NULL y makes that chain UNKNOWN, never TRUE. Since lab_result.encounter_id is nullable by design for standing orders, this query can only ever return zero rows.
Rewrite as NOT EXISTS, or LEFT JOIN with an IS NULL filter. Both treat a NULL key as no match rather than as unknown. Adding IS NOT NULL inside the NOT IN subquery also works, but NOT EXISTS is the habit that survives someone making another column nullable later.
Put the result_status predicate inside the correlated subquery or the ON clause, not in an outer WHERE. Outside, it turns the anti-join back into an inner join and an encounter whose only labs were cancelled disappears instead of qualifying.
For the timestamp variant, match on patient_id with specimen_collected_ts inside the stay rather than resulted_ts. A specimen drawn an hour before discharge can result the next day, and anchoring on resulted_ts would wrongly call that stay lab-free.
Restrict to encounter_type 'inpatient' and discharge_ts in 2025, and report open encounters with a NULL discharge_ts as a separate count instead of letting the filter swallow them.

Worked solution (20 min)

Demonstrate the cause: SELECT COUNT(*) FROM lab_result WHERE encounter_id IS NULL returns a non-zero number.
Write the NOT EXISTS version with result_status IN ('final','corrected') inside the correlated subquery.
Write the LEFT JOIN version with the same predicate in the ON clause and WHERE l.result_id IS NULL, then compare counts.
Build the timestamp variant joining on patient_id with specimen_collected_ts >= admit_ts AND < discharge_ts.
Report both counts side by side and explain which encounters differ between the two definitions.

Expected result: One row per qualifying inpatient encounter with encounter_id, patient_id, facility_id, admit_ts and discharge_ts. The count is strictly below total 2025 inpatient discharges and strictly above zero, while the NOT IN version returns exactly zero rows.

Verify it yourself

NOT EXISTS and LEFT JOIN with IS NULL return identical row counts.
Adding AND encounter_id IS NOT NULL inside the NOT IN subquery reproduces the NOT EXISTS count exactly.
No returned encounter_id appears anywhere in lab_result with result_status in ('final','corrected').
The patient-and-timestamp variant returns fewer encounters than the encounter_id variant, since it also catches labs whose encounter link is missing.

Follow-up

Write the LEFT JOIN form and say exactly where the result_status predicate must sit for the two forms to agree.
How would you separate encounters whose only labs were cancelled from encounters with no lab rows at all?
Some encounters carry member_id NULL because the person index did not match. What does that do to a payer-side version of this measure?

Report site activation to first randomization without hiding non-enrollers

Medium · grain collapse · left join · censoring

study_site holds one row per (study_id, site_id): site_activation_date, which is NULL for a site selected but never activated, and site_status. study_subject_visit holds subject_visit_id, study_id, site_id, subject_id, randomization_ts (NULL for screen failures), visit_name, visit_number, actual_visit_date and visit_status, with one row per subject per protocol visit, so randomization_ts repeats across every visit of a randomized subject and a site that has screened nobody has no rows there at all. For one study_id and a stated data cut date, return per activated site the activation date, the first randomization_ts and the days between them, with activated sites that have randomized nobody still present. Add a study-level summary giving median days to first randomization and the share of activated sites with no randomization by day 90.

Approach

Take the site universe from study_site, never from study_subject_visit. The visit table can only contain sites that have screened at least one subject, so a universe built from DISTINCT site_id there omits exactly the sites this metric exists to expose. Filter study_site to the study and to site_activation_date IS NOT NULL, then LEFT JOIN the per-site first randomization.
Collapse the grain before aggregating anything on the visit side. The table repeats randomization_ts across every visit of a randomized subject, so any count taken at row grain is inflated by visits per subject. MIN(randomization_ts) per site is safe, COUNT is not.
State the date boundary rule. randomization_ts is a timestamp and site_activation_date is a date, so a randomization at 23:40 local time can land on either side of a day boundary depending on the zone the timestamp is stored in. Cast once, deliberately, and say which zone.
Use PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days) for the median and be explicit that it ignores NULLs, so it is a median over enrolling sites only. That is exactly why the non-enroller share is reported beside it rather than instead of it.
Censor the day-90 denominator against the data cut. A site activated 40 days before the cut has not had 90 days to randomize, so it cannot be scored on that bar and belongs outside that denominator.

Follow-up

The same site_id appears under two studies. What breaks, and what is the correct grain for this metric?
Would you present this as a median, as a Kaplan-Meier curve of time to first randomization, or both, and what does the curve show that the median cannot?
A site randomized one subject who withdrew at visit 2. Does that site count as enrolling here, and how does that differ from a randomized-and-retained measure?

How would you approach a project involving Real-World Evidence (RWE) versus a standard clinical trial dataset?

Medium

Approach

Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Can you discuss the mathematical foundations behind LLMs and their potential applications in clinical documentation?

Medium

Approach

Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Design a switchback when a shared review queue breaks SUTVA

Medium · interference · switchback · carryover · clustered standard errors

A prior-authorisation auto-approval rule for a set of imaging procedure codes is meant to cut turnaround time. Requests sit in one shared queue worked by a fixed nurse pool across two review sites, so auto-approving one request frees reviewer capacity and speeds up the requests that were not auto-approved. Outcomes are median hours from received_date to adjudicated_at and the first-pass denial rate. Request-level randomisation is therefore invalid. Design the test: randomisation unit, block length, effective sample size, and the analysis.

Approach

Name the violation precisely. The treatment assigned to one request changes the outcome of other requests through a shared capacity constraint, so the stable unit treatment value assumption fails and a request-level difference in means estimates neither the direct effect nor the total effect.
Randomise time rather than requests: assign the policy on or off for whole (site, time block) cells. Every request inside a block experiences one consistent queue regime, so the block-level contrast is a genuine system-level effect including the capacity spillover you care about.
Choose block length from the carryover horizon, not from convenience. Estimate how long the queue takes to return to steady state after a load shock from the historical backlog series, then set blocks well longer than that and discard a burn-in equal to the drain time at the start of each block.
Analyse at the block level. The unit of analysis is the block mean, the effective sample size is the number of analysable blocks and not the tens of thousands of requests inside them, and standard errors must be cluster-robust or HAC because adjacent blocks share staffing and seasonality.
Balance the calendar deliberately: use whole weeks, or block on day of week, so the policy is not disproportionately on during Mondays. If the shared resource can instead be partitioned (separate nurse pods with separate queues), cluster randomisation on pods is simpler, but with two sites you have two clusters, which is not a design.

Worked solution (35 min)

Estimate the queue drain time from historical backlog after a volume spike, and set block length to several multiples of it (for example weekly blocks if the queue settles within a day).
Randomise policy on or off independently per (site, week), balanced so each site gets roughly half treated weeks.
Drop the burn-in window at the start of each block, then compute one block-level summary per (site, week) for each outcome.
Estimate the effect as a block-level regression of the summary on the treatment indicator with site and calendar-period fixed effects, using cluster-robust or HAC standard errors.
Power the study from the historical block-to-block standard deviation of the outcome, not from request-level variance.

Expected result: A switchback randomised at (site, week) with a discarded burn-in, analysed on block means with site and period fixed effects and HAC or cluster-robust standard errors. With two sites over 26 weeks the effective sample is about 52 block observations, fewer after burn-in, and that is the number the MDE must be computed from.

Verify it yourself

Lag-1 autocorrelation of block means after burn-in should be near zero; if it is not, the blocks are too short for the carryover.
Treated and control blocks should be balanced across day-of-week and month.
Run the randomisation labels on historical data as an A/A and confirm a null effect with correct interval coverage.
The block-level standard deviation should be materially larger than request-level SD over sqrt(requests per block), which is the tell that clustering matters.

Follow-up

Suppose you had randomised at the request level and 30 percent of the effect leaked to the control requests. What does that do to the required sample size?
Turnaround time is right-skewed with a long tail. Would you analyse block medians, block means, or something else?
How would you detect that your chosen block length is too short for the carryover you assumed?

Pooled persistency fell but no one lost coverage

Medium · interval sql · mix shift · cohort seasonality

Pooled 12-month enrollment persistency fell from 0.78 to 0.66 for the cohort anchored in a single month. member_enrollment carries span_id, member_id, plan_id, product_type, effective_date, termination_date, termination_reason_code, dual_eligible_flag and region_code. The metric allows at most one gap of 45 days or fewer. Product mix in that anchor month was not stable. Determine how much of the fall is real disenrollment rather than span bookkeeping or mix, and deliver a persistency series a contract owner could act on.

Approach

Inspect the query's unit of analysis first. Spans are not members, and a benefit or plan change closes one span and opens the next on the same day, so any logic that treats a non-null termination_date as loss of coverage counts an administrative event as churn.
Rebuild member-level coverage by unioning all spans per member and merging adjacent or overlapping ones, then apply the 45-day gap rule to the merged intervals rather than per span.
Cut the terminations by termination_reason_code. plan_change and eligibility_redetermination have entirely different meanings for a contract, and death is not churn you can address.
Split by product_type and stop pooling. Commercial open enrollment and managed medicaid redetermination produce structurally different churn on different calendars, and the pooled average of the two is a number that describes nobody.
Compare the anchor month to the same month in the prior year rather than to the adjacent month, because enrollment is strongly seasonal and an anchor month sitting on an open-enrollment or redetermination boundary is not comparable to its neighbour.
Apply the Kitagawa decomposition across product_type to separate how much of the 12 point fall is mix, meaning the anchor cohort simply contained more of a high-churn product, from how much is within-product deterioration.

Follow-up

Why 45 days, and what would change if the allowable gap were 30 or 60?
A member terminates and re-enrolls 90 days later under a different plan_id. Are they retained? Defend your answer against the opposite one.
What guardrail would catch someone improving persistency by declining to enroll members likely to churn?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

The project you argued against

Saying no well is a senior skill and it is rarely rehearsed. Think of a time you told someone their analysis was not worth doing, or that the experiment could not answer their question at the sample size available. Explain what you offered instead. Refusal without an alternative reads as obstruction rather than judgement.

How do you handle a situation where a stakeholder has a request that is not supported by the data?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

What is your experience with SQL and SAS for data manipulation and analysis?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What would you do differently if you ran that project again?
What did you decide not to do, and why?

How do you handle missing or noisy data in a clinical trial context?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

Can you explain your experience with Causal Inference and its application in clinical studies?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?