Interview guides · Bristol Myers Squibb
Bristol Myers Squibb Data Scientist Interview Questions & Guide 2026
Bristol Myers Squibb Data Scientist interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.
The 60-second brief
At Bristol Myers Squibb, a Data Scientist plays a pivotal role in transforming complex biological, clinical, and commercial data into actionable insights that directly impact patient lives. Operating at the intersection of advanced analytics, technology, and healthcare, you will work on challenges that range from optimizing clinical trial designs to modeling market dynamics for life-saving therapeutics. The data environments here are massive and highly complex, requiring a unique blend of scientific curiosity and technical rigor.
Learn the economics of the product category before the loop. Marketplaces, subscription products and ad-supported products turn on different core quantities (match rate and liquidity, retention and churn, fill rate and yield) and fail in different characteristic ways.
Focus areas
- Correct for claims runout before reporting recent months
- Separate censoring from events in time-to-event models
- Minimise protected health information in every extract
Overview
The work you do as a Data Scientist is critical to the mission of Bristol Myers Squibb: discovering, developing, and delivering innovative medicines that help patients prevail over serious diseases. Whether you are embedded in the Global Drug Development team, Commercial Analytics, or R&D, your models and dashboards will guide strategic multi-million dollar decisions. This means your technical solutions must not only be mathematically sound but also highly interpretable for non-technical stakeholders and executive leadership.
This role is ideal for individuals who thrive on high-impact work and enjoy navigating ambiguity. You will have the opportunity to leverage cutting-edge machine learning, statistical modeling, and data visualization tools to solve real-world healthcare problems. The collaborative culture at Bristol Myers Squibb ensures that you will regularly interface with clinicians, epidemiologists, and business leaders, making strong communication skills just as vital as your coding abilities.
The Bristol Myers Squibb interview process
Bristol Myers Squibb candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
HR Screening Call
Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.
What to demonstrate
- Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
- Whether you name what you have not done instead of stretching to cover every line of the posting
- Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage
How to prepare
- Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
- Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
- Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live
Hiring Manager Interview
Much of this round runs on your own history, but the manager is not collecting a project list. They are working out what it is like when something goes wrong on your watch: how late the bad news tends to arrive, and whether a number you hand over has been checked by anyone including you. That is why the strongest material is a project where you can describe the part that did not work and what it cost. A result you cannot take full responsibility for, however clean, gives them nothing to trust you with afterwards.
What to demonstrate
- Whether you volunteer the limits of a result you are proud of, or wait to be pushed onto them
- How errors surfaced in your past work, and whether you or somebody else found them
- Whether the scope you claim matches the level of detail you can still produce about it
- What you did the first time a stakeholder acted on something of yours that turned out to be wrong
How to prepare
- Rebuild one headline figure from memory down to the join and the filter, so a question about the denominator does not stall the conversation
- For each project you raise, write the sentence you would say to someone who had already acted on a number that later turned out wrong
- Mark which parts of a project were yours and which belonged to other people, and state that boundary yourself before anyone asks
Technical Evaluation
This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.
What to demonstrate
- Whether your row counts survive each join, and whether you notice on your own when they do not
- Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
- Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
- Reaching a defensible answer inside the window instead of a refined one after it
How to prepare
- Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
- Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
- Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither
Panel Interview
A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.
What to demonstrate
- Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
- Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
- Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience
How to prepare
- Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
- For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
- Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story
Where candidates lose points
Reading the most recent months of a claims-based series as real
Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.
Ignoring clustering by provider, facility, or site
Patients within a provider panel, and subjects within a trial site, are correlated on both case mix and practice pattern. Standard errors computed as if observations were independent are too narrow by roughly the design effect, 1 + (m - 1) * ICC, where m is the average cluster size. With 50 patients per provider and an intracluster correlation of only 0.02, variance is understated by about a factor of two, which manufactures significant provider differences out of noise. Cluster-robust errors or a random intercept per provider, and cluster-level randomisation when you design the test, are the corrections.
Building features from data that postdates the prediction time
Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.
Defining the cohort on a post-treatment condition
Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.
Questions & practice
Explain a complex statistical concept or machine learning model you used recently as if you were explaining it to a non-technical manager.
Medium · machine learning and modelling
Approach
- Set a baseline first, so any model has something honest to beat.
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
Follow-up
- Where could label leakage enter this setup?
- What would you monitor after launch to know the model is still valid?
How do you decide which machine learning algorithm is best suited for a specific predictive modeling task?
Medium · machine learning and modelling
Approach
- Set a baseline first, so any model has something honest to beat.
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Say how the offline result would be validated online before it is trusted.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- Where could label leakage enter this setup?
Build a claims lag triangle and complete recent months
Medium · chain ladder · runout · pivot
medical_claim_line has claim_id, claim_version, frequency_code, service_start_date, paid_date, allowed_amount, claim_status. Using 24 fully mature incurred months, build a development triangle of allowed dollars by incurred month and payment lag in whole months, derive cumulative completion factors by chain ladder, then estimate ultimate allowed for the three most recent incurred months at a stated paid-through date. Deduplicate to surviving claim versions before building the triangle. Return incurred_month, paid_to_date, completion_factor, estimated_ultimate.
Approach
- Deduplicate to surviving claim versions and keep paid lines first. A triangle built on all versions develops on adjustment churn rather than on payment timing, and replacements arrive late, so the distortion concentrates in exactly the tail you are trying to estimate.
- Compute lag as a whole-month difference between incurred month and paid month, not as day difference divided by 30. Ragged month lengths otherwise shuffle identical claims between lag buckets depending on which month they fall in.
- Pivot to incurred_month by lag, cumulate along lag, then form age-to-age factors as the ratio of the column L+1 total to the column L total, summed over incurred months mature enough to have both. Volume weighting is the chain ladder; a simple mean of per-month ratios lets one low-volume month dominate the factor.
- Chain the age-to-age factors from each lag to ultimate and invert: completion factor at lag L is the reciprocal of the product of factors from L onward. Apply as estimated_ultimate = paid_to_date / completion_factor.
- State the assumption you have just made, which is that the development pattern is stable. It is not, after a claims-system migration, a network change or a processing backlog, and the most recent month's factor is the least reliable because it rests on the fewest observations while carrying the largest adjustment.
Worked solution (35 min)
- Deduplicate to surviving versions, drop voids, keep claim_status 'paid', and derive incurred_month from service_start_date and paid_month from paid_date.
- lag = (paid_month.year - incurred_month.year) x 12 + (paid_month.month - incurred_month.month); drop negative lags and investigate them separately.
- Pivot to a triangle of summed allowed_amount, cumulate along the lag axis, and mask cells past the paid-through date so no cell contains a future payment.
- Compute volume-weighted age-to-age factors per lag over the mature rows, then chain them into cumulative completion factors.
- Apply the factor for each recent month's current maturity to its paid_to_date and return the four columns.
Expected result: Three rows. Completion is well below 1 at short lag and approaches 1 in the tail; for a mixed medical book a one-month-old incurred month commonly sits far from complete, while a twelve-month-old one is close. Exact values are book-specific, so the reviewable property is monotonicity and the holdout test below, not the numbers themselves.
Verify it yourself
- Cumulative completion factors are non-decreasing in lag and converge toward 1 at the tail.
- Hold out one fully mature incurred month, apply the factors at each of its intermediate maturities, and confirm each estimate lands within a few percent of its known total.
- estimated_ultimate is greater than or equal to paid_to_date for every row.
- A pharmacy-only triangle completes far faster than an inpatient-facility-only triangle, which confirms the lag computation is reading real payment timing.
Follow-up
- A processing backlog means the last two months developed slower than history. What does your estimate do, and how would you detect that before reporting?
- Pharmacy, professional and inpatient facility develop on different clocks. How do you split the triangle, and what breaks if the service mix shifts?
- What happens to a PMPM series if someone reports an unadjusted recent month, and in which direction does the error point?
Explain the difference between an inner join, left join, and outer join in SQL, and write a query demonstrating a complex multi-table join.
Medium · sql and data manipulation
Approach
- Say which table is the grain you start from, and join outward from it.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
Follow-up
- What breaks if events arrive late or out of order?
- How would you verify this result without re-running the same query?
Walk me through how you would write a script to automate a repetitive data ingestion and cleaning pipeline.
Medium · sql and data manipulation
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- State the window function and its partition and ordering out loud before writing it.
Follow-up
- What breaks if events arrive late or out of order?
- How would you verify this result without re-running the same query?
Report site activation to first randomization without hiding non-enrollers
Medium · grain collapse · left join · censoring
study_site holds one row per (study_id, site_id): site_activation_date, which is NULL for a site selected but never activated, and site_status. study_subject_visit holds subject_visit_id, study_id, site_id, subject_id, randomization_ts (NULL for screen failures), visit_name, visit_number, actual_visit_date and visit_status, with one row per subject per protocol visit, so randomization_ts repeats across every visit of a randomized subject and a site that has screened nobody has no rows there at all. For one study_id and a stated data cut date, return per activated site the activation date, the first randomization_ts and the days between them, with activated sites that have randomized nobody still present. Add a study-level summary giving median days to first randomization and the share of activated sites with no randomization by day 90.
Approach
- Take the site universe from study_site, never from study_subject_visit. The visit table can only contain sites that have screened at least one subject, so a universe built from DISTINCT site_id there omits exactly the sites this metric exists to expose. Filter study_site to the study and to site_activation_date IS NOT NULL, then LEFT JOIN the per-site first randomization.
- Collapse the grain before aggregating anything on the visit side. The table repeats randomization_ts across every visit of a randomized subject, so any count taken at row grain is inflated by visits per subject. MIN(randomization_ts) per site is safe, COUNT is not.
- State the date boundary rule. randomization_ts is a timestamp and site_activation_date is a date, so a randomization at 23:40 local time can land on either side of a day boundary depending on the zone the timestamp is stored in. Cast once, deliberately, and say which zone.
- Use PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days) for the median and be explicit that it ignores NULLs, so it is a median over enrolling sites only. That is exactly why the non-enroller share is reported beside it rather than instead of it.
- Censor the day-90 denominator against the data cut. A site activated 40 days before the cut has not had 90 days to randomize, so it cannot be scored on that bar and belongs outside that denominator.
Worked solution (25 min)
- CTE sites: site_id and site_activation_date from study_site for the study where site_activation_date IS NOT NULL. The (study_id, site_id) key already gives one activation date per site, so assert that key rather than de-duplicating defensively.
- CTE first_rand: site_id and MIN(randomization_ts) over study_subject_visit rows for the study where randomization_ts IS NOT NULL, grouped by site_id.
- LEFT JOIN sites to first_rand and compute days_to_first_randomization with an explicit cast of the timestamp to a date.
- Summary: PERCENTILE_CONT(0.5) over non-null days, plus the count and share of sites with no randomization by day 90, restricted to sites activated at least 90 days before the data cut.
- Compare the site-level row count with COUNT(DISTINCT site_id) in study_subject_visit for the study; the difference is the activated sites that have screened nobody, which is the population the metric is for.
Expected result: One row per activated site with site_id, site_activation_date, first_randomization_ts and days_to_first_randomization, NULL in both randomization columns for sites that have randomized nobody, including sites with no study_subject_visit rows at all; plus one summary row with median_days_over_enrolling_sites, sites_activated_90d_or_more and share_with_no_randomization_by_day_90.
Verify it yourself
- Site-level row count equals the number of study_site rows for the study with a non-null site_activation_date, and is greater than or equal to COUNT(DISTINCT site_id) in study_subject_visit for that study. Below it means either an activated site is missing from study_site or a visit row carries a site_id whose activation date is NULL, both data-quality findings.
- No days_to_first_randomization is negative. A negative value is a data-quality finding to raise, not a number to clamp at zero.
- The day-90 denominator excludes sites activated within 90 days of the data cut. Including them can only add sites that have not had a full 90 days, and any of those without a randomization would be miscounted as a failure.
Follow-up
- The same site_id appears under two studies. What breaks, and what is the correct grain for this metric?
- Would you present this as a median, as a Kaplan-Meier curve of time to first randomization, or both, and what does the curve show that the median cannot?
- A site randomized one subject who withdrew at visit 2. Does that site count as enrolling here, and how does that differ from a randomized-and-retained measure?
How do you ensure that your data visualizations remain intuitive and accessible for non-technical business partners?
Medium · product sense and metrics
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
What is your process for choosing the right visualization type (e.g., scatter plot, bar chart, heat map) for complex multivariate data?
Medium · product sense and metrics
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Fix the population and the time window before naming any metric.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How do you design a Tableau dashboard to track key performance metrics for a newly launched drug?
Medium · product sense and metrics
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Fix the population and the time window before naming any metric.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
Describe a situation where you had to manage competing priorities with tight deadlines. How did you decide what to prioritize?
Medium · product sense and metrics
Approach
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Restate the decision this analysis has to support, and who acts on the answer.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Walk me through a data science project listed on your resume, focusing on your specific contributions and the technical methodology you chose.
Medium
Approach
- Say what you would check first and why it is the highest-information step.
- State your assumptions explicitly before working the problem.
- Work from the decision backwards to the evidence you would need.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Diagnose a sample ratio mismatch in a randomised outreach test
Easy · sample ratio mismatch · randomisation audit · selection bias
A member-level outreach test hashed member_id into two arms and wrote assignments to a bucket table before any contact was made. The analysis extract, built by joining assignments to member_enrollment and to the vendor send file, contains 20,560 treatment and 19,440 control members out of 40,000 assigned. The intended split was 50/50. State the test you would run on the split, compute it, say what it means for the outcome analysis, and describe how you would locate the cause.
Approach
- Test the split against the intended allocation with a chi-square goodness-of-fit on one degree of freedom, or equivalently a two-sided normal test on the treatment count with standard deviation sqrt(N x 0.5 x 0.5). Both give the same p-value.
- Stop the outcome analysis before running it. A significant ratio mismatch means the analysis population is not the randomised population, so any treatment effect is confounded by whatever selected members out, and no covariate adjustment can be trusted to fix it.
- Instrument the pipeline stage by stage and recompute the ratio at each one: the assignment table, the join to member_enrollment, the vendor send file, the response join, and the final extract. The mismatch appears at exactly one stage, and that stage is the bug.
- Given this domain, check the eligibility join first. Coverage terminations are back-dated and land after the fact, so a join that requires active coverage at extract time silently drops members, and the drop is not independent of arm if the outreach itself triggered contact that refreshed eligibility records.
- Compare the dropped members with the retained ones on termination_date, termination_reason_code, product_type, and prospective_risk_score to characterise the selection, then rerun the test on a corrected extract rather than rebalancing the arms.
Worked solution (15 min)
- Expected count per arm is 20,000.
- z = (20,560 - 20,000) / sqrt(40,000 x 0.5 x 0.5) = 560 / 100 = 5.6.
- Equivalently chi-square = 560^2 / 20,000 + 560^2 / 20,000 = 15.68 + 15.68 = 31.36 on 1 df.
- Convert to a p-value: two-sided p is about 2e-8.
- Pull arm counts at each pipeline stage and find the first stage where the ratio departs from 50/50.
Expected result: z = 5.6, chi-square = 31.4 on 1 degree of freedom, p is about 2e-8. This is not chance. The experiment is not analysable until the differential drop is located and either fixed or demonstrated to be independent of assignment.
Verify it yourself
- A deterministic hash on member_id should produce an exact or near-exact 50/50 split at the assignment table; if it does not, the randomiser itself is the bug.
- The imbalance should appear at exactly one pipeline stage, not drift gradually across several.
- The dropped members should differ from the retained on at least one observable, which both confirms the selection and describes it.
Follow-up
- The assignment table itself is exactly 20,000 / 20,000. What does that tell you and where do you look next?
- A colleague proposes randomly downsampling the treatment arm to 19,440 to restore balance. Why is that not a fix?
- At what imbalance would you not investigate, and how did you pick that threshold?
Readmission rate rose while every facility improved
Medium · simpsons paradox · mix shift · risk adjustment
Quarter over quarter the 30-day unplanned readmission rate for an attributed population rose from 14.2 to 15.1 percent, yet all eleven facilities in encounter show a flat or falling rate over the same period. You have encounter (facility_id, admit_ts, discharge_ts, encounter_type, admission_type, discharge_disposition, drg_code, is_planned_admission, index_encounter_id) and twelve months of prior medical_claim_line for comorbidities. Explain the direction reversal numerically, and state what the population-level number should be reported as instead.
Approach
- Confirm the reversal is arithmetic, not a definition difference, by recomputing both the facility rates and the pooled rate from the same query with the same index-stay exclusions.
- Decompose the pooled change exactly with a Kitagawa split. With facility weights w and within-facility rates r, the change equals the sum over facilities of (w_1 - w_0) times (r_0 + r_1)/2, the mix term, plus the sum of (w_0 + w_1)/2 times (r_1 - r_0), the within term. If the mix term exceeds the total change and the within term is negative, the reversal is volume shifting toward facilities with higher baseline rates.
- Ask why the weights moved. Check index-stay volume by facility, and check whether an exclusion changed behaviour: acute-to-acute transfers (discharge_disposition 'transfer_acute') must not open an index stay, and if one facility began transferring more, its denominator shrinks toward its sickest retained patients.
- Check case mix inside the facilities too, using drg_code and comorbidities drawn from the 12 months of claims before admit_ts, excluding complications of the index stay itself. A facility rate can be flat while its case mix worsens.
- Replace the raw pooled rate with an observed over expected ratio, where expected is the sum of predicted probabilities from a model fitted on age, sex and prior comorbidities. Raw rates are not comparable across a population whose facility and case mix is moving.
Follow-up
- Your O/E ratio is 1.04 with a 95 percent interval of 0.96 to 1.13. What does that let you say to a facility, and what does it not?
- If one facility contributes 40 percent of index stays, how does that change how you present the decomposition?
- How would you handle a member with two index stays in the quarter, where the second is also a readmission from the first?
Suggested practice checklist
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Day 1 — Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Day 2 — Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Day 3 — Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Day 4 — The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Day 5 — Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Day 6 — Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Day 7 — Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Ownership when the result is inconvenient
Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.
Describe your biggest professional achievement as a Data Scientist and explain the measurable impact it had.
Medium · behavioural and stakeholder questions
Approach
- State the situation in two sentences and spend the rest on your reasoning.
- Close with what you would do differently, concretely.
- Name the disagreement or constraint, and how you resolved it with evidence.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Defend a null result against a programme sponsor
Medium · regression to the mean · difference-in-differences · stakeholder management
A care management programme enrolled the top 1 percent of members by prior-year allowed spend. The sponsor's deck shows allowed PMPM for enrollees falling 34 percent from the year before enrollment to the year after, and asks for budget to triple the programme. You rebuild the evaluation with a concurrent comparison group selected by the identical spend rule in the same period. The difference-in-differences estimate is a 3 percent reduction with a confidence interval spanning zero. Deliverable: how you present this, to whom, in what order, and what you propose next.
Approach
- Name the probe: whether you can deliver a finding that costs somebody their programme without softening it into uselessness or creating an adversary who routes around you next time.
- Lead with the mechanism, not the verdict. Show the comparison group's own unadjusted drop, which will be large, because a cohort selected on an extreme of the outcome regresses toward the mean whether or not anyone intervenes. The sponsor's 34 percent is mostly that, and it is a property of the selection rule rather than a criticism of their clinicians.
- Give the sponsor the finding privately before it appears in any deck their leadership sees. Being surprised in a room is what turns a methods disagreement into a political one.
- State the estimate with its interval and say what it rules out as well as what it fails to establish. A 3 percent point estimate whose interval crosses zero is not evidence of no effect; it is insufficient power to separate a modest effect from none, and those are different claims.
- Arrive with a design rather than only an objection. Propose a regression discontinuity at the enrollment threshold if the rule is applied sharply, or a randomised rollout across the next wave of eligible members, and state the sample size needed to detect the effect size the sponsor believes in.
Follow-up
- The sponsor says withholding the programme from a comparison group is unethical. What do you propose instead?
- Leadership wants one number for the board next week. What do you give them?
- What result would change your mind and make you believe the 34 percent?
Explain an incomplete cost series to a non-technical executive
Medium · claims runout · uncertainty · executive communication
An executive opens a dashboard showing risk-adjusted allowed PMPM down 6 percent across the last three incurred months and asks whether the cost initiative is working. You know pharmacy adjudicates within days, professional claims within weeks, and inpatient facility claims over months, so those three months are both undercounted and mix-shifted toward cheap services. There is also a genuine movement of roughly 1.5 percent in the older, complete months. Deliverable: a two-minute spoken answer, the change you make to the chart, and what you commit to telling them and when.
Approach
- The probe is whether you can be honest about uncertainty without sounding evasive. Executives read hedging as not knowing, so the answer has to end in a commitment.
- Separate the two claims immediately. The 6 percent is an artefact of incomplete data, and there is a smaller real movement in the months that are complete. Delivering only the debunk leaves them holding nothing.
- Explain the lag in their terms: the bills for the most expensive care arrive last, so an incomplete month always looks cheap and always looks like it is improving. Keep the phrase completion factor out of the first pass.
- Fix the chart instead of explaining around it. Shade or omit the incomplete months, print the paid-through date on the axis, and overlay the same series as it looked at equivalent maturity a year earlier so the shape is comparable rather than merely lower.
- Close with a date and a magnitude. Commit to a restated figure once runout matures, say roughly how far you expect the 6 percent to shrink, and name what you would need to see to call the initiative working.
Follow-up
- They ask for your best guess today, knowing it is provisional. What do you say?
- The restated number comes back at 1 percent. How do you handle having flagged the 6 percent at all?
- How do you stop this dashboard producing the same conversation next quarter?