cswork / libraryOpen practice workspace

Interview guides · Zoetis

Zoetis Data Scientist Interview Questions & Guide 2026

Diagnose a metric move for Zoetis data scientist prep: split by segment, rule out a logging change, separate mix shift from behaviour.

The 60-second brief

A Data Scientist at Zoetis plays a pivotal role in transforming the landscape of animal health. As the world’s leading animal health company, Zoetis leverages data science to discover, develop, manufacture, and commercialize medicines, vaccines, and diagnostic products for livestock and companion animals. In this role, you are not just building models; you are directly contributing to solutions that predict disease outbreaks, optimize therapeutic pipelines, and enhance the well-being of animals globally.

SQL is seldom the hardest round and is often the one that eliminates people. The working bar is usually window functions, correct deduplication, and joins that do not silently fan out rows, rather than obscure syntax.

Focus areas

  • Minimise protected health information in every extract
  • Risk-adjust outcomes before comparing providers or panels
  • Build member-month denominators from overlapping coverage spans

Overview

The impact of this position spans across multiple critical business units. You will collaborate with research and development (R&D) teams to analyze genomic and clinical trial data, work with supply chain leaders to optimize manufacturing yields, and partner with commercial teams to deliver precision veterinary medicine solutions. By translating complex biological and business datasets into actionable predictive models, you help veterinarians and livestock producers make faster, more informed decisions.

While a background in veterinary science or biology is highly valued, Zoetis prioritizes strong foundational data science skills, structured problem-solving, and the ability to translate technical findings into real-world business value.

The Zoetis interview process

Zoetis candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Resume Review

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

  • Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
  • Whether you name what you have not done instead of stretching to cover every line of the posting
  • Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

  • Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
  • Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
  • Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Presentation

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

  • Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
  • Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
  • Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

  • Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
  • Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
  • If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Deep-Dive Technical Discussions

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

  • Whether the query you write matches the plan you just described
  • What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
  • Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

  • Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
  • Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
  • Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Behavioral Assessments

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

  • Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
  • The counterfactual: what would have been done without your analysis, and why that default was worse
  • How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

  • Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
  • Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
  • Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Immortal time in adherence, treatment, and enrolment definitions

Classifying members as adherent, treated, or programme-enrolled requires them to survive and stay covered long enough to accumulate the defining events. That guaranteed event-free interval is assigned to the exposed group, so the exposure looks protective for reasons that have nothing to do with the treatment. Adherence studies are the classic case: measuring 12-month proportion of days covered and then comparing mortality builds survival into the exposure definition. Use time-varying exposure or a landmark analysis with the classification window excluded from follow-up.

Reading the most recent months of a claims-based series as real

Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.

Treating a non-significant result as proof of no effect

Say whether the confidence interval excludes the effect sizes you would have cared about. If it does not, the honest reading is that the test was underpowered, so report the minimum detectable effect the design could have found and what sample size would resolve it.

Writing SQL without stating NULL and tie-breaking behaviour

Before calling a query finished, say what it does with NULLs, ties and empty groups. NOT IN against a subquery containing a single NULL returns no rows at all, and RANK, DENSE_RANK and ROW_NUMBER differ precisely on ties, so name which one the question requires.

Questions & practice

Explain the difference between bias and recall, and discuss how you navigate this trade-off when optimizing a model.

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Check what information would not exist at prediction time, and exclude it.
  3. Set a baseline first, so any model has something honest to beat.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • Where could label leakage enter this setup?

What metrics would you use to evaluate a model where false negatives are significantly more costly than false positives?

Medium · machine learning and modelling

Approach

  1. Check what information would not exist at prediction time, and exclude it.
  2. Set a baseline first, so any model has something honest to beat.
  3. Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

  • How would you choose the decision threshold, and who owns that choice?
  • Where could label leakage enter this setup?

Can you explain how a random forest algorithm handles feature importance compared to a gradient boosted tree?

Medium · machine learning and modelling

Approach

  1. Say how the offline result would be validated online before it is trusted.
  2. Set a baseline first, so any model has something honest to beat.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • Where could label leakage enter this setup?
  • How would you choose the decision threshold, and who owns that choice?

Proportion of days covered with shifted, truncated refill intervals

Hard · interval merging · adherence · path dependence

pharmacy_claim has fill_id, member_id, therapeutic_class_code, fill_date, days_supply, reversal_flag, reversed_fill_id. For one therapeutic class and a fixed 12-month window, compute proportion of days covered per member: distinct days on which a dispensed days_supply covers the day, divided by days from the member's first in-window fill through the window end. An early refill shifts coverage forward rather than stacking, and coverage is truncated at the window end. Drop both rows of every reversed pair. Return member_id, pdc, and the share at pdc >= 0.80 among members with at least 2 fills and at least 91 days of follow-up.

Approach

  1. Remove reversals as pairs first. Drop every row with reversal_flag true, and also drop the fill_ids those rows point at through reversed_fill_id. Dropping only the flagged row leaves a dispense that was never collected in the exposure.
  2. Walk fills per member in fill_date order carrying a cursor: start = max(fill_date, previous_end + 1 day), end = start + days_supply - 1. The shift is path dependent, so a plain cumsum over days_supply does not reproduce it. Use itertools.accumulate or a per-member loop over numpy arrays, not a row-wise apply over the whole frame.
  3. Truncate the last interval at the window end before measuring. Without truncation a 90-day fill dispensed on the final day pushes the covered-day count past the denominator and PDC above 1.0.
  4. Use the denominator the definition states: first in-window fill_date through window end, inclusive. Not a flat 365, and not first fill to last fill, which is a different metric that rewards early discontinuation.
  5. Apply the eligibility filter before computing the >= 0.80 share, and report the size of that denominator next to the share. A share without its denominator is not reviewable.

Worked solution (40 min)

  1. Filter to the therapeutic class and the window, then remove reversed pairs by dropping flagged rows and the fill_ids in reversed_fill_id.
  2. Sort by member_id, fill_date. Per member, accumulate start = max(fill_date, prior_end + 1 day) and end = start + days_supply - 1.
  3. Clip every interval's end at the window end and drop intervals whose start is past it.
  4. Covered days per member = sum of (end - start + 1) over the shifted intervals, which are disjoint by construction.
  5. Denominator = (window_end - first_fill_date).days + 1. Divide, then filter to members with >= 2 fills and denominator >= 91 and compute the share at or above 0.80.

Expected result: Per-member PDC in [0, 1], plus one share. Worked case: fills at day 0 (30 days), day 20 (30 days) and day 55 (30 days) in a 180-day denominator give intervals 0-29, 30-59 and 60-89, so 90 covered days and PDC = 0.50. Change the window to 60 days with fills at days 0, 20 and 40 and the shifted intervals truncate to 0-59, giving PDC = 1.00 while a days-supply sum would report 90/60 = 1.50.

Verify it yourself

  • No member has PDC above 1.0. Any value above 1.0 means intervals stacked instead of shifting, or the final interval was not truncated.
  • A member with a single 30-day fill at the start of a 180-day denominator scores 30/180 = 0.167 and is excluded from the share by the two-fill rule.
  • For one member, rebuild covered days as the size of a Python set of date ordinals over the raw fills and confirm it matches the interval arithmetic.
  • Shifting every fill_date forward by the same number of days changes each denominator but leaves the covered-day counts unchanged except at the truncation boundary.

Follow-up

  • How does PDC differ from medication possession ratio, and which of the two can exceed 1.0?
  • A member switches to a different ingredient inside the same therapeutic class mid-window. Should the intervals chain, and what does that do to the class-level number?
  • Members who die or lose coverage mid-window get a short denominator and often a high PDC. If you then compare mortality by adherence category, what bias have you built in and how do you remove it?

Find inpatient encounters with no resulted lab during the stay

Medium · anti-join · null semantics · event matching

encounter holds encounter_id, patient_id, encounter_type, admit_ts, discharge_ts and facility_id. lab_result holds result_id, order_id, patient_id, encounter_id (NULL for outpatient standing orders), loinc_code, specimen_collected_ts, resulted_ts and result_status. Return every inpatient encounter discharged in 2025 with no final or corrected lab_result linked to it. A colleague wrote WHERE encounter_id NOT IN (SELECT encounter_id FROM lab_result) and got zero rows back. Say why, write the correct query, then say what changes when labs must instead be matched on patient_id with the specimen collected between admit_ts and discharge_ts.

Approach

  1. Name the mechanism rather than the symptom. x NOT IN (subquery) expands to x <> y1 AND x <> y2 AND ..., and a single NULL y makes that chain UNKNOWN, never TRUE. Since lab_result.encounter_id is nullable by design for standing orders, this query can only ever return zero rows.
  2. Rewrite as NOT EXISTS, or LEFT JOIN with an IS NULL filter. Both treat a NULL key as no match rather than as unknown. Adding IS NOT NULL inside the NOT IN subquery also works, but NOT EXISTS is the habit that survives someone making another column nullable later.
  3. Put the result_status predicate inside the correlated subquery or the ON clause, not in an outer WHERE. Outside, it turns the anti-join back into an inner join and an encounter whose only labs were cancelled disappears instead of qualifying.
  4. For the timestamp variant, match on patient_id with specimen_collected_ts inside the stay rather than resulted_ts. A specimen drawn an hour before discharge can result the next day, and anchoring on resulted_ts would wrongly call that stay lab-free.
  5. Restrict to encounter_type 'inpatient' and discharge_ts in 2025, and report open encounters with a NULL discharge_ts as a separate count instead of letting the filter swallow them.

Worked solution (20 min)

  1. Demonstrate the cause: SELECT COUNT(*) FROM lab_result WHERE encounter_id IS NULL returns a non-zero number.
  2. Write the NOT EXISTS version with result_status IN ('final','corrected') inside the correlated subquery.
  3. Write the LEFT JOIN version with the same predicate in the ON clause and WHERE l.result_id IS NULL, then compare counts.
  4. Build the timestamp variant joining on patient_id with specimen_collected_ts >= admit_ts AND < discharge_ts.
  5. Report both counts side by side and explain which encounters differ between the two definitions.

Expected result: One row per qualifying inpatient encounter with encounter_id, patient_id, facility_id, admit_ts and discharge_ts. The count is strictly below total 2025 inpatient discharges and strictly above zero, while the NOT IN version returns exactly zero rows.

Verify it yourself

  • NOT EXISTS and LEFT JOIN with IS NULL return identical row counts.
  • Adding AND encounter_id IS NOT NULL inside the NOT IN subquery reproduces the NOT EXISTS count exactly.
  • No returned encounter_id appears anywhere in lab_result with result_status in ('final','corrected').
  • The patient-and-timestamp variant returns fewer encounters than the encounter_id variant, since it also catches labs whose encounter link is missing.

Follow-up

  • Write the LEFT JOIN form and say exactly where the result_status predicate must sit for the two forms to agree.
  • How would you separate encounters whose only labs were cancelled from encounters with no lab rows at all?
  • Some encounters carry member_id NULL because the person index did not match. What does that do to a payer-side version of this measure?

Measure ninety-day primary care contact for an enrollment cohort

Easy · cohort analysis · conversion rate · date windows

member_enrollment gives span_id, member_id, product_type, effective_date and termination_date, which is NULL while a span is active. medical_claim_line gives member_id, service_start_date, procedure_code and claim_status. The cohort is members whose earliest 2025 span has effective_date in January 2025 and who stay covered at least 90 days from it. Return, by product_type, cohort size and the share with at least one paid claim line whose procedure_code is in a supplied list of evaluation-and-management office visit codes and whose service_start_date falls in the half-open window from effective_date to effective_date plus 90 days.

Approach

  1. Build the cohort as one row per member before touching claims. Take the earliest 2025 span per member, require its effective_date in January, and require coverage to effective_date + 90 days, which means termination_date IS NULL OR termination_date >= effective_date + 89.
  2. Use EXISTS for the numerator rather than a join. A member with four office visits must count once; joining and counting rows inflates the numerator and can push the rate above 1.
  3. Keep the window half-open and anchored per member. Every member has a different effective_date, so a single fixed calendar window is a different metric and will not match the cohort definition.
  4. Aggregate numerator and denominator in one pass with a conditional sum grouped by product_type, and carry the raw counts alongside the rate so the reader can see the denominator.
  5. State the reportability rule on the output: the January cohort is not final until 90 days plus professional-claims runout have passed, so label the paid-through date.

Follow-up

  • A member changes plan on day 40, closing one span and opening another. Is she still in the cohort, and which product_type does she report under?
  • What moves if denied lines are allowed into the numerator, and which definition is right for an access metric?
  • How would you show this as a cumulative curve by day since effective_date instead of one 90-day point?

Describe how you would approach solving a sample business problem where the available data is highly unstructured and incomplete.

Medium · product sense and metrics

Approach

  1. State what result would change your recommendation, so the answer is falsifiable.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

How would you design an experiment to test the effectiveness of a new digital diagnostic tool for veterinarians?

Medium · a/b testing and experimentation

Approach

  1. Say whether units interfere with each other, and switch design if they do.
  2. Name the guardrails that would stop a launch even on a positive primary result.
  3. State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

  • How would you handle interference between treated and control units?
  • What would you do if you could not randomise at all?

Walk me through your resume and highlight a project where you had to handle significant data ambiguity.

Medium

Approach

  1. Say what you would check first and why it is the highest-information step.
  2. Clarify what is being asked and what a complete answer would contain.
  3. Work from the decision backwards to the evidence you would need.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

What was your individual contribution to the collaborative project highlighted on your resume?

Medium

Approach

  1. Work from the decision backwards to the evidence you would need.
  2. State your assumptions explicitly before working the problem.
  3. Say what you would check first and why it is the highest-information step.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Estimate a programme effect at a fixed risk-score enrolment cutoff

Medium · regression discontinuity · regression to the mean · local average treatment effect

Care management enrols members whose prospective_risk_score in member_enrollment crosses a fixed annual threshold of 1.75; above the line enrolment is near-automatic and below it unavailable, and about 85 percent of members above the line actually enrol. Leadership refused a randomised holdout and wants the programme's effect on next-year allowed PMPM. Two years of scores and claims are available. State the design, the estimand it identifies, the assumptions, and the diagnostics you would run before reporting anything.

Approach

  1. Propose a fuzzy regression discontinuity on prospective_risk_score at 1.75. Uptake jumps but is not complete, so the estimator is the ratio of the jump in outcome to the jump in enrolment probability at the cutoff, which is a Wald ratio.
  2. State the estimand honestly up front: a local effect for members near 1.75, who are the least sick enrollees. It does not identify the effect on the top risk decile, which is usually the number leadership actually wants, and saying so before the result lands is what keeps the analysis credible.
  3. Say explicitly why the obvious alternative fails. A pre-post on enrolled members selects on a high score in year one, and such cohorts regress toward the mean in year two whether or not anything is done, so a pre-post design reports savings every time. The discontinuity is immune because it compares members on either side of an arbitrary line.
  4. Specify the estimation: local linear fits on each side, triangular kernel, a data-driven MSE-optimal bandwidth, and robust bias-corrected inference rather than a naive local-linear confidence interval. Do not fit a high-order global polynomial.
  5. Run the falsification suite before looking at the effect: a density test for manipulation of the running variable, continuity of pre-determined covariates at the cutoff, sensitivity across bandwidths, and placebo cutoffs away from 1.75.

Worked solution (30 min)

  1. Plot the outcome and the enrolment rate against the centred running variable in uniform bins to confirm a visible jump in both.
  2. Estimate the first stage (jump in enrolment probability at 1.75) and the reduced form (jump in next-year PMPM), then take the ratio for the fuzzy RD estimate.
  3. Select the bandwidth by an MSE-optimal rule with a triangular kernel and local linear fits, and report robust bias-corrected intervals.
  4. Run the density test at 1.75 and the covariate continuity tests on age, prior-year cost, dual_eligible_flag, and product_type.
  5. Re-estimate at 0.5x, 1x, and 2x the selected bandwidth, and at placebo cutoffs of 1.50 and 2.00.

Expected result: A fuzzy RD estimating a local average treatment effect on compliers near the 1.75 cutoff, equal to the outcome jump divided by the enrolment jump, with local linear fits, a triangular kernel, an MSE-optimal bandwidth, and robust bias-corrected inference. The result is explicitly not generalisable to the highest-risk members.

Verify it yourself

  • The density test should show no discontinuity at 1.75; a spike just above the line indicates manipulation and invalidates the design.
  • Pre-determined covariates should show discontinuities indistinguishable from zero at the cutoff.
  • The point estimate should be stable across 0.5x, 1x, and 2x bandwidth.
  • Placebo cutoffs away from 1.75 should return null effects.

Follow-up

  • Risk scores are built from coded diagnoses, and coding intensity is something people can influence. What would manipulation look like in the density, and what do you do if you see it?
  • The MSE-optimal bandwidth leaves 900 members. How do you decide whether to report an underpowered estimate at all?
  • Design an encouragement version instead: randomise the invitation rather than enrolment. What does instrumental variables buy you, and which assumption is most at risk?

First-pass denial rate doubled in one received week

Medium · pipeline failure · segment cuts · denominator definition

First-pass denial rate jumped from 6.1 to 11.8 percent in a single received_date week and stayed at the new level. Nothing in the adjudication rules changed that week. You have medical_claim_line: claim_id, claim_version, frequency_code, claim_status, denial_reason_code, member_id, received_date, adjudicated_at, billing_provider_npi. The metric counts lines whose first adjudication returns 'denied', over lines received in the window at claim_version = 1. Find the cause, say whether denial behaviour actually changed, and deliver the corrected series plus the query change that fixes it.

Approach

  1. Cut the excess by denial_reason_code before anything else. A behaviour change spreads across codes; an upstream failure concentrates in one. If a single eligibility-related code carries nearly all of the 5.7 point excess, the claims are fine and the eligibility data was not.
  2. Join the denied lines to member_enrollment and compare the span's created_at to the claim's received_date. Lines denied for members whose coverage span loaded after the claim arrived are a load-ordering failure, not a coding failure.
  3. Check resolution downstream: take those claim_ids and look for a later version with frequency_code 7 that adjudicates to 'paid'. A high resubmit-and-pay share confirms the denial was transient and recoverable.
  4. Verify the denominator is not the mover. Count lines at claim_version = 1 by received_date and check the weekday mix, because submission batching makes received_date strongly day-of-week patterned and a short or holiday week shifts the denominator without changing behaviour.
  5. Confirm runout: the metric should be held until 60 days of adjudication have passed, so check that the comparison weeks are equally mature rather than comparing a settled week to a fresh one.
  6. Restate the series with the affected reason code broken out as its own line, and change the query to carry denial_reason_code into the reported breakdown rather than only the headline rate.

Follow-up

  • The affected lines all get paid on resubmission. Is the first-pass denial rate then wrong, or right and uninteresting?
  • How would you monitor for this class of failure without waiting for someone to notice a chart?
  • What changes if the excess is spread evenly across denial_reason_code instead of concentrated?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

  • Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
  • For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
  • Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

  • Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
  • Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
  • Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

  • Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
  • Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
  • Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

  • Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
  • For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
  • Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

  • Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
  • Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
  • Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

  • Repeat the depth protocol on the second-ranked area with the same six-problem structure.
  • Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
  • Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

  • Re-run the six prompts from day one under the same clock and compare both correctness and time.
  • Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
  • Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Stories where someone disagreed with you

Have two ready. In one, the data was on your side and you had to move someone who outranked you. In the other, the pushback was correct and you changed position. The second is the harder story and it lands better, because it shows you separate being right from being attached to an answer. Name the person's actual objection.

If a business stakeholder asks you to explain a complex deep learning model's prediction, how do you communicate this without using technical jargon?

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Quantify the outcome, including what you would not claim credit for.

Follow-up

  • How did you know the outcome was caused by your change?
  • What did you decide not to do, and why?

Make an honest case for your own analytical impact

Hard · impact attribution · counterfactual · self-assessment

You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.

Approach

  1. The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
  2. State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
  3. Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
  4. Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
  5. Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
  6. Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.

Follow-up

  • Two other people also claim the contract measure. How do you split it?
  • Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
  • What is the strongest argument that your year had less impact than you just described?

Sequence three urgent requests with one analyst-week available

Medium · prioritisation · tradeoffs · stakeholder communication

Three requests land on Monday and you have one week. An actuarial team needs incurred-claims completion factors restated before a filing deadline on Thursday. A clinical programme owner wants a deterioration model refreshed because its calibration has drifted in one region. A trial operations team wants site enrolment forecasts for a portfolio review in two weeks. Each requester believes theirs is blocking. Deliverable: your sequence with the reasoning, the message you send to whoever is deprioritised, and the smaller artefact you hand each of the two you cannot fully serve.

Approach

  1. The probe is whether you prioritise on consequence and reversibility rather than on who asked loudest or most recently.
  2. Classify each request by what happens if it slips. A regulatory or contractual deadline is irreversible on its date, a drifting model is causing harm every day it keeps running, and a portfolio review can absorb a provisional number. That ordering is defensible to all three requesters because it does not depend on your preferences.
  3. Take the deadline-bound work first, but scope it to the minimum defensible output, because completion factors feeding a filing carry a different error tolerance than a slide.
  4. Do not let the drifting model simply wait. Quantify the harm cheaply by comparing calibration in the affected region against the rest, and if it is materially miscalibrated propose flagging or suppressing its output for that region within the hour rather than at the end of a refresh.
  5. Give each deprioritised requester something real: a provisional forecast with its uncertainty and a refresh date, or a diagnostic that tells them whether their problem is urgent. Say no explicitly with a date rather than going quiet, because silence is what produces escalation.

Follow-up

  • The programme owner escalates to your manager. What do you want your manager to be able to say?
  • Midweek the actuarial work needs two more full days than you estimated. What gives?
  • How does your answer change if the drifting model drives a clinical outreach list rather than a report?