Interview guides · Biohub
Biohub Data Scientist Interview Questions & Guide 2026
SQL practice for Biohub data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.
The 60-second brief
A Data Scientist at Biohub operates at the cutting edge of AI-powered biology, working on a mission as ambitious as it is vital: to help scientists cure, prevent, or manage all diseases by the end of this century. Supported by the Chan Zuckerberg Initiative, Biohub brings together world-class machine learning engineers, AI researchers, software developers, and experimental biologists. As a Data Scientist here, you do not just analyze data—you architect, standardize, and scale the massive data ecosystems that make AI-driven biological discovery possible.
Allocate prep to your weakest link rather than your favourite topic. Of the three things that usually gate the outcome (SQL that is correct under messy joins, sound reasoning about experiments, and structured framing of an open-ended problem), candidates tend to over-invest in modelling theory and under-invest in framing.
Focus areas
- Correct for claims runout before reporting recent months
- Reconstruct drug exposure intervals from dispensing records
- Minimise protected health information in every extract
Overview
Your work will directly power grand challenges such as building an AI-based virtual cell model, mapping complex biological systems with novel imaging technologies, and sensing tissue inflammation in real time. The datasets you will work with are unprecedented in scale and diversity: billions of single-cell transcriptomic data points, tens of thousands of donor-matched genomic samples, petabyte-scale dynamic imaging datasets, and terabyte-scale mass spectrometry datasets.
What makes this role uniquely rewarding is its commitment to open science. Rather than keeping these breakthrough datasets siloed, you will collaborate with teams to publish high-quality, standardized data products through public platforms like and the. These resources are used by tens of thousands of global researchers monthly, meaning your work directly accelerates scientific progress and drug discovery worldwide.
The Biohub interview process
Biohub candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Initial Screening
A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.
What to demonstrate
- Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
- Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
- Whether timeline, location and compensation expectations make the rest of the loop worth scheduling
How to prepare
- Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
- Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
- Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice
Technical Assessment
Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.
What to demonstrate
- Whether an ambiguous term becomes a specific column and filter before any computation happens
- Whether you read the schema for keys and cardinality rather than only for column names
- Whether the result answers the question at the grain it was asked at, per user or per session or per day
How to prepare
- Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
- On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
- Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over
Onsite Loop
A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.
What to demonstrate
- Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
- Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
- Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience
How to prepare
- Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
- For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
- Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story
Where candidates lose points
Reading the most recent months of a claims-based series as real
Claims incur before they are reported and paid, so recent incurred months are systematically undercounted until runout completes. The lag is not uniform: pharmacy adjudicates in days, professional claims in weeks, inpatient facility claims in months. That means recent data is both too low and mix-shifted toward cheap services, which reads as a cost improvement and a utilisation drop at once. The fix is to hold the last three incurred months back or apply completion factors, and to state the paid-through date on every chart.
Treating clinical measurements as missing at random
A lab result, a vital sign, or a screening exists because someone ordered it, and ordering tracks suspicion of disease, visit frequency, and site workflow. Imputing the mean or dropping incomplete rows biases the population estimate and can flip the sign of an association, because the untested are systematically healthier or systematically disengaged. The presence indicator is often more predictive than the value, which is a warning sign rather than a feature win: a model that learns test ordering will not transfer to a site with different protocols.
Building features from data that postdates the prediction time
Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.
Accepting a metric definition without asking about the denominator
Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.
Questions & practice
How would you set up a validation strategy to evaluate a machine learning model predicting cellular response to a genetic perturbation, ensuring no leakage of donor-specific features?
Medium · machine learning and modelling
Approach
- Set a baseline first, so any model has something honest to beat.
- Check what information would not exist at prediction time, and exclude it.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
Explain how you would train a multi-modal model that bridges single-cell transcriptomic data with spatial imaging data.
Medium · machine learning and modelling
Approach
- Say how the offline result would be validated online before it is trusted.
- Check what information would not exist at prediction time, and exclude it.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- What would you monitor after launch to know the model is still valid?
Collapse claim versions before summing allowed amounts by member
Easy · claim versioning · groupby · deduplication
You are given medical_claim_line as a pandas DataFrame: claim_line_id, claim_id, claim_version, frequency_code (1 original, 7 replacement, 8 void), member_id, service_start_date, procedure_code, allowed_amount, claim_status. Return total allowed_amount per member for service dates in one stated calendar month, counting each claim once at its surviving version and excluding any claim whose surviving version is a void. Multi-line claims must keep every line of that surviving version. Output columns: member_id, allowed_amount, sorted descending. Do not deduplicate on claim_id alone.
Approach
- Separate the two grains out loud before writing code: versions live at the claim level, dollars live at the line level. Every bug in this exercise comes from mixing them.
- Compute the surviving version per claim with a groupby transform of max over claim_version, then keep the lines whose claim_version equals that value. A transform keeps the frame at line grain, which a sort plus head(1) does not.
- Drop whole claims whose surviving version carries frequency_code 8. A void is not a zero-dollar line, it retracts the claim, so filtering rows rather than claims leaves the prior version's lines behind.
- Filter to claim_status 'paid' and state the assumption: denied, pended and reversed lines carry an allowed_amount that never became a liability, so including them overstates cost.
- Restrict service_start_date to the month boundaries, then groupby member_id and sum. Sort descending and return.
Worked solution (20 min)
- Build a max-version lookup per claim_id with groupby('claim_id')['claim_version'].transform('max') and keep rows where claim_version equals it.
- From the surviving rows, collect the claim_ids whose frequency_code is 8 and drop those claim_ids entirely.
- Filter claim_status to 'paid' and service_start_date into the month window.
- Group by member_id, sum allowed_amount, sort descending, reset the index.
Expected result: A two-column frame member_id, allowed_amount. On a claim with versions 1, 7, 7 only the third version's lines contribute; a claim whose latest version is frequency_code 8 appears nowhere. Total allowed is strictly below a naive sum over all rows whenever any claim in the sample was adjusted, and equal to it when none was.
Verify it yourself
- The distinct claim_id count in your filtered frame equals the count of distinct claim_ids whose surviving version is not a void.
- The line count you kept for each surviving claim equals that claim's line count at its max version, not 1.
- A naive sum over every row is greater than or equal to your total, with equality only when no claim has claim_version above 1.
Follow-up
- A claim has versions 1, 7, 7 where the second replacement has fewer lines than the first. What does your code return and is that correct?
- How would you handle a claim whose lines span two calendar months at the boundary of the reporting window?
- The same logic has to run over 400 million lines in a warehouse. What changes, and what stays the same?
What file formats (e.g., Zarr, HDF5, Parquet) would you select to store multi-modal genomic and imaging data, and how do you optimize them for parallel read/write operations?
Medium · sql and data manipulation
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Say which table is the grain you start from, and join outward from it.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
Follow-up
- How does the query change if the join becomes one-to-many?
- How would you verify this result without re-running the same query?
How do you balance the trade-offs between compute costs, storage costs, and query latency when architecting big data solutions on AWS or Google Cloud?
Medium · sql and data manipulation
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Say which table is the grain you start from, and join outward from it.
Follow-up
- How does the query change if the join becomes one-to-many?
- What breaks if events arrive late or out of order?
How would you structure a metadata schema to accommodate diverse biological experiments, ensuring it remains flexible yet strictly queryable?
Medium · sql and data manipulation
Approach
- State the window function and its partition and ordering out loud before writing it.
- Say which table is the grain you start from, and join outward from it.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
Follow-up
- How would you verify this result without re-running the same query?
- How does the query change if the join becomes one-to-many?
Turn overlapping coverage spans into fractional member-month denominators
Medium · intervals · gaps and islands · denominators
member_enrollment holds span_id, member_id, plan_id, product_type, effective_date and termination_date, which is NULL while the span is active. A member can hold several spans in a year, and back-dated plan changes make spans overlap. Return one row per calendar month of 2025 with total member-months, where a member contributes covered days in the month divided by that month's length, and a day covered by two overlapping spans counts once. Both span endpoints are inclusive. Collapse overlaps with window functions before joining a month calendar, and do not assume one span per member.
Approach
- Normalise the open end first. COALESCE(termination_date, DATE '9999-12-31') or clip to the reporting end, because a NULL termination_date makes every BETWEEN comparison unknown and drops exactly the members who are still covered.
- Merge overlapping spans per member before the calendar join. Order by effective_date, carry a running MAX of prior termination_date with a frame ending one row before, and open a new island where effective_date exceeds that running max plus one day, which keeps adjacent spans contiguous rather than splitting them.
- Clip each merged island to the reporting year, then cross join to a twelve-row month calendar and keep pairs that intersect.
- Compute covered days as LEAST(island_end, month_end) minus GREATEST(island_start, month_start) plus one, and divide by the actual length of that month rather than a flat 30, so February and the 31-day months are not distorted.
- Sum the fractions by month. Merging after the calendar join also works but forces a DISTINCT over member-days, which stops scaling at population size.
Worked solution (30 min)
- CTE bounded: clip spans to 2025-01-01 through 2025-12-31 with COALESCE on termination_date, discarding spans that do not intersect the year.
- CTE merged: gaps-and-islands per member using LAG and a running MAX over preceding rows to collapse overlapping and adjacent spans.
- CTE months: twelve rows of month_start, month_end and days_in_month.
- Join merged to months on interval intersection and compute covered_days per pair.
- SUM(covered_days / days_in_month::numeric) grouped by month, ordered by month.
Expected result: Twelve rows of (month, member_months) with fractional values. Any single member contributes at most 1.0 to a month and at most 12.0 across the year. A member covered 2025-01-01 through 2025-02-20 contributes 1.0 in January and 20/28 = 0.714 in February.
Verify it yourself
- For a member with spans 2025-01-01..2025-03-31 and 2025-03-15..2025-06-30, March contributes exactly 1.0, not 1.5.
- Total member-months is at most 12 times COUNT(DISTINCT member_id) and at least the count of members covered the whole year.
- Forcing every termination_date to NULL raises the total. If it drops to zero, the NULL handling is missing.
Follow-up
- Where does a continuous-enrolment filter with an allowable 45-day gap belong, and how does that change the island boundary rule?
- One human appears under two member_ids after a master person index merge. What does that do to member-months, and how would you detect it?
- How do you treat a span that terminates on the day it starts, and what does that row usually represent?
Describe your approach to dimensionality reduction (e.g., UMAP, t-SNE, VAEs) for single-cell data. What are the mathematical trade-offs of each?
Medium · product sense and metrics
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- How would you detect that the metric is being gamed rather than genuinely improving?
What loss functions and evaluation metrics are most appropriate when dealing with highly imbalanced biological datasets, such as rare disease variants or low-abundance proteins?
Medium · product sense and metrics
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- What would you do if the primary metric and the guardrail moved in opposite directions?
- Which segment would you cut first, and what would that rule out?
Explain how you would architect a public data portal like CELLxGENE to allow researchers to dynamically filter and analyze billions of cells in real-time.
Medium
Approach
- Clarify what is being asked and what a complete answer would contain.
- Work from the decision backwards to the evidence you would need.
- State your assumptions explicitly before working the problem.
Follow-up
- How would you know your answer was wrong?
- What assumption would you test first?
Reduce variance on a skewed cost outcome with CUPED
Medium · cuped · variance reduction · stratification · skewed outcomes
A member-randomised digital care programme will be read on 12-month allowed PMPM, built from surviving medical_claim_line versions plus unreversed pharmacy_claim rows over member-months from member_enrollment. The outcome is heavily right-skewed. Eighty percent of members have 12 months of prior continuous coverage and therefore a usable pre-period PMPM; among those, the correlation between pre-period and post-period member PMPM is 0.45. Propose a variance reduction plan, quantify the expected gain, and say how you handle the 20 percent with no pre-period.
Approach
- Apply CUPED with the pre-period PMPM as the covariate: theta = Cov(Y, X) / Var(X), and Y_adj = Y - theta x (X - mean(X)). Because X is measured strictly before randomisation it cannot be affected by treatment, so the adjusted difference in means stays unbiased while its variance falls by the factor 1 - rho^2.
- Quantify honestly: rho = 0.45 gives 1 - 0.2025 = 0.80, a 20 percent variance reduction inside the covered stratum, which is a real but modest gain. Do not quote consumer-experimentation figures of 40 to 50 percent, which come from outcomes far more autocorrelated than health spend.
- Handle the skew before handling the covariate. Pre-specify a truncation point (a percentile of the prior year, or a fixed stop-loss amount) applied identically to both arms and fixed before unblinding, and state plainly that you are now estimating an effect on truncated cost. Report the untruncated result as a secondary so nobody can claim the cap created the finding.
- Do not impute a mean pre-period for the 20 percent with none. Stratify: run CUPED inside the covered stratum, take the unadjusted difference in the no-pre-period stratum, and combine with fixed weights; equivalently, regress with a missingness indicator and an indicator-by-covariate interaction. Both keep the estimator unbiased because the stratum is defined pre-randomisation.
- Get some of the same gain for free at assignment time by stratifying or covariate-constrained randomisation on prior-year cost decile, which removes the post-hoc analytic choices that a sceptical reviewer will otherwise interrogate.
Worked solution (30 min)
- Estimate theta on the full randomised sample as Cov(Y, X) / Var(X), using both arms because X is pre-treatment.
- Form Y_adj = Y - theta x (X - mean(X)) inside the covered stratum and compute the variance ratio 1 - rho^2 = 1 - 0.45^2 = 0.80.
- Combine strata: assuming equal per-member outcome variance, the overall variance of the treatment-effect estimate falls by about 0.8 x 20 percent = 16 percent.
- Translate to MDE: sqrt(0.84) = 0.916, so the MDE falls about 8 percent; equivalently you save about 16 percent of the sample at fixed MDE.
- Validate on a historical A/A split: CUPED should return an effect indistinguishable from zero with the claimed variance reduction.
Expected result: theta = Cov(Y, X) / Var(X); variance falls to 0.80 of baseline inside the 80 percent with a pre-period, about 16 percent overall under equal per-member variance, which shrinks the MDE by roughly 8 percent. The no-pre-period members enter as their own stratum, never as imputed means.
Verify it yourself
- theta should be close to rho x sd(Y) / sd(X); a large gap points at an outlier driving the covariance.
- The CUPED and unadjusted treatment-effect point estimates should be close; a large divergence means X is not pre-treatment or the cap is doing the work.
- Re-estimate rho after truncation, since capping both tails usually moves it.
- Run the whole procedure on a historical A/A period and confirm a null effect with narrower intervals.
Follow-up
- rho drops to 0.20 after truncation. What do you do with the CUPED plan?
- Members with no pre-period are new enrollees. What does that do to your assumption of equal per-member variance across strata?
- Would you switch the estimand from mean PMPM to a rank-based or log-scale comparison, and what would that cost you with a finance audience?
Protocol deviations fell as new sites came online
Hard · reporting lag · denominator mix · clustering
Protocol deviations per 100 completed visits across a multi-site study fell from 4.6 to 2.9 over two months while randomisation accelerated. study_subject_visit carries study_id, site_id, subject_id, visit_number, visit_name, planned_visit_date, actual_visit_date, visit_window_days, visit_status, protocol_deviation_flag, deviation_severity, data_entry_ts and site_activation_date. Operations wants to credit a retraining rollout that began in that window. Establish whether deviation behaviour changed, and deliver a version of the series that is safe to review monthly.
Approach
- Treat deviation flags as late-arriving data. Build a lag triangle of data_entry_ts minus actual_visit_date, because deviations are largely identified during monitoring review weeks after the visit, so the two most recent months are structurally undercounted in exactly the way an improvement looks.
- Hold back the months that are not developed, or apply development factors fitted on fully developed months, and restate. Do this before any explanation involving training.
- Attack the denominator. It counts visits with visit_status 'completed', while out_of_window and missed visits are excluded even though out-of-window attendance is itself a deviation in most protocols, so a site that pushes visits out of window lowers the rate twice over.
- Standardise the visit mix. Newly activated sites contribute mostly screening and baseline visits, which carry fewer procedures and fewer opportunities to deviate than later treatment visits, so compute the rate within visit_number strata and reweight to a fixed mix before comparing months.
- Split major from minor deviation_severity and report them separately. A single major deviation can remove a subject from the per-protocol analysis set while ten minor ones may not, so a pooled rate can fall while the consequential series rises.
- Before attributing anything to training, model at the right level: subjects are nested in sites, so use a random intercept per site or cluster-robust standard errors. With average cluster size m and intracluster correlation rho the variance is understated by the design effect 1 + (m - 1) times rho, which at m = 60 and rho = 0.02 is about 2.2, so naive intervals are roughly 1.5 times too narrow.
Follow-up
- Design a defensible evaluation of the retraining rollout, given it was deployed site by site and cannot be withheld.
- Your reweighted series is flat but major deviations rose. What do you escalate and to whom?
- How would you decide the holdback period, and how would you communicate a provisional number without it being quoted as final?
Suggested practice checklist
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Day 1 — Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Day 2 — Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Day 3 — Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Day 4 — The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Day 5 — Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Day 6 — Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Day 7 — Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Being wrong in public
A number you shipped turned out to be wrong, and someone had already acted on it. That is one of the most useful stories a data person can carry. What is being scored is how fast you noticed, who you told first, and what you changed in the process so the same class of error could not repeat quietly.
Tell me about a time you advocated for open-science practices or open-source data standards, even when it required significant extra engineering effort.
Medium · behavioural and stakeholder questions
Approach
- Close with what you would do differently, concretely.
- Quantify the outcome, including what you would not claim credit for.
- Pick a story where you drove the decision, not one where you observed it.
Follow-up
- What would you do differently if you ran that project again?
- What did you decide not to do, and why?
Make an honest case for your own analytical impact
Hard · impact attribution · counterfactual · self-assessment
You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.
Approach
- The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
- State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
- Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
- Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
- Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
- Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.
Follow-up
- Two other people also claim the contract measure. How do you split it?
- Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
- What is the strongest argument that your year had less impact than you just described?
Sequence three urgent requests with one analyst-week available
Medium · prioritisation · tradeoffs · stakeholder communication
Three requests land on Monday and you have one week. An actuarial team needs incurred-claims completion factors restated before a filing deadline on Thursday. A clinical programme owner wants a deterioration model refreshed because its calibration has drifted in one region. A trial operations team wants site enrolment forecasts for a portfolio review in two weeks. Each requester believes theirs is blocking. Deliverable: your sequence with the reasoning, the message you send to whoever is deprioritised, and the smaller artefact you hand each of the two you cannot fully serve.
Approach
- The probe is whether you prioritise on consequence and reversibility rather than on who asked loudest or most recently.
- Classify each request by what happens if it slips. A regulatory or contractual deadline is irreversible on its date, a drifting model is causing harm every day it keeps running, and a portfolio review can absorb a provisional number. That ordering is defensible to all three requesters because it does not depend on your preferences.
- Take the deadline-bound work first, but scope it to the minimum defensible output, because completion factors feeding a filing carry a different error tolerance than a slide.
- Do not let the drifting model simply wait. Quantify the harm cheaply by comparing calibration in the affected region against the rest, and if it is materially miscalibrated propose flagging or suppressing its output for that region within the hour rather than at the end of a refresh.
- Give each deprioritised requester something real: a provisional forecast with its uncertainty and a refresh date, or a diagnostic that tells them whether their problem is urgent. Say no explicitly with a date rather than going quiet, because silence is what produces escalation.
Follow-up
- The programme owner escalates to your manager. What do you want your manager to be able to say?
- Midweek the actuarial work needs two more full days than you estimated. What gives?
- How does your answer change if the drifting model drives a clinical outreach list rather than a report?