Interview guides · Bullfrog Ai Management

Bullfrog Ai Management Data Scientist Interview Questions & Guide 2026

SQL practice for Bullfrog Ai Management data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.

The 60-second brief

A Data Scientist at Bullfrog AI Management operates at the critical intersection of computational biology, artificial intelligence, and drug discovery. The company's core mission is to leverage machine learning to analyze complex, multi-omic, and clinical datasets to streamline drug development, identify novel therapeutic targets, and subtype patient populations. In this role, you will not just be building standard predictive models; you will be directly contributing to a proprietary AI platform that has the potential to advance the next generation of lifesaving medicines.

Scope your preparation by the data you would actually touch, because the title will not tell you. A seat that lives in event logs and weekly readouts rewards fluency in aggregation and metric definitions; a seat that owns a model in production rewards fluency in train/serve skew, retraining cadence and drift monitoring. The fastest way to find out which one you are interviewing for is to ask what the team shipped last quarter and what it gets paged about.

Focus areas

Correct for claims runout before reporting recent months
Minimise protected health information in every extract
Reconstruct drug exposure intervals from dispensing records

Overview

The work is highly dynamic, intellectually rigorous, and client-aligned. You will handle massive, heterogeneous datasets—ranging from RNA-Seq differential expression and Genome-Wide Association Studies (GWAS) to electronic health records (EHR) and clinical trial data. Because Bullfrog AI Management operates in a highly specialized domain, your models must not only perform with high accuracy but must also be biologically interpretable. You will collaborate closely with a remote team of computational biologists, software engineers, and translational medicine experts to translate raw biological complexity into actionable therapeutic insights.

Whether you join as a Senior Data Scientist or a Principal Data Scientist, you will play a dual role of technical execution and strategic leadership. You will lead individual client-facing projects, design end-to-end machine learning pipelines, and pioneer new methods in multimodal deep learning, causal inference, and natural language processing (NLP) for life sciences. Your contribution will directly influence target discovery, clinical trial design, and precision medicine initiatives, making this one of the most impactful roles in the modern biopharma landscape.

Where to focus your preparation

Treat the sections below as preparation areas and confirm the format with your recruiter.

Preparation focus

No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.

What to demonstrate

Breadth across SQL, experimentation and product reasoning
Ability to state assumptions before choosing a method

How to prepare

Drill the practice exercises below and time yourself
Prepare three quantified stories about decisions you drove

Where candidates lose points

Rates built on member counts rather than exposure

Members join and leave mid-period, so dividing events by distinct members mixes a person covered for 30 days with one covered for 365. New joiners also have artificially low observed utilisation because their claims have not arrived yet and because care takes time to initiate. Denominators must be member-months or member-years, and comparative quality measures usually need a continuous-enrolment requirement with an explicit allowable gap, stated in days.

Ignoring clustering by provider, facility, or site

Patients within a provider panel, and subjects within a trial site, are correlated on both case mix and practice pattern. Standard errors computed as if observations were independent are too narrow by roughly the design effect, 1 + (m - 1) * ICC, where m is the average cluster size. With 50 patients per provider and an intracluster correlation of only 0.02, variance is understated by about a factor of two, which manufactures significant provider differences out of noise. Cluster-robust errors or a random intercept per provider, and cluster-level randomisation when you design the test, are the corrections.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Reading experiment results before checking the arm split

Compare observed arm counts against the intended allocation ratio, not an assumed even split, and set the alarm far below the conventional 0.05: at 0.05 roughly one healthy experiment in twenty trips it, which is why sample-ratio checks usually run at p < 0.001 or stricter. The test's power scales with sample size, so it misses a real diversion on a small experiment and fires on an imbalance too small to move the estimate on a very large one. A flag means go find the assignment or logging fault before reading any outcome, not report a mismatch.

Questions & practice

Describe a scenario where you used clustering algorithms on genomic data. How did you validate the biological relevance of the resulting clusters?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Say how the offline result would be validated online before it is trusted.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

How would you design a machine learning pipeline to integrate transcriptomic (RNA-Seq) data with clinical EHR data for patient subtyping?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Four data-quality rules over corrected and cancelled lab results

Easy · data quality · record supersession · unit drift

lab_result has result_id, order_id, patient_id, loinc_code, value_numeric, value_text, units, reference_low, reference_high, specimen_collected_ts, resulted_ts, result_status (preliminary, final, corrected, cancelled), supersedes_result_id. Return one row per rule with a failing count, a failing share, and one example result_id, for these rules: more than one surviving row per (order_id, loinc_code) after resolving corrections and dropping cancelled; resulted_ts earlier than specimen_collected_ts; a numeric result whose unit is not the modal unit for its loinc_code; value_numeric and value_text both null on a row with result_status 'final'.

Approach

Resolve the correction chain before anything else. Drop result_status 'cancelled', then within each (order_id, loinc_code) keep the row whose result_id appears in no other row's supersedes_result_id. That terminal node is the surviving value and the rule holds even if a correction is loaded with a backdated timestamp.
Build each rule as a boolean mask over a named frame rather than as a filtered copy, so the failing share has an explicit, per-rule denominator. Rule three is eligible only on rows with a non-null value_numeric; rule four only on final rows.
For the unit rule, derive the modal unit per loinc_code from the data instead of hardcoding an expected unit. The failure being hunted is one analyte reported in two units whose values differ by a fixed factor, which no range check catches because both sets of numbers look plausible.
Emit a tidy frame with columns rule, eligible_rows, failing_rows, failing_share, example_result_id. A printed report cannot be diffed between runs; a frame can be stored and alerted on.
Order the output by failing_share descending so the run has a lead finding rather than four equal-weight lines.

Worked solution (25 min)

Drop cancelled rows, then compute the set of result_ids referenced by supersedes_result_id and keep rows not in that set.
Rule one: group the survivors by (order_id, loinc_code) and count rows above 1.
Rule two: mask on resulted_ts < specimen_collected_ts over rows where both timestamps are present.
Rule three: compute the mode of units per loinc_code over rows with non-null value_numeric, then mask rows whose unit differs from it.
Rule four: mask rows with result_status 'final' and both value columns null. Assemble the four results into one frame with counts, shares and an example id.

Expected result: A four-row frame. After the resolver, rule one should report zero failing groups on a well-formed extract; a non-zero count there means the supersession chain branches or a duplicate load occurred, and that is the finding to lead with rather than a percentage.

Verify it yourself

Running the resolver a second time on its own output is a no-op, and exactly one row survives per (order_id, loinc_code).
No result_id appearing in any supersedes_result_id survives the resolver.
Seed one row with a converted unit for an analyte otherwise reported in a single unit and confirm rule three catches exactly that row and nothing else.
Each rule's failing_share uses its own eligible_rows, so a rule with no eligible rows returns share 0 rather than a division error.

Follow-up

The modal-unit rule fires on 8 percent of one analyte. How do you decide whether that is a unit-conversion bug or a second legitimate assay?
What would you add to catch a value that is inside its reference range but physiologically impossible?
Which of these four rules should block a downstream pipeline and which should only alert, and why?

Find inpatient encounters with no resulted lab during the stay

Medium · anti-join · null semantics · event matching

encounter holds encounter_id, patient_id, encounter_type, admit_ts, discharge_ts and facility_id. lab_result holds result_id, order_id, patient_id, encounter_id (NULL for outpatient standing orders), loinc_code, specimen_collected_ts, resulted_ts and result_status. Return every inpatient encounter discharged in 2025 with no final or corrected lab_result linked to it. A colleague wrote WHERE encounter_id NOT IN (SELECT encounter_id FROM lab_result) and got zero rows back. Say why, write the correct query, then say what changes when labs must instead be matched on patient_id with the specimen collected between admit_ts and discharge_ts.

Approach

Name the mechanism rather than the symptom. x NOT IN (subquery) expands to x <> y1 AND x <> y2 AND ..., and a single NULL y makes that chain UNKNOWN, never TRUE. Since lab_result.encounter_id is nullable by design for standing orders, this query can only ever return zero rows.
Rewrite as NOT EXISTS, or LEFT JOIN with an IS NULL filter. Both treat a NULL key as no match rather than as unknown. Adding IS NOT NULL inside the NOT IN subquery also works, but NOT EXISTS is the habit that survives someone making another column nullable later.
Put the result_status predicate inside the correlated subquery or the ON clause, not in an outer WHERE. Outside, it turns the anti-join back into an inner join and an encounter whose only labs were cancelled disappears instead of qualifying.
For the timestamp variant, match on patient_id with specimen_collected_ts inside the stay rather than resulted_ts. A specimen drawn an hour before discharge can result the next day, and anchoring on resulted_ts would wrongly call that stay lab-free.
Restrict to encounter_type 'inpatient' and discharge_ts in 2025, and report open encounters with a NULL discharge_ts as a separate count instead of letting the filter swallow them.

Follow-up

Write the LEFT JOIN form and say exactly where the result_status predicate must sit for the two forms to agree.
How would you separate encounters whose only labs were cancelled from encounters with no lab rows at all?
Some encounters carry member_id NULL because the person index did not match. What does that do to a payer-side version of this measure?

Compute allowed PMPM by incurred month without span fan-out

Hard · fan-out · many-to-many · runout

Report allowed PMPM by incurred month and product_type for 2025 from medical_claim_line (member_id, claim_id, claim_version, frequency_code, service_start_date, allowed_amount, claim_status), pharmacy_claim (member_id, fill_date, allowed_amount, reversal_flag, reversed_fill_id), member_enrollment (span_id, member_id, product_type, effective_date, termination_date) and member_risk_score (member_id, score_year, prospective_risk_score), which holds at most one row per member per score year. The numerator is allowed_amount on surviving claim versions plus unreversed fills, attributed to the product_type in force on the service or fill date. The denominator is member-months. Also return a risk-adjusted PMPM for each cell: allowed PMPM divided by the member-month-weighted mean prospective_risk_score for score_year 2025 over the members making up that cell's denominator. A colleague joined claim lines to member_enrollment on member_id alone and the annual total came back 1.4 times the finance figure. Diagnose that, then produce the correct monthly series.

Approach

Name the defect precisely. member_enrollment is one row per span, so joining on member_id alone is one-to-many and duplicates every claim line once per span that member ever held. The inflation factor is the spend-weighted mean span count per member, which is why 1.4 is plausible and why it is not a constant you can divide back out.
Make the join functional: member_id AND service date between effective_date and COALESCE(termination_date, DATE '9999-12-31'). That yields at most one span per line unless spans genuinely overlap, in which case pick a documented precedence rule, most recently effective wins being the usual one, and prove uniqueness with a duplicate check rather than asserting it.
Aggregate numerator and denominator separately and join them on (month, product_type). Computing both sides across one join is the second route to inflation, because the member-month denominator fans out over claim lines just as easily as the dollars fan out over spans.
Clean each numerator source on its own terms: surviving claim versions with voids removed and claim_status 'paid' on the medical side, reversal rows and their originals removed on the pharmacy side.
Apply completion before anyone reads a trend. State the paid-through date and either withhold the three most recent incurred months or apply completion factors, since the lag differs by service type and the incomplete tail reads as a cost improvement and a utilisation drop at once.
Risk-adjust last, and weight the score the way the denominator is weighted. The divisor for a cell is SUM(member_months * prospective_risk_score) / SUM(member_months) over that cell's members, not a plain AVG over distinct members: a member covered two months must not count the same as one covered twelve. member_risk_score is member-level, so join it after the member-month grain is fixed, or it becomes a second fan-out.
State the rule for members with no 2025 member_risk_score row: keep them in member_months, leave them out of the weighted mean, and publish the scored share of member-months beside the adjusted series. A thin score table moves the adjusted number while the raw one sits still, and without that share nobody can tell the two apart.

Worked solution (45 min)

CTE med: surviving claim version per claim_id, voids dropped, claim_status 'paid', service_start_date in 2025.
CTE rx: fills with reversal_flag FALSE and not referenced by any reversed_fill_id, via NOT EXISTS, fill_date in 2025.
Attribute each numerator row to one span with the date-bounded join, after asserting that row counts before and after the join are equal.
CTE denom: member-months by month, product_type and member_id from spans alone, with fractional partial months and overlapping spans collapsed first.
Join numerator to denominator aggregated on (incurred_month, product_type) and divide for allowed_pmpm.
Join denom to member_risk_score on member_id for score_year 2025, compute the cell divisor as SUM(member_months * prospective_risk_score) / SUM(member_months) over scored members, carry the scored share of member-months, and divide allowed_pmpm by the divisor for risk_adjusted_pmpm.
Withhold the three most recent incurred months and label the paid-through date on every row.

Expected result: One row per (incurred_month, product_type) with allowed_medical, allowed_pharmacy, member_months, allowed_pmpm, mean_risk_score, scored_member_month_share and risk_adjusted_pmpm, covering January through September when the paid-through date is year end and three months are withheld. Row count of the claim-line set is identical before and after the enrollment join.

Verify it yourself

Claim-line row count before and after the span join is unchanged. An increase is fan-out; a decrease is a service date with no covering span, which is a real finding about eligibility gaps.
SUM(allowed_amount) across product_types for a month equals the ungrouped monthly total computed without touching member_enrollment at all.
member_months for a month never exceeds the count of distinct members covered at any point in that month.
mean_risk_score for a cell lies between the minimum and maximum prospective_risk_score among its scored members, and risk_adjusted_pmpm equals allowed_pmpm exactly when that mean is 1.0. If the weighted mean equals a plain AVG over distinct scored members in every cell, the member-month weights were never applied.
A downward step in the last reported months means the runout holdback was not applied; the withheld tail should not be visible in the series at all.

Follow-up

A member holds two overlapping spans under different product_types on the service date. Which one gets the dollar, and how do you keep the member-month denominator consistent with that choice?
Write the check that proves the span join is one-to-one rather than assuming it.
Finance reports on paid_date and you reported on service_start_date. Reconcile one month between the two bases.

Can you explain the statistical considerations and quality control steps you would implement when conducting a Genome-Wide Association Study (GWAS)?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

What strategies do you use to address the "curse of dimensionality" when working with high-dimensional genomic datasets where the sample size ($N$) is much smaller than the number of features ($P$)?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

Reduce variance on a skewed cost outcome with CUPED

Medium · cuped · variance reduction · stratification · skewed outcomes

A member-randomised digital care programme will be read on 12-month allowed PMPM, built from surviving medical_claim_line versions plus unreversed pharmacy_claim rows over member-months from member_enrollment. The outcome is heavily right-skewed. Eighty percent of members have 12 months of prior continuous coverage and therefore a usable pre-period PMPM; among those, the correlation between pre-period and post-period member PMPM is 0.45. Propose a variance reduction plan, quantify the expected gain, and say how you handle the 20 percent with no pre-period.

Approach

Apply CUPED with the pre-period PMPM as the covariate: theta = Cov(Y, X) / Var(X), and Y_adj = Y - theta x (X - mean(X)). Because X is measured strictly before randomisation it cannot be affected by treatment, so the adjusted difference in means stays unbiased while its variance falls by the factor 1 - rho^2.
Quantify honestly: rho = 0.45 gives 1 - 0.2025 = 0.80, a 20 percent variance reduction inside the covered stratum, which is a real but modest gain. Do not quote consumer-experimentation figures of 40 to 50 percent, which come from outcomes far more autocorrelated than health spend.
Handle the skew before handling the covariate. Pre-specify a truncation point (a percentile of the prior year, or a fixed stop-loss amount) applied identically to both arms and fixed before unblinding, and state plainly that you are now estimating an effect on truncated cost. Report the untruncated result as a secondary so nobody can claim the cap created the finding.
Do not impute a mean pre-period for the 20 percent with none. Stratify: run CUPED inside the covered stratum, take the unadjusted difference in the no-pre-period stratum, and combine with fixed weights; equivalently, regress with a missingness indicator and an indicator-by-covariate interaction. Both keep the estimator unbiased because the stratum is defined pre-randomisation.
Get some of the same gain for free at assignment time by stratifying or covariate-constrained randomisation on prior-year cost decile, which removes the post-hoc analytic choices that a sceptical reviewer will otherwise interrogate.

Worked solution (30 min)

Estimate theta on the full randomised sample as Cov(Y, X) / Var(X), using both arms because X is pre-treatment.
Form Y_adj = Y - theta x (X - mean(X)) inside the covered stratum and compute the variance ratio 1 - rho^2 = 1 - 0.45^2 = 0.80.
Combine strata: assuming equal per-member outcome variance, the overall variance of the treatment-effect estimate falls by about 0.8 x 20 percent = 16 percent.
Translate to MDE: sqrt(0.84) = 0.916, so the MDE falls about 8 percent; equivalently you save about 16 percent of the sample at fixed MDE.
Validate on a historical A/A split: CUPED should return an effect indistinguishable from zero with the claimed variance reduction.

Expected result: theta = Cov(Y, X) / Var(X); variance falls to 0.80 of baseline inside the 80 percent with a pre-period, about 16 percent overall under equal per-member variance, which shrinks the MDE by roughly 8 percent. The no-pre-period members enter as their own stratum, never as imputed means.

Verify it yourself

theta should be close to rho x sd(Y) / sd(X); a large gap points at an outlier driving the covariance.
The CUPED and unadjusted treatment-effect point estimates should be close; a large divergence means X is not pre-treatment or the cap is doing the work.
Re-estimate rho after truncation, since capping both tails usually moves it.
Run the whole procedure on a historical A/A period and confirm a null effect with narrower intervals.

Follow-up

rho drops to 0.20 after truncation. What do you do with the CUPED plan?
Members with no pre-period are new enrollees. What does that do to your assumption of equal per-member variance across strata?
Would you switch the estimand from mean PMPM to a rank-based or log-scale comparison, and what would that cost you with a finance audience?

Allowed PMPM fell fourteen percent in three months

Easy · claims runout · completion factors · incurred vs paid

Allowed PMPM in the monthly series fell from $412 to $354 over the three most recent incurred months, and leadership wants to announce the saving. You have medical_claim_line (allowed_amount, service_start_date, received_date, paid_date, claim_id, claim_version, frequency_code, place_of_service_code), pharmacy_claim (allowed_amount, fill_date, reversal_flag) and member_enrollment coverage spans. The chart carries no paid-through date. Work out how much of the fall survives completion, and hand back a restated series with an explicit paid-through date and the completion factors you applied.

Approach

Get the paid-through date first and put it on the chart. Without it the series is uninterpretable, because every incurred month is a different age.
Build a lag triangle: for each incurred month, cumulative allowed_amount by the number of months between service_start_date and paid_date, on surviving claim versions only (drop frequency_code 8 voids and keep the highest claim_version per claim_id).
Estimate development factors separately for pharmacy fills, professional lines and inpatient facility lines. Pharmacy adjudicates within days, professional within weeks, inpatient facility over months, so one blended factor understates the correction on the newest month and overstates it on the oldest.
Gross each of the last three incurred months up by the reciprocal of its cumulative development factor, using factors fitted on months that have already fully run out.
Rebuild the denominator independently as member-months from coverage spans. Member-months are complete on day one, so if the denominator also shows a lag pattern the enrollment file is late, which is a different bug with a different fix.
Restate the series, mark the last three months as estimated, and report the residual movement after completion as the only part worth investigating.

Follow-up

The completed series still shows a three percent fall. What do you look at next, and in what order?
How would you detect that the fall is a mix shift toward cheaper services rather than lower volume?
A contract settles on this number at a fixed paid-through date. What do you owe the other party about the estimate you just made?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

How do you handle batch effects and normalization issues when combining RNA-Seq datasets from multiple different clinical trials or laboratories?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.
Quantify the outcome, including what you would not claim credit for.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

Defend a null result against a programme sponsor

Medium · regression to the mean · difference-in-differences · stakeholder management

A care management programme enrolled the top 1 percent of members by prior-year allowed spend. The sponsor's deck shows allowed PMPM for enrollees falling 34 percent from the year before enrollment to the year after, and asks for budget to triple the programme. You rebuild the evaluation with a concurrent comparison group selected by the identical spend rule in the same period. The difference-in-differences estimate is a 3 percent reduction with a confidence interval spanning zero. Deliverable: how you present this, to whom, in what order, and what you propose next.

Approach

Name the probe: whether you can deliver a finding that costs somebody their programme without softening it into uselessness or creating an adversary who routes around you next time.
Lead with the mechanism, not the verdict. Show the comparison group's own unadjusted drop, which will be large, because a cohort selected on an extreme of the outcome regresses toward the mean whether or not anyone intervenes. The sponsor's 34 percent is mostly that, and it is a property of the selection rule rather than a criticism of their clinicians.
Give the sponsor the finding privately before it appears in any deck their leadership sees. Being surprised in a room is what turns a methods disagreement into a political one.
State the estimate with its interval and say what it rules out as well as what it fails to establish. A 3 percent point estimate whose interval crosses zero is not evidence of no effect; it is insufficient power to separate a modest effect from none, and those are different claims.
Arrive with a design rather than only an objection. Propose a regression discontinuity at the enrollment threshold if the rule is applied sharply, or a randomised rollout across the next wave of eligible members, and state the sample size needed to detect the effect size the sponsor believes in.

Follow-up

The sponsor says withholding the programme from a comparison group is unethical. What do you propose instead?
Leadership wants one number for the board next week. What do you give them?
What result would change your mind and make you believe the 34 percent?

Own an analysis that shipped wrong and was acted on

Easy · retrospective · ownership · data quality

Describe an analysis you delivered that was wrong, where someone acted on it before the error surfaced. Pick something with a mechanism you can name, not a typo. Cover what the number claimed, what was actually true, how much time passed before it was caught, who caught it, and what you told the people who had already acted. Deliverable: a five-minute account that ends with the specific control you added afterwards, plus one occasion since when that control has fired.

Approach

Read the probe: the interviewer is testing whether you detect your own errors and whether the correction was structural. Pick an error with a nameable mechanism (a denominator built from distinct members instead of member-months, a series read before claims runout completed, a cohort selected on the outcome) so the story has a diagnosis rather than a mood.
Open with the decision that was made on your number, not with the bug. The severity of an analytic error is measured in the action it caused: a rate filing, a staffing plan, a programme expansion.
State the counterfactual number. 'The true figure was 14.1 admissions per 1,000 member-years, not 9.8' is a fact; 'it was off by a lot' is not.
Describe the detection path honestly, including whether you found it or somebody else did. Say how long it took, because time to detection is the part you actually control next time.
Close on the control, and make it something that protects a reader who has never met you: a unit test on the denominator, a paid-through date stamped on every chart, a required exclusions block in the spec template. Name one later date it caught something.

Follow-up

What would have had to be true for you to catch this before delivery rather than after?
Did the stakeholder change how much they trust your work, and did you want them to?
Give an example of a control you added that turned out to be the wrong control.