Interview guides · Castlight

Castlight Data Scientist Interview Questions & Guide 2026

Case practice for Castlight data scientist candidates: scope a vague prompt into a measurable question before you analyze it.

The 60-second brief

As a Data Scientist at Castlight, you sit at the intersection of healthcare analytics, product innovation, and user behavior. Your core mission is to transform complex health and benefits data into actionable insights that empower users to make informed healthcare decisions while driving measurable business value. This role requires a balance of rigorous statistical thinking, product sense, and engineering execution to build scalable data products that impact millions of members navigating their health benefits.

Nearly every loop contains a round whose deliverable is a recommendation to someone non-technical. Practise stating a conclusion, the confidence attached to it, and the cost of being wrong in each direction, because that triple is the artifact being graded.

Focus areas

Build member-month denominators from overlapping coverage spans
Risk-adjust outcomes before comparing providers or panels
Reconstruct drug exposure intervals from dispensing records

Overview

You will collaborate closely with product managers, software engineers, and clinical experts to design, build, and evaluate features across Castlight digital health platforms. Whether you are analyzing engagement trends in healthcare navigation tools, predicting cost utilization patterns, or designing personalization algorithms, your work directly shapes the user experience. You will own the analytical lifecycle from exploratory data analysis to productionizing models and measuring their long-term impact on user outcomes and business metrics.

This role offers a compelling mix of intellectual challenge and social impact. Operating in the digital health space means dealing with unique data complexities, strict privacy standards, and the mandate to deliver intuitive solutions to complex problems. You will need to thrive in fast-paced environments where data ambiguity is common, and where your ability to communicate complex findings to non-technical stakeholders is just as important as your modeling skills.

The Castlight interview process

Castlight candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Outreach

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Screening Round

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Technical Rounds

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Final Interviews

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Treating clinical measurements as missing at random

A lab result, a vital sign, or a screening exists because someone ordered it, and ordering tracks suspicion of disease, visit frequency, and site workflow. Imputing the mean or dropping incomplete rows biases the population estimate and can flip the sign of an association, because the untested are systematically healthier or systematically disengaged. The presence indicator is often more predictive than the value, which is a warning sign rather than a feature win: a model that learns test ordering will not transfer to a site with different protocols.

Immortal time in adherence, treatment, and enrolment definitions

Classifying members as adherent, treated, or programme-enrolled requires them to survive and stay covered long enough to accumulate the defining events. That guaranteed event-free interval is assigned to the exposed group, so the exposure looks protective for reasons that have nothing to do with the treatment. Adherence studies are the classic case: measuring 12-month proportion of days covered and then comparing mortality builds survival into the exposure definition. Use time-varying exposure or a landmark analysis with the classification window excluded from follow-up.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Averaging per-user rates to produce a population rate

Decide which quantity you want: the mean of per-user ratios and the ratio of summed numerator to summed denominator are different estimands, and heavy users dominate one but not the other. For a ratio metric, aggregate numerator and denominator separately and use the delta method for its variance.

Questions & practice

How do you prevent overfitting when working with high-dimensional behavioral tracking data?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.
Check what information would not exist at prediction time, and exclude it.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

Sessionise transfer chains into episodes, then count readmissions

Hard · sessionisation · cumsum grouping · readmission

encounter has encounter_id, patient_id, facility_id, encounter_type, admit_ts, discharge_ts, admission_type, discharge_disposition, is_planned_admission, principal_diagnosis_code. Chain acute-to-acute transfers into episodes of care: an inpatient encounter discharged as transfer_acute, followed by another inpatient encounter for the same patient at a different facility admitting within 24 hours of that discharge, belongs to the same episode. Then compute the 30-day unplanned readmission rate over episodes, excluding index episodes that are planned, that end in expired, hospice or against medical advice, or that are still open. Return encounter_id to episode_id, plus the rate.

Approach

Restrict to inpatient encounters for the chaining step and sort by patient_id, admit_ts. Observation and emergency encounters are not acute inpatient stays and pooling them changes both the chain and the denominator.
Build a boolean continues flag: the previous row is the same patient, its discharge_disposition is transfer_acute, its facility_id differs from this row's, and admit_ts minus previous discharge_ts is between 0 and 24 hours. Episode id = cumsum of the negation of that flag. This is the vectorised form of a sessionisation loop and is why the pattern generalises to any event stream.
Collapse to episode grain taking first admit_ts, last discharge_ts, last discharge_disposition, and any() of is_planned_admission. The episode inherits its exit state, not its entry state, which is why a transfer chain ending at home is one home discharge rather than two transfers.
Apply index exclusions at episode grain. Dropping episodes that end in death is not optional: a dead patient cannot be readmitted, so leaving them in inflates the denominator and depresses the rate for exactly the panels caring for the sickest members.
For each surviving index episode, find the next unplanned acute inpatient episode for that patient admitting within 30 days of the index discharge_ts. Count events at episode grain, since one patient can contribute several index episodes. Report the raw rate and say plainly that it is not comparable across panels without risk adjustment, which is what the observed-over-expected form exists for.

Follow-up

discharge_ts is null on an open stay in the middle of a chain. What does your flag do, and what should it do?
A patient is transferred out and then back to the originating facility within 24 hours. Should that chain, and does your facility_id condition handle it?
Is a readmission itself eligible to serve as a later index episode? Say what you chose and what the choice does to the rate.

Build a claims lag triangle and complete recent months

Medium · chain ladder · runout · pivot

medical_claim_line has claim_id, claim_version, frequency_code, service_start_date, paid_date, allowed_amount, claim_status. Using 24 fully mature incurred months, build a development triangle of allowed dollars by incurred month and payment lag in whole months, derive cumulative completion factors by chain ladder, then estimate ultimate allowed for the three most recent incurred months at a stated paid-through date. Deduplicate to surviving claim versions before building the triangle. Return incurred_month, paid_to_date, completion_factor, estimated_ultimate.

Approach

Deduplicate to surviving claim versions and keep paid lines first. A triangle built on all versions develops on adjustment churn rather than on payment timing, and replacements arrive late, so the distortion concentrates in exactly the tail you are trying to estimate.
Compute lag as a whole-month difference between incurred month and paid month, not as day difference divided by 30. Ragged month lengths otherwise shuffle identical claims between lag buckets depending on which month they fall in.
Pivot to incurred_month by lag, cumulate along lag, then form age-to-age factors as the ratio of the column L+1 total to the column L total, summed over incurred months mature enough to have both. Volume weighting is the chain ladder; a simple mean of per-month ratios lets one low-volume month dominate the factor.
Chain the age-to-age factors from each lag to ultimate and invert: completion factor at lag L is the reciprocal of the product of factors from L onward. Apply as estimated_ultimate = paid_to_date / completion_factor.
State the assumption you have just made, which is that the development pattern is stable. It is not, after a claims-system migration, a network change or a processing backlog, and the most recent month's factor is the least reliable because it rests on the fewest observations while carrying the largest adjustment.

Worked solution (35 min)

Deduplicate to surviving versions, drop voids, keep claim_status 'paid', and derive incurred_month from service_start_date and paid_month from paid_date.
lag = (paid_month.year - incurred_month.year) x 12 + (paid_month.month - incurred_month.month); drop negative lags and investigate them separately.
Pivot to a triangle of summed allowed_amount, cumulate along the lag axis, and mask cells past the paid-through date so no cell contains a future payment.
Compute volume-weighted age-to-age factors per lag over the mature rows, then chain them into cumulative completion factors.
Apply the factor for each recent month's current maturity to its paid_to_date and return the four columns.

Expected result: Three rows. Completion is well below 1 at short lag and approaches 1 in the tail; for a mixed medical book a one-month-old incurred month commonly sits far from complete, while a twelve-month-old one is close. Exact values are book-specific, so the reviewable property is monotonicity and the holdout test below, not the numbers themselves.

Verify it yourself

Cumulative completion factors are non-decreasing in lag and converge toward 1 at the tail.
Hold out one fully mature incurred month, apply the factors at each of its intermediate maturities, and confirm each estimate lands within a few percent of its known total.
estimated_ultimate is greater than or equal to paid_to_date for every row.
A pharmacy-only triangle completes far faster than an inpatient-facility-only triangle, which confirms the lag computation is reading real payment timing.

Follow-up

A processing backlog means the last two months developed slower than history. What does your estimate do, and how would you detect that before reporting?
Pharmacy, professional and inpatient facility develop on different clocks. How do you split the triangle, and what breaks if the service mix shifts?
What happens to a PMPM series if someone reports an unadjusted recent month, and in which direction does the error point?

Given a table of user session logs, write a query to identify the top three most frequent user pathways through our application.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Write a query using SQL window functions to calculate the running total of user interactions with a specific health benefit over a 30-day period.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Turn overlapping coverage spans into fractional member-month denominators

Medium · intervals · gaps and islands · denominators

member_enrollment holds span_id, member_id, plan_id, product_type, effective_date and termination_date, which is NULL while the span is active. A member can hold several spans in a year, and back-dated plan changes make spans overlap. Return one row per calendar month of 2025 with total member-months, where a member contributes covered days in the month divided by that month's length, and a day covered by two overlapping spans counts once. Both span endpoints are inclusive. Collapse overlaps with window functions before joining a month calendar, and do not assume one span per member.

Approach

Normalise the open end first. COALESCE(termination_date, DATE '9999-12-31') or clip to the reporting end, because a NULL termination_date makes every BETWEEN comparison unknown and drops exactly the members who are still covered.
Merge overlapping spans per member before the calendar join. Order by effective_date, carry a running MAX of prior termination_date with a frame ending one row before, and open a new island where effective_date exceeds that running max plus one day, which keeps adjacent spans contiguous rather than splitting them.
Clip each merged island to the reporting year, then cross join to a twelve-row month calendar and keep pairs that intersect.
Compute covered days as LEAST(island_end, month_end) minus GREATEST(island_start, month_start) plus one, and divide by the actual length of that month rather than a flat 30, so February and the 31-day months are not distorted.
Sum the fractions by month. Merging after the calendar join also works but forces a DISTINCT over member-days, which stops scaling at population size.

Worked solution (30 min)

CTE bounded: clip spans to 2025-01-01 through 2025-12-31 with COALESCE on termination_date, discarding spans that do not intersect the year.
CTE merged: gaps-and-islands per member using LAG and a running MAX over preceding rows to collapse overlapping and adjacent spans.
CTE months: twelve rows of month_start, month_end and days_in_month.
Join merged to months on interval intersection and compute covered_days per pair.
SUM(covered_days / days_in_month::numeric) grouped by month, ordered by month.

Expected result: Twelve rows of (month, member_months) with fractional values. Any single member contributes at most 1.0 to a month and at most 12.0 across the year. A member covered 2025-01-01 through 2025-02-20 contributes 1.0 in January and 20/28 = 0.714 in February.

Verify it yourself

For a member with spans 2025-01-01..2025-03-31 and 2025-03-15..2025-06-30, March contributes exactly 1.0, not 1.5.
Total member-months is at most 12 times COUNT(DISTINCT member_id) and at least the count of members covered the whole year.
Forcing every termination_date to NULL raises the total. If it drops to zero, the NULL handling is missing.

Follow-up

Where does a continuous-enrolment filter with an allowable 45-day gap belong, and how does that change the island boundary rule?
One human appears under two member_ids after a master person index merge. What does that do to member-months, and how would you detect it?
How do you treat a span that terminates on the day it starts, and what does that row usually represent?

How would you determine if a decline in user engagement is due to a seasonal trend or a bug in the product?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

What metrics would you track to evaluate the long-term value of a personalized wellness recommendation engine?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

How would you approach building a predictive model for healthcare cost utilization, and what evaluation metrics would you prioritize?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

How would you design a metric to measure the success of a newly launched health navigation feature?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

Explain the concept of statistical power and how you calculate the required sample size for an experiment with low baseline conversion rates.

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Name the randomisation unit first; it decides the variance and what the test can detect.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

How would you design an A/B test for a new pricing or incentive structure within our digital benefits platform?

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Name the guardrails that would stop a launch even on a positive primary result.
Say whether units interfere with each other, and switch design if they do.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

We notice a sudden drop in weekly active users on our mobile app. How would you investigate and diagnose the root cause?

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.
Work from the decision backwards to the evidence you would need.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Propose a proxy for care that never happened

Hard · proxy metrics · informative missingness · bias direction · churn

Leadership asks for a metric for unmet need: care a member should have received and did not. Claims record billing events, so no claim and no need look identical in the data. Using member_enrollment, medical_claim_line, encounter, lab_result and pharmacy_claim, propose the best available proxy, state the direction of its bias and what drives the magnitude, and name one decision the proxy is good enough to support and one it is not. Say plainly whether the quantity itself is identified by this data.

Approach

Start by refusing the direct metric and saying why in one sentence: every table here records an action someone took and billed for, so the absence of a row is the absence of a billed action, and no combination of these tables distinguishes a member who needed nothing from a member nobody reached. The quantity is not identified.
Build the proxy only where non-treatment leaves a downstream trace. Two families work: ambulatory care sensitive admissions per 1,000 member-years, where an inpatient stay for short-term diabetes complications, heart failure, adult asthma, bacterial pneumonia or urinary infection implies outpatient care that did not land; and guideline gap measures on an identified cohort, for example members with a qualifying diagnosis on at least two medical_claim_line rows at least 30 days apart, continuously enrolled, with no matching lab_result loinc_code and no matching procedure_code in the last 12 months.
State the bias direction without hedging. Both families require prior contact: a gap measure needs a diagnosis, which needs someone to have coded it, and an admission needs the member to reach a facility. The proxy therefore undercounts unmet need, and it undercounts it most in exactly the population that has the most of it, namely members with no contact at all.
Name the magnitude drivers so the number can be read over time: coverage churn (short spans fail continuous-enrolment filters and drop out of the denominator), coding intensity (more diagnosis coding enlarges the identified cohort and can raise measured gaps with no change in care), and site-level ordering habits in lab_result, since a test exists because somebody chose to order it.
Pair it with a denominator-side companion that moves the opposite way: the count and share of continuously enrolled members with zero claims of any kind in 12 months. Reporting it beside every unmet-need number blocks the improvement that comes from losing the least-observed members.
Close with the usability boundary: the proxy supports relative prioritisation of outreach across panels within the same product and region, where the bias is shared; it does not support a year-over-year claim that unmet need fell, because churn and coding intensity move the proxy on their own.

Worked solution (40 min)

Write the identification argument in three sentences: what generates rows in each table, what a missing row can mean, and why no filter separates the two meanings.
Specify one gap measure end to end: cohort definition with the two-claims-30-days-apart rule, continuous-enrolment requirement with the allowable gap in days, the expected event defined by loinc_code in lab_result or procedure_code in medical_claim_line, and the 12-month window.
Specify the ambulatory care sensitive admission rate per 1,000 member-years from encounter and member_enrollment, with the diagnosis list named and planned admissions excluded.
Write the bias paragraph: direction, the three magnitude drivers, and the specific scenario in which the proxy improves while care worsens.
Define the zero-utiliser companion metric and state that it is published on the same page as the proxy, not in an appendix.
Write the two-line usability boundary naming the one decision it supports and the one it does not.

Expected result: A written proposal stating that unmet need is not identified by claims or clinical data; offering a two-part proxy of guideline gap rates on a diagnosis-identified, continuously enrolled cohort plus ambulatory care sensitive admissions per 1,000 member-years; declaring the bias as a systematic undercount concentrated in members with no prior contact; naming churn, coding intensity and site ordering habits as the magnitude drivers; pairing the proxy with the zero-utiliser share; and limiting its use to cross-panel prioritisation within a product and region rather than year-over-year claims.

Verify it yourself

Recompute the gap rate with the continuous-enrolment requirement removed. A large move confirms churn is driving the denominator, which is the bias the write-up claims.
Split the identified cohort by number of qualifying diagnosis claims. If the gap rate differs sharply between members at exactly two claims and members at many, the cohort definition is picking up coding intensity as much as disease.
Compare gap rates across sites with known differences in lab ordering volume. Divergence there is the ordering-habit driver, not a care difference.
Check that the zero-utiliser share and the gap rate are not moving in opposite directions unexplained; if they are, the proxy is improving by losing members.

Follow-up

Your gap measure improves 3 points year over year and the zero-utiliser share rose 1.5 points. What do you tell leadership?
Would adding the presence indicator for lab orders as a model feature help or hurt, and what happens when that model is deployed at a site with different ordering protocols?
What data outside these five tables would actually identify unmet need, and what would it cost to get it?

Pooled persistency fell but no one lost coverage

Medium · interval sql · mix shift · cohort seasonality

Pooled 12-month enrollment persistency fell from 0.78 to 0.66 for the cohort anchored in a single month. member_enrollment carries span_id, member_id, plan_id, product_type, effective_date, termination_date, termination_reason_code, dual_eligible_flag and region_code. The metric allows at most one gap of 45 days or fewer. Product mix in that anchor month was not stable. Determine how much of the fall is real disenrollment rather than span bookkeeping or mix, and deliver a persistency series a contract owner could act on.

Approach

Inspect the query's unit of analysis first. Spans are not members, and a benefit or plan change closes one span and opens the next on the same day, so any logic that treats a non-null termination_date as loss of coverage counts an administrative event as churn.
Rebuild member-level coverage by unioning all spans per member and merging adjacent or overlapping ones, then apply the 45-day gap rule to the merged intervals rather than per span.
Cut the terminations by termination_reason_code. plan_change and eligibility_redetermination have entirely different meanings for a contract, and death is not churn you can address.
Split by product_type and stop pooling. Commercial open enrollment and managed medicaid redetermination produce structurally different churn on different calendars, and the pooled average of the two is a number that describes nobody.
Compare the anchor month to the same month in the prior year rather than to the adjacent month, because enrollment is strongly seasonal and an anchor month sitting on an open-enrollment or redetermination boundary is not comparable to its neighbour.
Apply the Kitagawa decomposition across product_type to separate how much of the 12 point fall is mix, meaning the anchor cohort simply contained more of a high-churn product, from how much is within-product deterioration.

Follow-up

Why 45 days, and what would change if the allowable gap were 30 or 60?
A member terminates and re-enrolls 90 days later under a different plan_id. Are they retained? Defend your answer against the opposite one.
What guardrail would catch someone improving persistency by declining to enroll members likely to churn?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Being wrong in public

A number you shipped turned out to be wrong, and someone had already acted on it. That is one of the most useful stories a data person can carry. What is being scored is how fast you noticed, who you told first, and what you changed in the process so the same class of error could not repeat quietly.

Describe a time when you took ownership of an ambiguous problem and drove it from exploratory analysis to a deployed solution.

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Close with what you would do differently, concretely.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Push back on an over-broad extract without blocking the work

Hard · minimum necessary · de-identification · governance

A researcher asks for a one-off extract covering 12,000 members: member_id, date of birth, five-digit residential postal code, full lab_result rows including value_text, and clinical note text, held on their laptop for a retrospective analysis of one lab analyte over time. They describe it as de-identified because names are excluded. A flat refusal is not an acceptable answer. Deliverable: the column list you will actually provide with a justification per retained field, the sentence explaining why free text defeats an enumerated-identifier approach, your small-cell rule for the published stratification, and what you deliver this week.

Approach

The probe is whether privacy is a design skill for you or a compliance reflex. Flat refusal and unquestioned fulfilment both fail, for opposite reasons.
Reconstruct the analysis from the research question rather than from the request. A longitudinal trend in one analyte needs a stable pseudonymous subject key, the analyte's loinc_code, value_numeric, units, the reference bounds, and a date. It does not need names, notes, or a precise birth date.
Replace each identifier with the least precise version that still answers the question: a salted per-study pseudonym instead of member_id, age in years or an age band instead of date of birth, a consistent per-subject date offset if only intervals matter, and a coarser geography than five-digit postal code.
Explain the free-text problem in one sentence. Removing an enumerated list of identifier fields does not de-identify a note, because notes carry names, relationships, employers, dates and rare clinical detail in prose, so only a reviewed extraction into structured fields or a formal statistical determination addresses it.
Set the publication rule before the analysis runs. Suppress cells below a stated threshold and suppress complements too, because a reader holding a row total and all but one cell recovers the suppressed count by subtraction.
Keep them moving. Deliver the reduced extract into a controlled environment this week rather than onto a laptop, and offer to derive any note-based variable yourself as a structured flag so they never need the prose.

Follow-up

They say the age band loses resolution their model needs. What do you do?
How do you set the suppression threshold, and who signs off on it?
The analysis later needs linking to a second data source. What changes?

Make an honest case for your own analytical impact

Hard · impact attribution · counterfactual · self-assessment

You are asked to account for a year of your work. Constraint: you may only use measures that existed before you knew the outcome, and you must include one project that moved nothing. The portfolio contains a measure specification that changed how a contract settles, a predictive model that shipped, a self-serve dashboard, and an evaluation that killed a proposed programme. Deliverable: how you attribute value to each, which of them you claim credit for and which you do not, and the one number you would accept being judged on next year.

Approach

The probe is whether you can reason counterfactually about your own work, which is the same discipline this domain demands of you for programmes, applied where it is uncomfortable.
State the counterfactual for each item: what would have happened without you. A measure specification that changed how a contract settles has a dollar counterfactual you can compute. A dashboard's counterfactual is analyst hours, which is real and small, and attaching a revenue number to it is where these accounts usually turn into fiction.
Treat the killed programme as your strongest item and say why. Avoided spend is real value, it is attributable because the decision turned on your evaluation, and it is the one item nobody else will count on your behalf.
Report the shipped model the way you would report anyone's model: calibration in the subgroups that matter, adoption by the people it was built for, and whether any decision changed. A model that is live but ignored has zero impact, and saying so is the credibility test in this answer.
Name the failed project with its mechanism and its cost, then say what earlier signal you would watch to kill it sooner. Time to kill is a legitimate impact measure.
Choose next year's number so that it can go down. A metric that only ratchets upward is not a commitment.

Follow-up

Two other people also claim the contract measure. How do you split it?
Your model is live and used, but the decisions it informs would have been the same without it. Is that impact?
What is the strongest argument that your year had less impact than you just described?