Interview guides · Catalyst Labs

Catalyst Labs Data Scientist Interview Questions & Guide 2026

Catalyst Labs data scientist prep: probability, sampling distributions, and stating the assumption behind every test you run.

The 60-second brief

The role of a Data Scientist at Catalyst Labs is pivotal in harnessing data to drive health outcomes and business decisions. As a Data Scientist, you will work on complex datasets to derive insights that inform product development and improve user experiences. Your analyses will contribute to innovative healthcare solutions, impacting both patients and healthcare providers significantly.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

Measure churn only on renewal-eligible accounts
Strip CI, retry and synthetic traffic first
Separate contracted seats from actively used seats

Overview

In this role, you will engage with diverse teams, pushing the boundaries of data analytics while working on real-world problems in healthcare. Whether it's developing predictive models or conducting exploratory data analysis, your contributions will directly influence the quality and effectiveness of products that serve a critical mission. The complexity and scale of the data at Catalyst Labs make this opportunity both challenging and rewarding, as you will be at the forefront of technology and analytics in the health sector.

The Catalyst Labs interview process

Catalyst Labs candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Preliminary Phone Interview

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Coding Challenge

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Multiple Interview Rounds

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Final Interview

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Treating raw request or usage volume as engagement

Most traffic in this domain is emitted by machines. Continuous-integration pipelines, scheduled batch jobs, synthetic monitors, backfills and client retries can all grow by an order of magnitude from one configuration change made by one engineer, and none of it represents a new decision to use the product. The inversion is what makes it dangerous: when the platform degrades, clients retry, so error-driven retry volume rises at the exact moment the customer is most likely to leave, and an engagement dashboard built on raw counts shows growth immediately before a churn. Filter on traffic_class and on successful status before anything else, and keep failed-request volume as its own separate series.

Reporting a mean over accounts when account revenue is heavy-tailed

When a small number of accounts hold most of the revenue, the sample mean is dominated by whichever of them happens to be in the sample, and the sample variance keeps growing as more data arrives instead of stabilising. In that regime the usual central-limit-based confidence interval understates uncertainty, and a single renewal or a single large account's batch job can flip the sign of a measured effect. The fixes are to pre-register a winsorisation or capping rule before looking at the outcome, to report account counts crossing a threshold alongside the revenue figure, or to define the estimand on a bounded transform. Choosing the cap after seeing the result is a separate and worse problem, because the cap then encodes the answer.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Questions & practice

Can you explain the importance of cross-validation in model evaluation?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

Walk us through a scenario where you had to troubleshoot a failing model.

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

Permutation-test a consumption experiment randomised at account level

Hard · permutation-test · heavy-tails · cuped · experiment-readout

An experiment randomised 900 accounts into two arms. You have one row per account: account_id, arm, consumption_28d (billable units after launch) and consumption_pre (the 28 days before). Consumption is heavy-tailed and the largest account is several percent of the total. Write a permutation test from scratch: winsorise at the pooled 99th percentile as a pre-registered rule, use the difference in arm means of the winsorised outcome as the statistic, and obtain a two-sided p-value from 20,000 relabellings of the account-level arm vector. Report the observed effect, the p-value, and the same test on a CUPED-adjusted outcome.

Approach

Be precise about what the permutation test needs. Under the sharp null of no effect for any account, the outcomes are exchangeable across arm labels, and the test is valid for ANY statistic T(outcomes, labels) provided the identical function is applied to the observed labels and to all 20,000 relabellings. The pooled 99th percentile is a function of the outcome vector alone, so recomputing it inside the loop returns the same number 20,000 times: that is wasted CPU, not a bias, and hoisting it out is an optimisation rather than a correctness fix. Say plainly that capping at all changes the estimand from mean consumption to mean capped consumption; it is not a neutral cleaning step.
The mistake that does invalidate the test is an asymmetry between the observed statistic and the permuted ones, and the easiest way to create it is to derive the cleaning rule from the observed arm labels and then freeze it — winsorise each arm at its own observed 99th percentile, hold those two caps fixed, and permute. The observed value is then computed with caps matched to its own partition while every relabelling is scored with caps belonging to a different one, so the null distribution no longer answers the question the p-value claims to answer. A per-arm cap recomputed consistently inside every permutation is a valid test, but it estimates a contrast whose two sides are capped at different thresholds, so prefer the pooled cap on estimand grounds and pre-register it.
Permute the account-level arm vector, because the account is the randomisation unit. Relabelling anything finer — users, workspaces, requests — generates a null distribution narrower than the design actually supports and returns p-values that are anti-conservative.
Vectorise the null: tile the treatment indicator into a (B, n) matrix and permute along axis 1 with rng.permuted(..., out=...). The statistic is a difference of means, so the treated sum alone determines it and the whole null is one matrix-vector product. Use the two-sided p-value (1 + count(|stat_perm| >= |stat_obs|)) / (B + 1); the plus-one on each side is not cosmetic, it keeps the p-value away from exactly zero and keeps the test valid at finite B.
For CUPED, fit theta = cov(y, x) / var(x) on the pooled data and use that same theta for the observed statistic and every relabelling. Pooled theta, like the pooled cap, carries no label information, so where in the loop you compute it is again only a performance question; fitting theta within arms is what goes wrong, because the adjusted outcome then depends on the labels and an observed-label fit frozen across all 20,000 relabellings breaks the match between observed and permuted statistics. x must be measured entirely before launch, which consumption_pre is. Expected variance reduction is about 1 - corr(y, x)^2; measure the achieved reduction from the two null distributions rather than asserting it.

Worked solution (45 min)

cap_y = np.quantile(df.consumption_28d, 0.99); y = np.minimum(df.consumption_28d.to_numpy(float), cap_y); cap_x = np.quantile(df.consumption_pre, 0.99); x = np.minimum(df.consumption_pre.to_numpy(float), cap_x)
t = (df.arm == 'treatment').to_numpy(); n1 = int(t.sum()); n0 = len(t) - n1; obs = y[t].mean() - y[~t].mean()
rng = np.random.default_rng(11); L = np.tile(t.astype(np.int8), (20_000, 1)); rng.permuted(L, axis=1, out=L); s1 = L @ y; stats = s1/n1 - (y.sum() - s1)/n0
p = (1 + int(np.sum(np.abs(stats) >= abs(obs)))) / (20_000 + 1)
theta = np.cov(y, x, ddof=1)[0,1] / np.var(x, ddof=1); y_adj = y - theta*(x - x.mean()); repeat steps 2 to 4 on y_adj and compare stats.std(ddof=1) between the two runs.

Expected result: A two-sided p-value strictly between 1/20001 and 1, an observed effect expressed in capped billable units per account, and a CUPED null whose standard deviation is smaller than the unadjusted one by roughly sqrt(1 - corr(y, x)^2). The CUPED point estimate stays close to the unadjusted one, since the adjustment removes variance rather than shifting the effect.

Verify it yourself

The permutation null is centred near zero: |mean(stats)| is a small fraction of std(stats).
An A/A run, relabelling arm at random before the test and repeating over seeds, gives roughly uniform p-values with about 5% below 0.05.
std(stats_cuped) < std(stats) whenever corr(y, x) is materially non-zero, and the ratio matches sqrt(1 - r^2) to within simulation noise.
Moving the pooled cap computation inside the permutation loop reproduces the same p-value bit for bit at the same seed — the direct demonstration that a pooled cap carries no label information — while setting the cap to infinity reproduces the uncapped test exactly, so the winsorisation switch is independently verifiable.

Follow-up

The p-value is 0.04 with the cap and 0.31 without it. What do you report, and what did you pre-register?
Colleagues in a shared workspace can see the treated behaviour. How does that change the design and the estimate?
How many accounts would you need to detect a 5% lift given this outcome's distribution?

What libraries do you commonly use for data manipulation and analysis in Python?

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Write a function to compute the confusion matrix from a set of predictions and actual values.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Net revenue retention on a cohort frozen twelve months back

Hard · retention · as-of join · cohort arithmetic

fct_subscription_period carries subscription_period_id, account_id, arr_cents, plan_code, term_start_date, term_end_date, booked_at, amendment_type, superseded_by_id (the subscription_period_id of the version that replaced this one, null on the live version of a lineage) and is_current. Compute net revenue retention for month M: the summed arr_cents at M for the set of accounts holding arr_cents > 0 at M-12, divided by that same set's arr_cents at M-12. An account can hold more than one live subscription, a churned account contributes zero rather than dropping out, and nothing signed after M-12 may enter either side. Return the ratio plus the expansion, contraction and churn components in cents.

Approach

Write one reusable as-of ARR snapshot parameterised by a date: rows whose term brackets the date AND whose booked_at is at or before the date, then only the version of each lineage that is still live at that date, then sum arr_cents per account. The booked_at guard matters because an amendment signed in advance otherwise co-exists with the term it replaces and double counts the account.
Collapse the lineage on superseded_by_id, not on any attribute of the contract. Keep a row when superseded_by_id IS NULL, or when the successor it points at was booked after the date. Ranking with ROW_NUMBER() OVER (PARTITION BY account_id, plan_code ...) instead is wrong in both directions: an amendment that moves the account from one plan_code to another puts the old and new versions in different partitions, so both are rank 1, both bracket the date, and the account's arr_cents is counted twice; and two genuinely concurrent subscriptions that happen to share a plan_code land in one partition, so one of them is deleted.
Resolve the successor with a LEFT JOIN back to fct_subscription_period on subscription_period_id, and treat a missing successor as not superseded. An inner join would silently delete an account's ARR on a dangling pointer, which is a data-quality bug in the source, not a retention movement.
Never use is_current for the M-12 side. is_current describes today; using it at the historical snapshot backdates the present contract onto last year's cohort and makes retention look like 100 percent by construction.
Freeze the cohort from the M-12 snapshot where arr_cents > 0, then LEFT JOIN the M snapshot onto it and COALESCE the missing side to zero. An inner join deletes exactly the churned accounts, which is the single largest way this number gets overstated.
Return a ratio of sums, not a mean of per-account ratios. The two are different estimands: contraction is floored at zero while expansion is unbounded, so the mean of ratios is both biased relative to the aggregate and far noisier on a skewed revenue base.
Decompose per account on the delta: positive delta is expansion, negative delta with a non-zero M value is contraction, a zero M value is churn. The three components must reconcile to numerator minus denominator.
Prove no leakage: any account whose first contract began after M-12 must be absent from both sides, and the cohort row count must be identical in the numerator and denominator.

Worked solution (45 min)

Write arr_asof(d) as a CTE or lateral: from fct_subscription_period s take rows with term_start_date <= d AND term_end_date >= d AND booked_at <= d, LEFT JOIN fct_subscription_period succ ON succ.subscription_period_id = s.superseded_by_id, keep the row when s.superseded_by_id IS NULL OR succ.subscription_period_id IS NULL OR succ.booked_at > d, then sum arr_cents per account_id across every surviving version.
Sanity-check the lineage rule on one amended account before going further: at a date after the amendment, the account must contribute exactly one version per lineage even when the amendment changed plan_code, term dates or both.
Materialise base = arr_asof(M-12) filtered to arr_cents > 0, and curr = arr_asof(M).
LEFT JOIN curr onto base on account_id and COALESCE(curr.arr_cents, 0) AS arr_now.
Compute nrr = sum(arr_now)::numeric / NULLIF(sum(base.arr_cents), 0), and the three components with SUM(...) FILTER on the sign of arr_now - base.arr_cents and on arr_now = 0.
Reconcile: assert sum(arr_now) - sum(base.arr_cents) = expansion - contraction - churn, and assert the cohort account count is identical on both sides.

Expected result: A single row with nrr as a ratio, plus expansion_cents, contraction_cents and churn_cents as positive magnitudes satisfying numerator minus denominator equals expansion minus contraction minus churn, over a cohort whose membership is fixed at M-12.

Verify it yourself

At each snapshot date, the surviving row count per account equals the number of distinct live subscription lineages on that account, not the number of contract versions bracketing the date.
For an account whose amendment changed plan_code, the snapshot returns one version and its snapshot ARR equals the amended arr_cents; swapping in a ROW_NUMBER partition on (account_id, plan_code) returns two versions for that account and inflates the side it sits on, which is the fastest way to see the bug.
The three components reconcile to the numerator-denominator delta to the cent.
The distinct account count is the same in the numerator and the denominator.
No account with a first term_start_date after M-12 appears anywhere in the output.
Re-running after new accounts sign leaves the ratio unchanged, which is the defining property of a frozen cohort.

Follow-up

Net revenue retention can rise while the business shrinks. Show one mechanism and name the guardrail that catches it.
How do you handle an account that co-terms two subscriptions into one mid-window, so the subscription count changes but the money does not?
Finance computes this from invoiced amounts and gets a different number. Which is right for which question?

How do you prioritize tasks when working on multiple projects simultaneously?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

If given a messy dataset, what steps would you take to clean and prepare it for analysis?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Given a dataset with various features affecting a binary outcome, explain your analysis strategy.

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Measure a repackaging that moves usage between metered SKUs

Hard · mix shift · sku migration · metering lag · unit economics

A repackaging moves work previously billed as compute_hours into build_minutes at a different rate. Early readouts show compute_hours down 30% and total net_amount_cents flat. You have fct_usage_daily (account_id, sku_code, usage_date, billable_quantity, net_amount_cents, cogs_cents, is_restated, first_written_at, restated_at) and fct_api_request (account_id, environment, traffic_class, http_status, compute_ms, egress_bytes, request_at). Design the metric that says whether real consumption changed rather than merely moving between SKUs, state the settling window you will wait for and how you measured it, and name the unit-economics guardrail. Deliver the metric definition and the readout you would take to a pricing decision.

Approach

Refuse to compare SKU-level quantity series across the change. billable_quantity carries a different unit per sku_code, so a 30% fall in compute_hours cannot be added to or netted against a rise in build_minutes. Any metric that sums billable_quantity across sku_code produces a number with no unit and no meaning.
Build a SKU-invariant measure of delivered work from fct_api_request instead: successful, non-synthetic, non-load-test production requests with sum(compute_ms) and sum(egress_bytes) per account per week. That series is unaffected by which SKU the billing system attributed the work to, which is exactly the property this question needs.
Measure the settling window before reading anything. For a sample of historical usage_date values, compare the total as first written against the settled total after restatement, and exclude as many trailing days as it takes for the delta to fall under a stated tolerance. The tail of every usage series slopes downward for this reason alone and is routinely misread as a regression, then explained, then fixed.
Compare like for like at the account grain against each account's own pre-period. Split accounts by whether their workload was in scope of the repackaging, and read the change in delivered work per account relative to its own 8-week pre-period level, reporting the median and interquartile range. A fleet mean is dominated by whichever account happened to scale one batch job, because account consumption is heavy-tailed.
Guardrail the economics rather than the revenue line. Compute gross margin per account from net_amount_cents and cogs_cents and report it as a distribution. Flat revenue with identical work moved onto a SKU carrying a higher cogs_cents allocation is a real margin loss that a flat revenue line conceals completely.

Worked solution (40 min)

Measure the settling window: for 30 historical usage_date values, compute total net_amount_cents as first written against the current settled value, and take the number of trailing days after which the delta falls below a stated tolerance such as 0.5%.
Build weekly per-account delivered work from fct_api_request: successful request count, sum(compute_ms) and sum(egress_bytes), filtered to traffic_class in ('interactive','batch') and environment = 'production'.
Split accounts into in-scope and out-of-scope for the repackaging and compute the change in delivered work per account relative to its own 8-week pre-period, reporting median and interquartile range rather than a mean.
Compute per-account monthly gross margin before and after from net_amount_cents and cogs_cents, and report the count and revenue share of accounts whose margin crossed below zero.
Reconcile the two sources and state why request-derived work and fct_usage_daily quantities will not tie exactly, naming retries, failed requests and restatement.

Expected result: A three-part readout: the empirically measured settling window in days, the median per-account change in delivered work (successful production requests and compute_ms) for in-scope against out-of-scope accounts measured relative to each account's own pre-period, and the per-account margin distribution before and after. The conclusion states whether consumption actually changed or merely moved between SKUs, and at no point adds quantities across sku_code.

Verify it yourself

Out-of-scope accounts should show no change in delivered work; movement there means the scope split is wrong or something else shipped in the same window.
The settling window must be measured rather than assumed; re-running the readout a week later must not move the final week's figures by more than the stated tolerance.
Total net_amount_cents across all SKUs before and after must reconcile to the invoiced total; a gap means restated rows were read mid-settlement.

Follow-up

Revenue is flat and delivered work is flat. Name the one thing that could still have got worse, and exactly where you would see it.
Packaging changes are usually applied to everyone at once. How would you design this as an experiment rather than a before-and-after, and what would you give up?

Blended gross margin fell while every segment improved

Medium · mix shift · simpsons paradox · as-of join

Blended gross margin across paying accounts fell from 71 percent to 66 percent over two quarters, yet margin improved inside every plan_tier and every deployment_model. You have fct_usage_daily (account_id, sku_code, usage_date, net_amount_cents, cogs_cents), fct_subscription_period (account_id, plan_tier, term_start_date, term_end_date) and dim_account (account_id, deployment_model, is_internal, effective_from, effective_to). Produce an exact decomposition of the five-point move into mix, rate and interaction terms, and name the segment shift that carries it.

Approach

Write blended margin as a revenue-weighted average of segment margins, m = sum over i of w_i * m_i, where w_i is the segment's share of net_amount_cents. Margin is a ratio of sums, so only revenue weights reproduce the blended figure.
Apply the exact three-term decomposition: delta_m = sum(delta_w_i * m_i0) + sum(w_i0 * delta_m_i) + sum(delta_w_i * delta_m_i), read as mix, rate and interaction. It is exact by construction, so the three terms must sum to the observed change with no residual.
Build segments from as-of joins: take the fct_subscription_period version whose term brackets each usage_date and the dim_account version whose effective_from and effective_to bracket it, rather than joining on is_current and backdating today's attributes over last year's usage.
Run the decomposition on more than one segmentation, at minimum plan_tier, deployment_model and sku_code, since the mix that actually moved may not be the one anybody already suspected.
Once the driving segment is identified, determine whether the shift came from acquisition landing new accounts in a lower-margin segment or from existing accounts migrating, because those have different owners and different remedies.

Follow-up

The interaction term is sometimes large. What does it mean operationally, and when would you prefer a log-mean decomposition that distributes it across the other two terms?
Revenue-weighted average margin is one construction. What changes if leadership wants margin weighted by account count, and which question does each version answer?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Being wrong in public

A number you shipped turned out to be wrong, and someone had already acted on it. That is one of the most useful stories a data person can carry. What is being scored is how fast you noticed, who you told first, and what you changed in the process so the same class of error could not repeat quietly.

What does teamwork mean to you in the context of a data science project?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Announce a metric fix that cuts the headline number

Medium · metric definitions · stakeholders · communication

Weekly active organisations, the count on the company dashboard, has never excluded rows where dim_account.is_internal is true, and it counts traffic with traffic_class in synthetic_monitor and load_test. Correcting both reduces that count by 11 percent and removes most of the growth reported over two quarters. The figure appears in a board deck and in two teams' quarterly goals, one written on the count and one on the weekly active organisation ratio, whose denominator is accounts whose account_status was in ('trial','free','active_paid') through the week. Decide the order in which you tell people, what the dashboard shows during the transition, and what you propose happens to goals already set against the old definition.

Approach

The interviewer is probing whether you can land a correction as an operational change with a plan attached, rather than as an announcement other people then have to clean up after.
Quantify each exclusion separately before telling anyone: internal accounts, synthetic monitors, load tests. Three known quantities are a discussion; one alarming total is an argument.
Be precise about which side of the metric each exclusion touches, because one team's goal is on a count and the other's is on a ratio. The traffic-class filters remove requests, so they shrink the numerator only. Dropping internal accounts removes them from the ratio's denominator as well, since internal accounts carry ordinary account_status values and therefore sit in that denominator. Internal accounts are active in almost every week while the real base is not, so the numerator loses a larger share than the denominator and the ratio falls by less than the count does. Compute both and say which one the 11 percent is before anybody assumes.
Check whether the trend changes, not only the level. A constant 11 percent shift is a rebasing and nothing more. A shift that widens over time means the reported growth was partly internal or synthetic, which makes the existing goals unachievable as written and changes what you are asking teams to do.
Sequence the disclosure: the metric owner and the two teams whose goals move first and privately, then the board channel with a written bridge, then the dashboard. The dashboard is last because a number that changes without explanation is read as instability rather than as a fix.
Run both series for one reporting period with the bridge visible, restate history rather than letting the series break at a date, and set the date the old series is removed.
Propose the goal treatment yourself: rebase each target by the shift measured on the metric that target is written against, rather than leaving each team to negotiate individually, which is where corrections of this kind usually die.

Follow-up

One team's quarterly goal is now unreachable. Rebase the target or let it miss, and what does each choice teach the organisation?
How would this have been caught when the metric was first defined?
What else on that dashboard shares this failure mode, and how would you find out this week?

Walk through an analysis you later discovered was wrong

Easy · data quality · error ownership · metering

Six weeks ago you reported that consumption fell 9 percent in the last week of the month, and a team spent a sprint investigating the cause. The fall was an artefact: rows in fct_usage_daily land late and are restated in place, and you queried before the tail had settled. Describe how you found the error, what you told the people who acted on it, and the control you put in place so this class of mistake cannot reach a dashboard again. Be specific about how the settling window was measured.

Approach

The interviewer is probing whether you self-report errors before someone else finds them, and whether your fix is structural rather than a promise to be more careful. Say plainly that the number was wrong and that a sprint was spent on it, before describing any diagnosis.
Establish the artefact quantitatively instead of asserting that data lands late. For each usage_date, compare the total as of first_written_at against the settled total and read the settling time off that curve, for example 97 percent of final by day three and 99.5 percent by day five.
Correct the record the same day, in the channel the original number went out in, to the same audience. The cost of the wasted sprint belongs in the correction, not in a footnote.
Make the fix structural: exclude a trailing lag window from every reportable figure, and make the reporting view return no rows inside that window rather than returning partial ones. A dashboard that shades unsettled days still gets read as a decline.
State what generalises. Any fact table restated in place has this failure mode, so the guard belongs at the source rather than on the one dashboard that embarrassed you. A strong answer ends with the class of error closed; a generic one ends with a lesson learned.

Follow-up

How did you choose the completeness threshold behind the lag window, and what would make you recalibrate it?
What did you say to the team that lost the sprint, and what did they say back?
Is there a legitimate case for showing the unsettled tail at all, and to whom?