Interview guides · Cloud Software Group

Cloud Software Group Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Cloud Software Group: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

As a Data Scientist at Cloud Software Group, you are at the forefront of transforming complex, industrial-scale data into actionable intelligence. This role is highly strategic, sitting at the intersection of advanced analytics, field research and development (R&D), and customer success. You will not just be building models in isolation; you will be directly influencing how enterprise customers leverage cloud solutions to optimize their industrial operations.

Coding rounds for this role are usually data-manipulation shaped rather than data-structure shaped: group-bys, joins, time windows, ranking within a partition. Confirm the format before spending a week on graph traversal.

Focus areas

Separate contracted seats from actively used seats
Power experiments for heavy-tailed account revenue
Measure churn only on renewal-eligible accounts

Overview

The impact of this position is deeply tied to the success of our clients and the evolution of our product offerings. Whether you are embedded in a Field R&D team focusing on industrial analytics or driving insights as a Lead Customer Success Data Scientist, your work directly shapes the user experience and the business value our platforms deliver. You will tackle high-scale, complex problem spaces, translating ambiguous client needs into robust data pipelines and machine learning solutions.

Working here means engaging with a diverse portfolio of projects. You can expect to dive deep into industry-specific analytics, collaborating closely with cross-functional teams to ensure that our data strategies align with real-world applications. It is a role that demands both deep technical rigor and the ability to communicate complex findings to stakeholders, making it an exciting opportunity for data professionals who want to see their research drive immediate, tangible results.

The Cloud Software Group interview process

Cloud Software Group candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Interview

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Behavioral Interview

Most of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.

What to demonstrate

Whether you can state the other side's argument accurately before you explain why you disagreed
What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
Whether you distinguish being overruled from being wrong, and can give an example of each

How to prepare

Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.

Final Round

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Computing monthly churn against the entire customer base when contracts are annual

An annual contract has no opportunity to churn except at its renewal date, so an account that is eleven months from renewal is in the denominator while being incapable of appearing in the numerator. The resulting rate is smaller than the real one by roughly the ratio of the base to the renewal-eligible base, and it oscillates with the seasonality of when deals were originally signed rather than with anything about the customers. The corresponding trap on the other side is counting a churn on the date the record was updated rather than on term_end_date, which shifts losses into whichever month the operations team did its paperwork.

Reading consumption metrics before the metering lag window has closed

Usage pipelines land late and correct themselves, which is exactly what is_restated and restated_at record. A dashboard queried on day T sees a partially populated tail for the last several days, so the most recent points always slope downward and always look like a regression. Analysts then explain the artefact, and sometimes ship a change to fix it. Establish the empirical settling time by measuring how much a given usage_date's total moves between first_written_at and its final value, exclude that many trailing days from every reportable figure, and never compare a fresh period against a settled one.

Crediting a treatment for regression to the mean

Selecting a group because it is extreme (lowest-engagement users, accounts having their worst month, the bottom decile of a score) moves that group's expected next-period value back toward the average even under no treatment, by exactly as much as the selecting measure is imperfectly correlated with its own later value. Compare against units that met the same selection rule and went untreated, or use two pre-periods so the bounce-back is visible before the intervention starts. A pre-post number on a group chosen for being extreme measures the selection rule, not the treatment.

Over-explaining the method and under-explaining the implication

Lead with the answer and what you would do about it, then give the approach when asked. Roughly one sentence of method per three of implication is the right ratio for a stakeholder-facing answer; the interviewer already knows what a regression is.

Questions & practice

What is the probability of getting exactly two heads if you flip a fair coin three times?

Medium · statistics and probability

Approach

Quantify uncertainty explicitly rather than reporting a point estimate alone.
Sanity-check the answer against a simple bound or a simulated case.
Write down the assumption the method needs before you use the method.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

Explain the bias-variance tradeoff and how you manage it in your models.

Medium · statistics and probability

Approach

Translate the result into the decision it informs, in one plain sentence.
Say what the estimate is of, and over what population it generalises.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

Which assumption here is most likely to be violated in practice?
How would you explain this result to someone who does not know statistics?

What is the difference between generative and discriminative models?

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Sanity-check the answer against a simple bound or a simulated case.
Quantify uncertainty explicitly rather than reporting a point estimate alone.

Follow-up

Which assumption here is most likely to be violated in practice?
How would you explain this result to someone who does not know statistics?

How do you evaluate the performance of an unsupervised learning algorithm?

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Say how the offline result would be validated online before it is trusted.
Set a baseline first, so any model has something honest to beat.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Walk me through a machine learning project you recently completed from start to finish.

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Walk me through the mathematical intuition behind logistic regression.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Implement seven-day activation rate from its written definition

Easy · metrics · pandas · activation

Given dim_account (account_id, created_at, is_internal, is_current) and fct_api_request (account_id, request_at, http_status, api_key_id, traffic_class), implement activation_rate(accounts, requests, week_start). Definition: the numerator is accounts whose first request with http_status < 400, api_key_id not null and traffic_class != 'synthetic_monitor' occurs no later than 168 hours after created_at; the denominator is non-internal accounts created during the ISO week starting week_start. All timestamps are tz-aware UTC. Return the rate and both counts, and refuse to report a week until every account in it has had its full 168 hours.

Approach

Reduce dim_account to one row per account_id before joining anything. It is a type 2 dimension, so several versions of the same account exist; joining the versioned table to requests multiplies the denominator by the number of plan changes an account happened to make.
Build the denominator first and freeze it: is_internal == False and week_start <= created_at < week_start + 7 days. Everything after this is a filter on the numerator only, because an account that never sent a request must still sit in the bottom of the fraction.
Filter requests to qualifying rows and only then take groupby('account_id').request_at.min(). The first qualifying request is not the same object as the global first request filtered afterwards, and the two answers differ for every account whose first call was a 4xx.
Left-join the first qualifying timestamp onto the cohort and test (first_ok - created_at) <= Timedelta(hours=168). NaT propagates to False in that comparison, which is the behaviour you want, but assert it rather than assume it.
Guard reportability explicitly: if week_start + 7 days + 168 hours exceeds the maximum request_at in the data, the week is censored and will read as a drop, so return None or raise rather than emit a number.

Worked solution (20 min)

cohort = accounts.loc[~accounts.is_internal & accounts.created_at.between(week_start, week_start + pd.Timedelta(days=7), inclusive='left'), ['account_id','created_at']].drop_duplicates('account_id')
ok = requests[(requests.http_status < 400) & requests.api_key_id.notna() & (requests.traffic_class != 'synthetic_monitor')]
first_ok = ok.groupby('account_id', as_index=False).request_at.min(); m = cohort.merge(first_ok, on='account_id', how='left'); assert len(m) == len(cohort)
activated = (m.request_at - m.created_at) <= pd.Timedelta(hours=168); return {'rate': float(activated.mean()), 'numerator': int(activated.sum()), 'denominator': len(m)}

Expected result: A dict whose denominator is the count of distinct non-internal accounts created in that ISO week, numerator <= denominator, and rate = numerator / denominator; accounts with no qualifying request sit in the denominator with NaT and evaluate to False.

Verify it yourself

len(m) == cohort.account_id.nunique(); the merge must not add rows.
Dropping the synthetic_monitor filter can only raise the numerator, never lower it.
Widening the window from 168h to 336h is monotone: the numerator must not decrease.
An account whose only qualifying request falls at exactly created_at + 168h counts as activated, since the comparison is inclusive.

Follow-up

Median time-to-first-call is more informative. What breaks if you take the median over activated accounts only, and what estimator fixes it?
How would you decide whether 168 hours is the right window rather than 72 or 336?
An account signs up, does nothing for 20 days, then integrates heavily. Where does it land in this metric, and is that what you want?

Collapse retried requests into logical operations per account

Medium · window functions · deduplication · null semantics

fct_api_request carries request_id, account_id, api_key_id, idempotency_key (null when the caller supplied none), is_retry, request_at and http_status. For one ISO week, count logical operations rather than HTTP calls: requests sharing (account_id, api_key_id, idempotency_key) collapse to the earliest one, while every request with a null idempotency_key is its own operation. Return per account the raw request count, the logical operation count and the ratio between them, and state which direction a rising 5xx rate pushes that ratio.

Approach

Split the stream before deduplicating. GROUP BY and PARTITION BY treat NULLs as equal to one another, while the = operator does not, so a single grouping pass folds every null-key request in an account into one 'operation' and can cut the count by orders of magnitude.
Deduplicate the keyed rows with ROW_NUMBER() OVER (PARTITION BY account_id, api_key_id, idempotency_key ORDER BY request_at, request_id) and keep rn = 1. Including request_id in the ORDER BY makes the result deterministic when two rows share a millisecond, which they will.
UNION ALL the null-key rows back in unchanged; they need no dedup and must not pass through the partition.
Aggregate per account: count() over the raw week, count() over the union, and the ratio of the two. Report the ratio, not the difference, so accounts of very different sizes are comparable.
Cross-check against is_retry, which flags a resend of the same idempotency key: raw minus logical should sit close to the count of is_retry rows, and a large gap means clients are resending without a key and the dedup is under-counting duplicates.
Name the inversion explicitly: 5xx responses provoke client retries, so amplification rises exactly when reliability falls, and any engagement metric built on raw request counts will show growth during an outage.

Follow-up

Which of these two counts belongs in the billable-units metric, and which in an engagement metric?
An account's amplification ratio jumps from 1.05 to 3.0 in a day. Name three causes and the query that separates them.

Running commitment burn-down and the date consumption crosses it

Medium · window functions · running totals · committed consumption

fct_subscription_period gives committed_amount_cents, term_start_date and term_end_date for each account's current version where pricing_model = 'committed_consumption'. fct_usage_daily gives account_id, workspace_id, sku_code, usage_date and net_amount_cents, at one row per workspace and SKU per day. Inside each account's current term, return the running total of net_amount_cents by usage_date, the first usage_date on which that running total reaches committed_amount_cents, and the fraction of the term elapsed at that point. Accounts that have not reached their commitment must still appear, with a null crossing date.

Approach

Collapse usage to one row per (account_id, usage_date) first. The fact is grained by workspace and SKU, so a raw running total leaves several rows per date and the first-crossing date becomes dependent on the arbitrary order of rows inside that day.
Restrict to the term with usage_date BETWEEN term_start_date AND term_end_date on the account's current row, so no prior term's consumption leaks into this term's burn-down.
Compute SUM(net_cents) OVER (PARTITION BY account_id ORDER BY usage_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW). Name ROWS explicitly: the default frame is RANGE, which includes all peer rows at the same ORDER BY value and would hand back the whole day's total on each of that day's rows.
Extract the crossing with MIN(usage_date) FILTER (WHERE running_cents >= committed_amount_cents) grouped per account, which returns NULL for accounts still under commitment instead of dropping them.
Elapsed fraction is (crossing_date - term_start_date)::numeric / NULLIF(term_end_date - term_start_date, 0); Postgres date subtraction yields whole days, so the guard matters for same-day terms.
Exclude the trailing days still inside the metering settling window, measured from first_written_at against restated_at, and say how many days you cut and why.

Worked solution (30 min)

Build terms: SELECT account_id, committed_amount_cents, term_start_date, term_end_date FROM fct_subscription_period WHERE is_current AND pricing_model = 'committed_consumption' AND committed_amount_cents IS NOT NULL.
Build daily: join fct_usage_daily to terms on account_id with usage_date inside the term, then GROUP BY account_id, usage_date summing net_amount_cents.
Add running_cents with SUM(...) OVER (PARTITION BY account_id ORDER BY usage_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW).
Aggregate per account with MIN(usage_date) FILTER (WHERE running_cents >= committed_amount_cents) AS crossing_date, then compute the elapsed fraction with the NULLIF guard.
LEFT JOIN that summary back onto terms so every committed account appears, and verify the last running_cents per account equals a plain SUM over the term.

Expected result: A per-account-per-day series of running_cents inside the current term, plus one summary row per committed account carrying crossing_date and term_elapsed_fraction, with crossing_date NULL exactly for accounts whose final running total is below committed_amount_cents.

Verify it yourself

The maximum running_cents per account equals SUM(net_amount_cents) over that account's term computed independently.
crossing_date is NULL if and only if the final running total is below the commitment.
term_elapsed_fraction falls in [0, 1] for every non-null crossing date.
running_cents is non-decreasing within each account, which fails immediately if a negative credit row was included without comment.

Follow-up

Turn this into an end-of-term overage forecast. What breaks if you extrapolate a linear run rate on a consumption product?
An account crosses its commitment at 40 percent of the term. Is that an expansion signal or a billing-surprise risk, and what would you check to tell them apart?

How do you ensure your models remain accurate after they are deployed into production?

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.
Say what you would check first and why it is the highest-information step.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Size an account-randomised test of a consumption feature

Easy · power · mde · heavy-tailed-outcomes

You have 2,800 non-internal paying accounts. The outcome is the monthly sum of billable_quantity for sku_code = 'compute_hours' from fct_usage_daily, with a mean of 640 and a standard deviation of 2,100 across accounts. Product wants to detect a 5% lift from a scheduling feature, randomised 50/50 at the account level, at 80% power and two-sided alpha 0.05. Give the minimum detectable effect the available sample actually supports, say whether the 5% target is reachable, and state what you would change in the design or the metric before agreeing to run it.

Approach

Write the two-sample formula and plug it before discussing anything else: MDE = (z_0.975 + z_0.80) * sd * sqrt(1/n1 + 1/n2) = 2.80 * 2100 * sqrt(2/1400) = 222 compute-hours, which is 34.7% of the 640 mean. The driver is the coefficient of variation of 3.3, not the account count, so quoting a sample size without it is not a power calculation.
Invert for the stated target. A 5% lift is 32 compute-hours, needing N = 4 * (2.80 * 2100 / 32)^2 = about 135,000 accounts, roughly 48 times the eligible population. Say the test is not powerable on this metric, and do not offer to run it longer: the account population is fixed, and extra weeks add repeated observations on the same clusters rather than new clusters.
Attack the variance instead of the sample. Pre-register winsorisation at the 99th percentile of the pre-period distribution, or move the estimand to a bounded one such as the share of accounts exceeding a usage floor. State explicitly that this changes the estimand rather than cleaning the data, because a cap chosen after seeing the result encodes the answer.
Add CUPED on the account's pre-period compute_hours. At a pre/post correlation of 0.80 the residual variance multiplier is 1 - 0.80^2 = 0.36, the standard error falls to 0.60 of its unadjusted value, and the MDE drops from 222 to 133 compute-hours (20.8% relative). That is still four times the requested 5%.
Fix the randomisation unit and the balance in the same breath: randomise account_id because colleagues share workspaces, and stratify assignment on pre-period usage decile so that a handful of very large accounts cannot land disproportionately in one arm and dominate the difference in means.
Close with an operating decision rather than a number. Run only if the team will act on a roughly 20% MDE after CUPED and winsorisation; otherwise re-specify the primary outcome as a bounded indicator and treat compute-hours as a secondary reported with intervals only.

Worked solution (25 min)

Compute the coefficient of variation, 2100 / 640 = 3.28, and check the top-decile share of total compute_hours before treating the mean as a summary of anything.
Compute MDE = 2.80 * 2100 * sqrt(1/1400 + 1/1400) = 2.80 * 2100 * 0.0378 = 222 compute-hours, 34.7% relative.
Invert to the required N for a 32-hour effect: 4 * (5883 / 32)^2 = about 135,000 accounts, 48x the population.
Recompute under CUPED at rho = 0.80: variance multiplier 0.36, SE multiplier 0.60, MDE 133 compute-hours, 20.8% relative.
Recompute on the binary variant at base rate 0.46 and write the recommendation as one decision with one operating point.

Expected result: MDE is about 222 compute-hours (34.7% relative) as specified and about 133 (20.8%) with CUPED at rho = 0.80. A 5% lift would require roughly 135,000 accounts, about 48 times the eligible population, so the test is declined on this metric and re-specified on a bounded outcome where the MDE falls to about 5.3 percentage points on a 0.46 base.

Verify it yourself

sqrt(1/1400 + 1/1400) = 0.0378, so the MDE must land within a few percent of 2.80 * 2100 * 0.0378 = 222. A materially different number means a one-sample or one-sided formula was used.
Required N scales with the inverse square of the effect size: halving the target MDE must quadruple N. If it does not, the formula is wrong.
The CUPED variance multiplier is 1 - rho^2, never 1 - rho. At rho = 0.80 the standard error falls by 40%, not by 20%.

Follow-up

The feature only applies to the 310 accounts on committed_consumption contracts. What is the MDE on that subset, and does it change the recommendation?
Suppose the primary outcome becomes a binary indicator, account exceeded 100 compute-hours in the month, at a base rate of 0.46. What is the MDE now, and why did it improve so much?
What would make you stop this test early, and how do you write that rule so it is not peeking?

Billable units per account jumped while nothing shipped

Medium · retries · idempotency · guardrail metrics

Billable units per paying account rose 22 percent month over month with no pricing or packaging change. From fct_api_request (account_id, endpoint, http_status, is_retry, idempotency_key, traffic_class, billable_units, request_at) and fct_usage_daily (account_id, sku_code, usage_date, billable_quantity), determine how much of the rise is delivered value and how much is duplicated work. Deliverable: the decomposed figure, the accounts it concentrates in, and a recommendation on whether to report the 22 percent at all.

Approach

Check the guardrail before the headline: compute the customer-visible server error rate per account for both months, numerator http_status >= 500 and denominator excluding synthetic_monitor and load_test. Metered volume rising alongside an error rate is the known failure mode in this domain.
Deduplicate logical work by counting billable_units once per (account_id, idempotency_key) at the first successful request rather than once per row. Rows with a null idempotency_key cannot be deduplicated, so report their share as an explicit uncertainty band instead of assuming they are all unique.
Split by traffic_class before interpreting anything, because one change to a continuous-integration configuration can multiply request volume overnight without a human deciding anything about the product.
Test concentration: compute the per-account distribution of the increase and its top-decile share. A rise carried by a few accounts scaling one batch job is a different finding from a broad shift and gets a different recommendation.
Reconcile against fct_usage_daily for the same accounts and dates, and be ready to explain the expected gap in two sentences: request rows include retries and failures carrying zero billable_units, and the usage table restates after first write.

Follow-up

A client retrying a request the server already completed produces duplicate billed work. What protocol or product change removes that, and what would you measure to confirm it worked?
If the duplicated volume was genuinely invoiced, what should finance do, and how does that change what belongs in the metric?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

What the behavioural round is actually scoring

Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.

Tell me about a time you had to work with messy or incomplete data. What was your process for handling it?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Tell me about a time you disagreed with a colleague on the best approach to a data problem. How did you resolve it?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Close with what you would do differently, concretely.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Scope an open-ended request to predict account churn

Medium · scoping · churn modelling · operating point

A customer success director asks for a list of accounts about to churn. You know only that the team has six people and that contracts are annual. Available data is fct_subscription_period, fct_usage_daily, fct_api_request, fct_support_ticket and dim_account. Before writing any code, produce the questions you need answered, a proposed definition of about to churn, and the shape of the artefact you would hand back, including the operating point that turns a score into a decision.

Approach

The interviewer is probing whether you convert a vague request into a decision with a capacity constraint attached. A candidate who starts talking about model families has already failed the exercise.
Pin the event and the horizon first. Churn is only possible at term_end_date, so the population is accounts renewing in the next 60 to 90 days, not the whole base. Ask explicitly whether contraction and downgrade count as churn or only full non-renewal, because the three have different base rates and different interventions.
Pin the action and the capacity. Six people times a realistic number of meaningful interventions per week gives k, and k is what the list is ranked to. Evaluate on precision at k rather than a global AUC over accounts that will never be contacted.
Audit leakage before choosing features. Every feature needs a timestamp proving it existed before the prediction date. A downgrade amendment, a churn reason code, and a ticket opened after the renewal conversation started are all leaks that will make the offline number look excellent and the live list useless.
Ask for the counterfactual now rather than later. Coverage is assigned deliberately, so without a held-out slice agreed at the start the intervention can never be evaluated, and you will be asked for its impact in nine months regardless.
Propose the smallest artefact that closes the loop: a weekly ranked list sized to capacity with two or three inspectable reasons per row, plus a stated policy for accounts below the line.

Follow-up

The director insists all accounts are in scope, not only those renewing soon. How do you answer without simply refusing?
Historical non-renewals number about 30 a year. At what point do you tell them a model is the wrong tool and a rules list is better?
Which candidate features would you drop purely because you cannot date them?