Interview guides · Atlas Copco Group

Atlas Copco Group Data Scientist Interview Questions & Guide 2026

Modeling drills for a Atlas Copco Group data scientist interview: baselines, regression interpretation, class imbalance, model defense.

The 60-second brief

As a Data Scientist at Atlas Copco Group, you are at the intersection of industrial innovation and digital transformation. This role is pivotal in leveraging data to optimize the complex machinery and service models that define the company’s global leadership. You will work within a collaborative, tech-forward environment where your analytical insights directly influence product efficiency, customer satisfaction, and operational excellence.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

Decompose a metric move by segment and mix
Size an experiment before anyone launches it
Separate novelty effects from durable behaviour change

Overview

You will contribute to a culture that values both technical rigor and cross-functional partnership, particularly with IT and engineering teams. Whether you are working on predictive maintenance for industrial equipment or developing customer-focused models, your impact is tangible. Expect to navigate a modern, fast-paced environment where the focus is on creating scalable, data-driven solutions that solve real-world industrial challenges.

The Atlas Copco Group interview process

Atlas Copco Group candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Initial Assessment

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Interviews with Team Leaders

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Where candidates lose points

Comparing cohort retention curves of different maturities, or building the curve from users who are still present

A cohort four weeks old has no week-8 value, so an average taken across cohorts silently drops young cohorts from the later columns and keeps them in the earlier ones. The curve then bends upward at the tail, and the reading that 'retention is improving over time' is an artefact of which cohorts survived to be measured. The same error appears in the denominator when retention is computed over users active in the current period rather than over the full original cohort, which conditions on survival and guarantees a flattering number. The fix is a triangle: fix the cohort at signup, bound every window on both sides, and only compare cells where every cohort has had the full elapsed time, publishing the rest as blank rather than as a partial average.

Slicing a flat experiment until a segment reaches significance

Testing one metric across twenty segments at a nominal 5% level produces a significant result about two thirds of the time when nothing is happening anywhere, and the segment that surfaces is by construction the one with the most favourable noise. The reported effect in that slice is then badly overstated, because selection on significance conditions the estimate on being large. What makes it dangerous rather than merely wrong is that a post-hoc segment always has a plausible story attached, so it survives the meeting. The controls are declaring the small number of segments of interest before launch, correcting across the ones tested, and treating anything discovered afterwards as a hypothesis that needs its own adequately-powered test rather than a finding.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Sizing estimates built on unnamed, unrevisable assumptions

Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.

Questions & practice

Describe a project where you had to collaborate closely with IT teams to deploy a model into production.

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Implement seven-day activation from its written definition

Medium · metric definition · cohorts · pandas · joins

Implement the seven-day activation rate. Inputs: dim_user with user_id, account_created_at_utc and is_internal; fct_event with user_id, occurred_at_utc and is_core_action. A user activates when core-action events carrying a non-NULL user_id fall on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). The denominator is every non-internal user whose account_created_at_utc lands in the cohort week, including users with no events at all. Return one row per cohort week with numerator, denominator and rate, publishing only weeks whose last signup is at least eight days old.

Approach

Build the denominator first, from dim_user alone, filtered on is_internal = False. Deriving it from the join is the standard way to lose every user who never fired an event, which is exactly the population the metric is about.
Join events to users on user_id with a left join from the user side, then apply the window as a half-open interval: occurred_at >= created AND occurred_at < created + 7 days. The right bound is exclusive, so an event at exactly created + 7 days does not count.
Count distinct UTC dates per user, not distinct events. Floor occurred_at_utc to date before the nunique, and do it in UTC rather than local time so the threshold does not move with the user's country.
Apply the >= 2 threshold, aggregate to cohort week, and compute the rate by re-summing numerator and denominator per week rather than averaging any per-user or per-day rate. Fix the week anchor explicitly: cohort_week is the Monday of the signup week in UTC, which is what Postgres DATE_TRUNC('week') returns and what any SQL version of this metric will produce. In pandas, subtract dt.weekday days from the floored timestamp. If you reach for periods instead, the anchor that matches is to_period('W') (equivalently 'W-SUN'), whose weeks end Sunday and therefore start Monday; to_period('W-MON') labels weeks that end on Monday, so it runs Tuesday through Monday and its start_time is a Tuesday. Mixing the two shifts every cohort label by one day and silently moves Mondays into the previous week.
Suppress immature weeks: drop any cohort week whose maximum account_created_at_utc is within 8 days of the data cut, and return them as absent rather than as a partial number.

Follow-up

The threshold is 2 distinct days. What changes in the reported history if someone moves it to 3, and how would you publish that change?
Invited seats and SSO-provisioned users have no pre-signup session. Should they be in this denominator at all, and what does including them do to the rate for sales-assisted accounts?
How would you produce the same metric at account grain, and which of the two would you put on the dashboard?

Cluster bootstrap for a per-session rate randomised on users

Hard · cluster bootstrap · randomisation unit · variance

An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.

Approach

Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.

Worked solution (35 min)

per_user = df.groupby(['variant','user_id'])['converted'].agg(['sum','size']); split into two arrays per arm.
point = (t_sum.sum() / t_n.sum()) - (c_sum.sum() / c_n.sum()).
For b in range(B): idx = rng.integers(0, len(t_sum), len(t_sum)); ratio_t = t_sum[idx].sum() / t_n[idx].sum(); same for control; store the difference. Vectorise by drawing a (B, n) index matrix if memory allows.
ci = np.percentile(diffs, [2.5, 97.5]); naive_se = sqrt(p_t*(1-p_t)/n_sessions_t + p_c*(1-p_c)/n_sessions_c); naive_ci = point +/- 1.96*naive_se.
width_ratio = (ci[1]-ci[0]) / (naive_ci[1]-naive_ci[0]).

Expected result: A point estimate identical to the full-data ratio difference, a percentile interval containing that point estimate, and a width ratio greater than 1 whose value approximates sqrt(1 + (m-1)*rho) for the data's mean cluster size m and intraclass correlation rho. A ratio of approximately 1.0 means sessions were resampled instead of users.

Verify it yourself

Construct data where every user has exactly one session: the cluster and naive intervals then agree to within Monte Carlo error, since m = 1 makes the design effect 1.
Construct data where converted is constant within each user: rho is 1, and the width ratio should approach sqrt(m).
The bootstrap distribution's mean sits close to the point estimate; a large offset means the resampled arms have drifted in size.
Doubling B narrows the Monte Carlo noise in the percentile endpoints without shifting the interval's centre.

Follow-up

Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?

Write a query using SQL window functions to calculate a rolling average of machine failure rates over the last 30 days.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Describe how you would join multiple disparate tables to build a feature set for a classification model.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Pair flow starts to completions within a thirty-minute bound

Medium · event-streams · conditional-aggregation · flow-funnel

fct_event holds event_id, flow_id, flow_instance_id, event_name, occurred_at_utc, surface, app_version. Using only rows where event_name is 'flow_started', 'flow_completed' or 'error_shown', compute the daily core-flow completion rate by surface and app_version. Numerator: distinct flow_instance_id whose flow_completed occurs within 30 minutes of its flow_started with no error_shown for the same instance in between. Denominator: distinct flow_instance_id with a flow_started that day. Report rows with a NULL flow_instance_id as a separate coverage count, not inside either side.

Approach

Collapse the stream to one row per flow_instance_id using conditional aggregation: MIN(occurred_at_utc) FILTER (WHERE event_name = 'flow_started') AS started, MIN(...) FILTER (WHERE event_name = 'flow_completed') AS completed, MIN(...) FILTER (WHERE event_name = 'error_shown') AS first_error. This makes the 'first' semantics explicit and avoids a self-join entirely.
Apply the predicates on that single row: completed IS NOT NULL AND completed <= started + interval '30 minutes' AND (first_error IS NULL OR first_error > completed). Note the last clause encodes 'in between' literally, so an error surfaced after a successful completion does not disqualify the attempt.
Bucket by started::date, surface and app_version. Splitting by app_version is not decoration: a completion-rate regression is nearly always confined to one client build, and a pooled daily rate hides it behind the installed base.
Count NULL flow_instance_id rows as a separate coverage figure. They cannot be attributed to an attempt at all, so they belong in neither numerator nor denominator, and their share tells you how much of the rate is unmeasurable.
For any roll-up to week or to all surfaces, re-sum the numerator and denominator rather than averaging the daily rates.

Worked solution (25 min)

Write the per-instance collapse CTE and confirm COUNT(*) equals COUNT(DISTINCT flow_instance_id).
Add the three predicates one at a time, recording how many instances each removes.
Aggregate by day, surface and app_version with FILTER on the clean-completion flag.
Compute the NULL-instance coverage count as a separate select and report it alongside.
Pick the worst day-surface-version cell and pull ten raw instances to confirm the disqualification reason.

Expected result: One row per (day, surface, app_version) with started, completed_clean and a rate bounded by 0 and 1, plus a scalar coverage count of events carrying a NULL flow_instance_id. Instances completing after 30 minutes appear in the denominator only.

Verify it yourself

No rate exceeds 1.
COUNT(*) equals COUNT(DISTINCT flow_instance_id) in the collapsed CTE.
Total started across cells equals the count of distinct flow_instance_id values carrying a flow_started event in the window.
Instances with an error strictly between start and completion are excluded from the numerator but present in the denominator.

Follow-up

occurred_at_utc is client-supplied. What do you do with an instance whose completion timestamp precedes its start?
flow_instance_id is missing on 4 percent of starts for one app_version. What can and cannot you conclude about that build?
An instance starts at 23:55 and completes at 00:04. Which day owns it, and what does the alternative do to the daily series?

How do you balance trade-offs between long-term product health and short-term performance gains?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Design a set of core metrics for a new digital service portal aimed at industrial clients.

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

If we notice a sudden drop in our customer engagement metrics, what is your systematic process for diagnosing the root cause?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you define the success of a new predictive maintenance feature for our compressors?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

What are the most common experimentation pitfalls you have encountered, and how do you mitigate them?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Say whether units interfere with each other, and switch design if they do.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

How do you determine the appropriate sample size for an A/B test in an environment with low traffic volume?

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.
Say whether units interfere with each other, and switch design if they do.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

Define success for halving the free trial length

Hard · monetisation · cohort horizon · trials

A proposal cuts the free trial from 30 days to 14. Trial-to-paid conversion is defined on fct_subscription_period: numerator, subscription_id whose first row with is_first_paid_period = TRUE has period_status IN ('active','past_due') and period_start_utc no later than 14 days after trial end; denominator, subscription_id whose first row has period_status = 'trialing' and period_start_utc in the cohort week. dim_account (account_id, account_created_at_utc, signup_surface) is also available and joins to fct_subscription_period on account_id. Explain why conversion rate alone cannot decide this, define the metric that can, and state the horizon and lag your readout needs. Deliverable: the decision metric and the readout schedule.

Approach

Show the incomparability numerically before arguing about it: at a fixed calendar readout date, a larger share of the 14-day arm's cohorts have completed trial end plus 14 days plus settlement, so the short arm leads mechanically and the lead shrinks as both arms mature.
Fix the censoring first, and do not mistake a rescaling for a fix. The given conversion rate already has trial starts as its denominator, so 'paid accounts per 1,000 trial starts' is that same ratio multiplied by 1,000 and answers nothing new. What the censoring needs is a fixed cohort age: read both arms at account_created_at_utc plus 60 days, which clears the long arm's full 30 + 14 + 3 = 47-day path to a settled first payment, so neither arm is censored at readout.
Then move the denominator upstream, which is a separate fix and the only one that catches a higher share of a smaller population: paid accounts per 1,000 dim_account rows created in the cohort week. Precondition — this diverges from the conversion rate only when trial length is visible before the trial starts, on pricing pages, in ads or in the signup flow. If the arms are assigned at trial start and nothing upstream differs, trial starts per 1,000 new accounts is equal across arms by construction and the two metrics are the same comparison rescaled. Report that ratio per arm and say which regime you are in rather than assuming one.
Add the leg neither denominator can see: a shorter trial converts users who had less time to reach activation, so read month-3 gross revenue churn and week-4 retention of the converted cohort, and read activation at trial day 7 to see whether the converted population changed composition.
Audit 'past_due' before comparing, since the definition counts it as converted: report the rate with and without it per arm, because a gain that sits entirely inside past_due is a billing artefact rather than a conversion effect.
Be explicit that the twelve-month consequence is not observable inside a planning cycle: pre-register that the decision is made on the 60-day metric, that the twelve-month cohort read will be published as a check, and what action follows if the two disagree.

Worked solution (35 min)

Tabulate, at a fixed calendar readout date, the share of each arm's cohorts that have completed trial end plus 14 days plus 3 days of settlement, and show the gap.
Define the decision metric as paid accounts per 1,000 dim_account rows created in the cohort week, evaluated at account_created_at_utc plus 60 days for both arms, and write the 30 + 14 + 3 = 47 arithmetic that justifies 60 as the common clock.
Report trial starts per 1,000 new accounts per arm beside it, so the reader can see whether the upstream denominator is doing any work in this test or whether it is arithmetically pinned to the conversion rate.
Compute the earliest honest readout date from the enrolment window: 60 days after the last account creation in the window plus a 3-day settlement lag, with no partial-cohort comparison permitted before it.
Add the quality leg: month-3 gross revenue churn, week-4 retention of converted accounts, and day-7 activation per arm.
Report the conversion rate with and without past_due per arm, and state which version the decision uses.

Expected result: A decision metric stated as paid accounts per 1,000 new accounts at a fixed 60-day cohort age measured from account creation, the trial-start rate published per arm so it is explicit whether that denominator change carries any information here, a readout date computed from the enrolment window, and at least two quality metrics capable of overturning a conversion-rate win.

Verify it yourself

Both arms must have identical elapsed time at readout: compute min and max of (readout_date minus account_created_at_utc) per arm and confirm the two ranges match.
Trial starts per 1,000 new accounts must be reported per arm. If that ratio is equal across arms, the conversion rate and the per-new-account metric are the same comparison rescaled and cannot diverge — so never present them as two independent confirmations. Only a gap in that ratio gives the per-new-account metric information the conversion rate does not carry.
The past_due share of the numerator should be similar across arms; a gap there is a payment-failure effect and does not belong in a conversion claim.

Follow-up

What happens to a user who would have converted on day 20, and how would you detect that population in the data?
The short arm has lower day-7 activation but higher conversion. Reconcile those two facts into one story.
What randomisation unit do you use here, and what goes wrong with the obvious alternatives?

Size every candidate cause of a trial-to-paid decline

Hard · cohort-maturity · experiment-readout · mix-shift · contribution-sizing

Trial-to-paid conversion on weekly trial-start cohorts from fct_subscription_period reads 3.1 points below the trailing eight-week mean for the three most recent cohorts. Three things happened in that window: a pricing experiment reached 50% of new trials, a payment processor migration added settlement delay, and paid_search spend tripled. Using fct_subscription_period, fct_experiment_exposure and dim_user, rank the causes by their contribution in points of the headline, state the remainder, and give the decision you would take on Monday.

Approach

Kill the immature cohorts first, because everything downstream is computed on them. The metric is lagged by the trial length plus a 14-day conversion window plus a settlement allowance, and a processor migration lengthens exactly that last term; recompute each cohort at a fixed cohort age rather than as of today, and confirm the newest cohort's value is still climbing day over day.
Hold the experiment analysis to the exposed population. Join fct_experiment_exposure on unit_id with is_in_analysis_population = TRUE rather than reading an assignment log, then check the variant split for a sample-ratio mismatch before believing any effect at all. Contribution to the headline is the variant effect multiplied by the exposed share, which is not the same number as the variant effect.
Decompose the cohort mix by dim_user.first_touch_channel using the same weight-times-rate arithmetic as any other mix question, so the paid_search increase is sized as a weight change at a measured conversion rate rather than asserted from the spend figure.
Convert all three to points of the headline, sum them, and print the residual against the historical week-to-week standard deviation of the metric. If the residual is inside that band, say so and stop looking; if it is outside, name what you would investigate next rather than leaving it implied.
Land the decision. Only one of the three is actionable on Monday, so state whether the experiment has accrued enough exposed units to stop at the pre-declared horizon, and state separately what the settlement-lag correction does to the published series and its lag rule.

Follow-up

How do you choose the fixed cohort age, and what do you lose by choosing it too long?
If the pricing variant is genuinely 1.2 points worse, does that settle whether to stop it? What else is on the other side of that decision?
The trailing eight-week mean spans the processor migration. What is the right baseline instead?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

How do you handle situations where you have conflicting priorities from different product stakeholders?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Defend a flat experiment readout against a post-hoc segment

Medium · experiments · segmentation · pushback

A feature you evaluated is flat on seven-day activation: +0.05pp with a 95% interval of [-0.47pp, +0.57pp], from 61,000 exposed users per arm in fct_experiment_exposure joined to dim_user and fct_event. Baseline activation is 32%. The launch team asks you to drop every surface except mobile_web, where the point estimate is +1.1pp, and re-run. You have ten minutes in their planning meeting. Deliver a spoken position: what you will and will not do, and the decision you recommend.

Approach

Recognise what is being probed: whether you hold a statistical position under social pressure without becoming either rigid or apologetic. A generic answer says the segment is not significant; a strong one separates the request into a question that is answerable (is the mobile_web number real?) and one that is not (can we ship on it?), and answers both.
Price the multiplicity out loud. The slice was chosen after seeing the results, so its estimate is selected on favourable noise and is biased away from zero. With k independent looks at a nominal 5% level, the chance of at least one false positive is 1 - 0.95^k: 26% at six segments, 64% at twenty. Quote the k you actually inspected, not the k you reported.
Use the arithmetic already in front of you. On the point estimates, a +1.1pp mobile_web effect combined with a pooled +0.05pp implies the remaining surfaces average negative in proportion to mobile_web's share of exposures. State that as a testable implication of their story rather than as a rebuttal of it.
Ask the one question that settles the category: was mobile_web named in the analysis plan before launch? If it was, it is a planned comparison and gets a corrected reading. If it was not, it is a hypothesis, and the honest move is to size the test that would confirm it.
Convert the refusal into a cost. Size a mobile_web-only confirmatory test at the claimed effect, state the weeks of mobile_web traffic it needs, and close with the recommendation: do not ship this as a lift, and note that the interval already rules out anything at or above +0.6pp, which is itself a useful input to the roadmap.

Follow-up

The confirmatory test you sized needs nine weeks of mobile_web traffic and the team has three. What do you recommend instead?
Suppose mobile_web was pre-registered. How does your reading change, and what correction do you apply?
Your interval excludes +0.6pp. Is that the same as saying the feature does nothing?

Walk through an analysis you got wrong and what changed

Easy · postmortem · data quality · self-assessment

Describe an analysis of yours that turned out to be wrong after somebody had already acted on it. You have four minutes. The account must name the defect mechanically, the join, the filter, the window or the identity key, rather than describing it as a communication problem. It must also say who did what because of the wrong number, how the error surfaced, how long it stood, and what control you put in place so that class of error cannot reach a decision again. Do not pick an error nobody acted on.

Approach

Recognise what is being probed: whether you can be specific about your own failure without minimising it or performing contrition. The discriminator is whether the defect has a mechanism the listener could reproduce in their own warehouse.
Choose the case by blast radius rather than by comfort. An error nobody acted on tests nothing, and picking one signals that you are managing the interview instead of answering it.
Structure the account in six beats: the number, the decision it drove, the defect, the detection, the correction, the control. Keep the defect to one reproducible sentence, for example an inner join to fct_subscription_period that dropped accounts with no subscription row and so computed retention over payers only.
State the direction of the bias, not only its existence. A filter or join that removes rows usually moves a metric predictably, and knowing which way shows you diagnosed the mechanism rather than patched the symptom.
Be exact about detection and elapsed time. 'A colleague noticed' and 'the row-count assertion failed before publication' are different answers about the same organisation, and the second one is the one your control is supposed to produce next time.
End on the control, its cost, whether it has fired since, and one thing it does not cover.

Follow-up

What did the control cost, and has it fired since? If it never has, how do you know it works?
How long did the wrong number stand before anyone questioned it, and what does that say about the review path it went through?
What is the equivalent mistake you are most likely to make in this role, given the tables you would be working in?