Interview guides · Hitachi Rail

Hitachi Rail Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Hitachi Rail: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

A Data Scientist at Hitachi Rail sits at the intersection of complex physical infrastructure and cutting-edge digital intelligence. You will be responsible for transforming vast streams of telemetry, operational, and financial data into actionable insights that enhance the reliability, safety, and efficiency of global rail networks. Whether you are optimizing predictive maintenance schedules or refining financial forecasting models, your work directly impacts the daily commute of millions and the operational sustainability of a world-class engineering firm.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

Define numerator, denominator and window precisely
Size an experiment before anyone launches it
Separate novelty effects from durable behaviour change

Overview

This role is critical because Hitachi Rail is transitioning from a traditional manufacturing entity into a digital-first leader in the transport sector. You will operate in a space where precision is paramount; your models don't just exist in a vacuum—they inform decisions that affect physical hardware, passenger safety, and multi-million dollar assets. You can expect a high-impact environment where you are encouraged to bridge the gap between abstract data science methodologies and the concrete realities of rail engineering and business operations.

The Hitachi Rail interview process

Hitachi Rail candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Video Screen

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Final Round

Where a loop ends with a senior leader, that conversation is rarely another skills test. The technical signal already exists by then, so the questions tend to open up: what you would look at first, where a metric you have heard about could mislead, what you would push back on. The decision being made is scope, which in practice means level and how much you would be trusted to own unsupervised. Treating it as a formality is the usual mistake. An open question late in the day is still being scored, and a vague answer reads as someone who has not run anything themselves.

What to demonstrate

Whether your view of the business has anything specific behind it, given that you are working only from what is public and are expected to say so
Whether the scope of work you describe owning matches the scope of the role, instead of sitting a level below it
Whether you can disagree with something concrete and stay useful about it, rather than agreeing with everything said in the room
Whether your questions are ones only this person could answer, as opposed to ones the recruiter already covered

How to prepare

Build one view you could defend for two minutes using only public information: what the funnel probably looks like, which metric likely drives decisions, and where that metric could mislead. Being wrong for a stated reason survives this round; having no view does not
Write down the largest piece of work you have owned from question to decision, who else touched it, and what you decided alone, then check that it reads at the level you are interviewing for
Prepare one thing you would want changed if you joined and phrase it as a question rather than a verdict, so it opens a conversation instead of closing one

Where candidates lose points

Treating last-touch attribution as the causal value of a channel

The attribution label on dim_user is the output of a rule that assigns full credit to whichever touch happened to be recorded last inside a lookback window, and that rule systematically rewards channels that sit close to the conversion, especially branded search and retargeting, which largely intercept demand that already existed. Reallocating spend on those labels moves budget toward the channels that are best at being last, which is why attributed return on ad spend often improves while total signups do not. Nothing in the touchpoint data can settle this, because the counterfactual of not running the channel was never observed. The credible reads are a geo holdout or a scheduled pause, sized in advance on the total-signups metric rather than on the attributed one, and the honest framing in the meantime is that the label describes correlation with conversion and not incremental contribution.

Watching an experiment daily and stopping when it crosses significance

A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.

Explaining an aggregate move without decomposing the mix shift

Split the change in the aggregate into within-segment movement and movement in segment weights before you explain it. Every segment's rate can fall while the overall rate rises, purely because volume shifted toward segments that already had higher rates.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Questions & practice

If we notice a sudden drop in our model’s performance metrics, how would you go about diagnosing the root cause?

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Audit a one-day event extract for structural defects

Easy · data quality · late arrival · pandas

You receive a one-day extract of fct_event as a DataFrame with event_id, occurred_at_utc, received_at_utc, visitor_id, user_id, account_id, event_name, is_bot_flagged and surface. Write a function returning one row per data-quality rule with the rule name, the failing row count and the failing share of the extract. Cover at minimum: duplicate event_id, received_at_utc earlier than occurred_at_utc, occurred_at_utc later than the extract's maximum received_at_utc, account_id present while user_id is NULL, and rows whose occurred_at date differs from their received_at date. Do not drop rows; report only.

Approach

Compute the extract's own reference clock first: max(received_at_utc). Wall-clock now() is wrong here because the extract may be replayed days later, which would turn every row into a future-dated failure.
Express each rule as a boolean Series over the same index so the checks compose, then aggregate with .sum() and divide by len(df). Building a list of (name, mask) pairs keeps the rule set extensible and keeps one code path for counting.
For the duplicate rule, decide and state the convention: df.duplicated('event_id', keep=False).sum() counts every member of a duplicated group, df.duplicated('event_id').sum() counts only the surplus copies. Either is defensible; an unstated choice is not. The rest of this item assumes keep=False.
Treat received_at < occurred_at as clock skew, not corruption: occurred_at is client-supplied. Separate it from the date-mismatch rule, which is the one that actually breaks a daily metric keyed on occurred_at.
Know which rules imply which before you read the counts. A row whose occurred_at exceeds max(received_at_utc) has its own received_at no later than that maximum, so it is necessarily a clock-skew row as well: the future-dated mask is a subset of the skew mask, always. Neither is a subset of the date-mismatch mask, because skew of a few minutes inside one UTC date mismatches nothing.
Return a tidy DataFrame sorted by failing_share descending, and add a boolean column saying whether the rule should block publication, so the output is a decision rather than a list of numbers.

Worked solution (20 min)

Parse both timestamp columns with utc=True and assert the dtype, since a silently-object column makes every comparison string-wise and wrong.
Set ref = df['received_at_utc'].max() and build the five masks against it.
Assemble results as pd.DataFrame(rows) with columns rule, failing_rows, failing_share, blocks_publication.
Keep the masks addressable (a dict of name to Series) rather than only their sums, so the overlap between rules can be asserted rather than assumed.
Verify the function is pure: assert the input frame's shape is unchanged after the call.

Expected result: A DataFrame with one row per rule, failing_share equal to failing_rows divided by len(df) for every row, and the input frame returned unmodified. The rules overlap rather than partition the extract, and one containment is structural: every future-dated row is also a clock-skew row. The counts therefore must not be summed or presented as a total.

Verify it yourself

Plant violations deliberately and assert an exact expected vector rather than assuming one hit per rule. On a 10-row frame carrying two rows that share an event_id, one same-date skew row, one row whose occurred_at sits an hour past max(received_at_utc), one row with account_id set and user_id NULL, and one row whose occurred_at falls on the previous UTC date: duplicate_event_id is 2 under the keep=False convention, clock_skew is 2 (the planted skew row plus the future-dated row, which cannot avoid tripping it), future_dated is 1, orphan_account is 1, date_mismatch is 1.
Assert the containment directly rather than by counting: (future_dated_mask & ~clock_skew_mask).sum() is 0. A non-zero value means the reference clock was not max(received_at_utc).
Feed a clean frame and confirm all counts are 0 and no rule errors on empty masks.
Confirm len(input) is identical before and after, since the deliverable is a report and not a cleaned frame.

Follow-up

The date-mismatch count is 2.1 percent on this extract. What late-arrival rule would you write for a daily metric, and how many days would you hold the number open?
Duplicate event_id values appear only on the 'core_action_completed' event. What upstream cause would you check before deduplicating?
Which of these rules should fire an alert at the pipeline, and which should only appear in a weekly review?

Cluster bootstrap for a per-session rate randomised on users

Hard · cluster bootstrap · randomisation unit · variance

An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.

Approach

Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.

Follow-up

Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?

Describe how you would handle missing data points in a time-series dataset using SQL.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Given a table of event logs, how would you identify the first and last interaction of a user session?

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

How do you optimize a query that is performing poorly on a large telemetry dataset?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Weekly visit-to-signup conversion split by acquisition channel

Easy · funnel · conversion · aggregation

From fct_session (session_id, visitor_id, started_at_utc, referrer_channel, is_bot_flagged, consent_state) and fct_event (visitor_id, occurred_at_utc, event_name), compute visit-to-signup conversion for one ISO week, split by channel. session_id is the unique key of fct_session. Denominator: distinct visitor_id with a session starting in the week, is_bot_flagged = FALSE and consent_state <> 'denied'. Numerator: those visitors with a 'signup_completed' event in the same week. Label each visitor with the referrer_channel of their first session in the window. Return channel, visitors, signups and rate, plus one all-channel total row.

Approach

Build a visitor spine that is one row per visitor: filter sessions to the week, drop is_bot_flagged and consent_state = 'denied', then take the first session per visitor with ROW_NUMBER() OVER (PARTITION BY visitor_id ORDER BY started_at_utc, session_id) = 1 to carry the channel label. Collapsing to one row here is what makes the channel buckets mutually exclusive and the totals additive.
session_id is the unique key, so that ordering is total and the label is reproducible. If the table carried no unique key you would have to write an explicit tie rule instead, because two sessions on different channels at the identical timestamp would otherwise label the visitor differently between runs.
Attach the outcome as a semi-join (EXISTS on a signup_completed event for that visitor inside the same week) rather than a join to the event table, so a visitor who fires the event twice does not count twice and inflate the numerator past the denominator.
Aggregate with COUNT() as visitors and COUNT() FILTER (WHERE signed_up) as signups, and compute the rate as signups::numeric / NULLIF(visitors, 0) so an empty channel returns NULL rather than a division error.
Produce the total with GROUP BY GROUPING SETS ((channel), ()), which re-sums numerator and denominator for the total row. Averaging the channel rates gives a different and wrong number whenever channel volumes differ, which they always do.
Verify the spine before trusting the output: COUNT(*) must equal COUNT(DISTINCT visitor_id), and the per-channel visitor counts must sum to the total row.

Worked solution (20 min)

Write the filtered session CTE and check its row count against an unfiltered count, so you know how much volume the bot and consent filters removed.
Add the ROW_NUMBER first-session pick and assert one row per visitor.
Add the EXISTS outcome flag and aggregate with FILTER.
Add GROUPING SETS for the total and format the rate to four decimal places.
Spot-check one channel by hand: pull its visitor list, count signups directly, compare.

Expected result: One row per referrer_channel plus a total row, with visitors, signups and a decimal rate. Channels with zero signups appear with rate 0, not missing. The total rate lies strictly between the minimum and maximum channel rate unless one channel holds all the volume.

Verify it yourself

Per-channel visitor counts sum exactly to the total row's visitor count.
COUNT(*) equals COUNT(DISTINCT visitor_id) on the spine CTE.
Replacing the EXISTS with a plain inner join changes the signup count only if duplicate signup_completed events exist; if it does change, say by how much.
Every rate is between 0 and 1.
Re-running the first-session pick with session_id removed from the ORDER BY returns the same labels; if it does not, visitors hold same-timestamp sessions on different channels and the tie rule is doing real work.

Follow-up

The denominator is distinct visitors. If a browser release shortens cookie lifetime, what happens to this rate, and how would you tell that apart from a genuine drop?
A visitor's first session is direct and their signup session is paid search. Your label says direct. When is that the wrong answer for the decision being made?
How do you roll four weeks into a month, and why is averaging the four weekly rates wrong?

How do you define "success" for a system designed to optimize train scheduling?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you design a metric to measure the success of a new predictive maintenance feature?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you determine the sample size required for a valid experiment?

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

How would you handle interference between treated and control units?
What would you do if you could not randomise at all?

How do you account for seasonality when evaluating the results of an experiment?

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Name the randomisation unit first; it decides the variance and what the test can detect.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

Define success for halving the free trial length

Hard · monetisation · cohort horizon · trials

A proposal cuts the free trial from 30 days to 14. Trial-to-paid conversion is defined on fct_subscription_period: numerator, subscription_id whose first row with is_first_paid_period = TRUE has period_status IN ('active','past_due') and period_start_utc no later than 14 days after trial end; denominator, subscription_id whose first row has period_status = 'trialing' and period_start_utc in the cohort week. dim_account (account_id, account_created_at_utc, signup_surface) is also available and joins to fct_subscription_period on account_id. Explain why conversion rate alone cannot decide this, define the metric that can, and state the horizon and lag your readout needs. Deliverable: the decision metric and the readout schedule.

Approach

Show the incomparability numerically before arguing about it: at a fixed calendar readout date, a larger share of the 14-day arm's cohorts have completed trial end plus 14 days plus settlement, so the short arm leads mechanically and the lead shrinks as both arms mature.
Fix the censoring first, and do not mistake a rescaling for a fix. The given conversion rate already has trial starts as its denominator, so 'paid accounts per 1,000 trial starts' is that same ratio multiplied by 1,000 and answers nothing new. What the censoring needs is a fixed cohort age: read both arms at account_created_at_utc plus 60 days, which clears the long arm's full 30 + 14 + 3 = 47-day path to a settled first payment, so neither arm is censored at readout.
Then move the denominator upstream, which is a separate fix and the only one that catches a higher share of a smaller population: paid accounts per 1,000 dim_account rows created in the cohort week. Precondition — this diverges from the conversion rate only when trial length is visible before the trial starts, on pricing pages, in ads or in the signup flow. If the arms are assigned at trial start and nothing upstream differs, trial starts per 1,000 new accounts is equal across arms by construction and the two metrics are the same comparison rescaled. Report that ratio per arm and say which regime you are in rather than assuming one.
Add the leg neither denominator can see: a shorter trial converts users who had less time to reach activation, so read month-3 gross revenue churn and week-4 retention of the converted cohort, and read activation at trial day 7 to see whether the converted population changed composition.
Audit 'past_due' before comparing, since the definition counts it as converted: report the rate with and without it per arm, because a gain that sits entirely inside past_due is a billing artefact rather than a conversion effect.
Be explicit that the twelve-month consequence is not observable inside a planning cycle: pre-register that the decision is made on the 60-day metric, that the twelve-month cohort read will be published as a check, and what action follows if the two disagree.

Worked solution (35 min)

Tabulate, at a fixed calendar readout date, the share of each arm's cohorts that have completed trial end plus 14 days plus 3 days of settlement, and show the gap.
Define the decision metric as paid accounts per 1,000 dim_account rows created in the cohort week, evaluated at account_created_at_utc plus 60 days for both arms, and write the 30 + 14 + 3 = 47 arithmetic that justifies 60 as the common clock.
Report trial starts per 1,000 new accounts per arm beside it, so the reader can see whether the upstream denominator is doing any work in this test or whether it is arithmetically pinned to the conversion rate.
Compute the earliest honest readout date from the enrolment window: 60 days after the last account creation in the window plus a 3-day settlement lag, with no partial-cohort comparison permitted before it.
Add the quality leg: month-3 gross revenue churn, week-4 retention of converted accounts, and day-7 activation per arm.
Report the conversion rate with and without past_due per arm, and state which version the decision uses.

Expected result: A decision metric stated as paid accounts per 1,000 new accounts at a fixed 60-day cohort age measured from account creation, the trial-start rate published per arm so it is explicit whether that denominator change carries any information here, a readout date computed from the enrolment window, and at least two quality metrics capable of overturning a conversion-rate win.

Verify it yourself

Both arms must have identical elapsed time at readout: compute min and max of (readout_date minus account_created_at_utc) per arm and confirm the two ranges match.
Trial starts per 1,000 new accounts must be reported per arm. If that ratio is equal across arms, the conversion rate and the per-new-account metric are the same comparison rescaled and cannot diverge — so never present them as two independent confirmations. Only a gap in that ratio gives the per-new-account metric information the conversion rate does not carry.
The past_due share of the numerator should be similar across arms; a gap there is a payment-failure effect and does not belong in a conversion claim.

Follow-up

What happens to a user who would have converted on day 20, and how would you detect that population in the data?
The short arm has lower day-7 activation but higher conversion. Reconcile those two facts into one story.
What randomisation unit do you use here, and what goes wrong with the obvious alternatives?

Decide whether a one-day core-action drop is real

Easy · seasonality · day-of-week · late-arriving-data

A daily dashboard counts distinct fct_event.user_id with is_core_action = TRUE, filtered on occurred_at_utc, and is read at 09:00 UTC. This morning it shows yesterday down 22% against the day before. fct_event is partitioned on received_at_utc. You have fct_event, fct_session and dim_user with thirteen months of history. Deliver a one-paragraph verdict, escalate or do not escalate, with the evidence that settles it, before anyone proposes a product hypothesis.

Approach

Identify which two weekdays the comparison actually spans, then pull the same weekday-pair transition for the last 52 weeks and place the observed 22% inside that distribution. A day-over-day comparison in a product with a weekday pattern is a comparison of two different populations, so the reference class is the same transition historically, not the prior day.
Measure partition completeness rather than assuming it. For each of the last 30 days compute the share of that day's occurred_at_utc rows that had landed by 09:00 UTC the following morning, split by surface; mobile clients buffer events offline, so the freshest partition is systematically short and the shortfall is not uniform across surfaces.
Recompute the same series keyed on received_at_utc. If the drop survives on both keys it is not a lateness artefact; if it exists only on occurred_at_utc it is the partition filling in.
Check the two exclusion flags before segmenting anything: a change in is_bot_flagged coverage or a batch of is_internal accounts entering or leaving moves a distinct-user count with no user behaviour behind it.
Only if the movement survives all of the above, begin the segment decomposition. Say explicitly in the verdict which of these four checks the movement passed, so the next reader does not repeat them.

Follow-up

What publication lag would you set for this dashboard, and how would you derive the number rather than pick it?
If you switch the metric to received_at_utc, what does that break for anyone comparing to historical figures?
How would you detect the same problem automatically, so a human does not have to notice it each morning?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Tell me about a project where you had to pivot your approach due to unexpected data limitations.

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Choose between three teams' requests with one analyst-week

Medium · prioritisation · stakeholder · judgement

You have one analyst-week. Three requests land the same morning. A growth team wants a paid-channel readout before a Friday spend decision. A billing team wants gross monthly revenue churn rebuilt, because the current figure recognises cancellation at canceled_at_utc rather than period_end_utc and is therefore wrong. A product team wants a dashboard for a feature launching in six weeks. All three sponsors are peers of your manager. Deliver your ranking, the explicit rule that produced it, and the message you send to the two teams you defer.

Approach

Recognise what is being probed: whether you prioritise on decision value and reversibility or on who asked most recently and most loudly. The generic answer sorts by importance; the strong one states a rule, applies it, and accepts the ranking it produces even where that is uncomfortable.
Score each request on three statable things: the decision it unblocks and the date that decision is made, the cost of being wrong in the meantime, and whether the work is one-off or compounding. A wrong published churn figure compounds, because it is quoted downstream and enters forecasts; the channel readout has a fixed date that cannot move; the dashboard has six weeks of slack.
Notice the tension between value and urgency rather than resolving it by feel. The churn defect is the most valuable item and the least urgent one, which is exactly the shape of work that never gets done.
Break the churn item in two. A one-hour severity check, sizing the gap between the two recognition points in MRR, is cheap enough to do before ranking anything and may promote the item outright. Do that first, then rank.
Make the deferrals concrete. Each deferred team gets a date, a reason expressed as another team's decision deadline rather than as relative importance, and the smallest thing you can hand them immediately.

Follow-up

The dashboard team escalates to your manager. What do you say in that conversation?
Your severity check shows churn is overstated by 15%. Does the ranking change, and does anybody need to be told today regardless of the ranking?
A fourth request arrives Wednesday with a Thursday deadline. What comes off the list, and who do you tell first?

Handle a request for numbers supporting a decision already made

Hard · integrity · framing · stakeholder

A senior leader has already decided to sunset a plan tier and asks you for the analysis showing it is the right call. Accounts on that tier carry 6% of MRR at constant FX and have the highest licensed-seat utilisation in the book. The leader's support matters to your next review cycle, and the decision is being presented in four days. Deliver what you produce, what you decline to produce, and the exact sentence you will say in the meeting where the number appears on a slide.

Approach

Recognise what is being probed: whether you can find the legitimate request inside an illegitimate framing instead of either complying or refusing on principle. The generic answer promises to push back; the strong one produces something genuinely useful and states its limits in the room, without ambushing anybody.
Separate the decision from the justification. Sunsetting the tier may be correct for reasons the data does not hold, such as support cost, roadmap surface area or sales motion. What you decline is a one-sided document. What you produce is the case read both ways, which also happens to be more useful to the leader.
Build the symmetric analysis: MRR at risk at constant FX, the share of affected accounts with a plausible migration path given seats_licensed and billing_term, the recovery rate assumed for that migration and where it came from, and the downside case in which high-utilisation accounts treat the sunset as a reason to re-evaluate the vendor entirely.
Surface the inconvenient fact privately and early. The highest seat utilisation in the book is a retention signal, and the leader should hold it before the room does, so they can incorporate it rather than be caught by it.
Agree the meeting sentence in advance with the leader, so that nobody is surprised. Something to the effect that the tier is 6% of MRR and its accounts are the most heavily used in the book, and that the case for sunsetting rests on cost and focus rather than on revenue. That is true, it supports the decision on its real grounds, and it stops the deck claiming the numbers endorse it.
Decide your own line before you need it: what you will not put your name to, and that the route if asked anyway is your own manager rather than a confrontation in the meeting.

Follow-up

The deck circulates with your analysis included and the downside case removed. What do you do, and by when?
What changes if the honest analysis says the sunset is clearly the wrong call?
How do you write the same memo when the leader is your skip-level and the meeting is tomorrow?