AutoScout24 Data Scientist Interview Questions & Guide 2026

The 60-second brief

As a Data Scientist at AutoScout24, you sit at the intersection of Europe’s largest online automotive marketplace and advanced data-driven decision-making. Your work directly influences how millions of users search for, compare, and purchase vehicles. By leveraging massive datasets, you are tasked with optimizing product features, refining search algorithms, and providing the analytical rigor necessary to maintain the company’s market-leading position.

Learn the economics of the product category before the loop. Marketplaces, subscription products and ad-supported products turn on different core quantities (match rate and liquidity, retention and churn, fill rate and yield) and fail in different characteristic ways.

Focus areas

• Decompose a metric move by segment and mix
• Pick a randomisation unit that respects interference
• Size an experiment before anyone launches it

Overview

The role demands more than just technical proficiency; it requires a strong product mindset. You will work closely with cross-functional teams, including engineering, product management, and business stakeholders, to translate complex business problems into actionable models and experiments. Whether you are improving recommendation engines or designing metrics to track feature success, your contributions are critical to the company’s goal of creating a seamless, transparent, and efficient automotive buying experience.

The team highly values candidates who can bridge the gap between technical complexity and business value. Be prepared to explain not just how your model works, but why it is the right solution for the business problem at hand.

The AutoScout24 interview process

AutoScout24 candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

• Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
• Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
• Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

• Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
• Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
• Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Assessment

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

• Whether the query you write matches the plan you just described
• What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
• Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

• Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
• Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
• Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Deep-Dive Interviews

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

• Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
• The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
• Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

• Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
• Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
• Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Where candidates lose points

Treating last-touch attribution as the causal value of a channel

The attribution label on dim_user is the output of a rule that assigns full credit to whichever touch happened to be recorded last inside a lookback window, and that rule systematically rewards channels that sit close to the conversion, especially branded search and retargeting, which largely intercept demand that already existed. Reallocating spend on those labels moves budget toward the channels that are best at being last, which is why attributed return on ad spend often improves while total signups do not. Nothing in the touchpoint data can settle this, because the counterfactual of not running the channel was never observed. The credible reads are a geo holdout or a scheduled pause, sized in advance on the total-signups metric rather than on the attributed one, and the honest framing in the meantime is that the label describes correlation with conversion and not incremental contribution.

Reading a pooled rate that moved because the mix moved, not because any behaviour changed

A pooled conversion rate is a weighted average, and a shift in the weights can move it in the opposite direction to every one of its parts. A paid campaign that brings low-converting traffic drops overall signup conversion even if desktop, mobile web and app conversion each rose that week, which is Simpson's paradox and it is the single most common cause of an inexplicable dashboard move. The discipline is to decompose before explaining: recompute the rate holding last period's segment weights fixed, and compare that counterfactual to the actual, so the mix effect and the rate effect are separated numerically rather than argued about. Segment on the dimensions that actually reweight, which in this domain are almost always device_type, referrer_channel, country and new versus returning.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Questions & practice

Audit a one-day event extract for structural defects

Easy · data quality · late arrival · pandas

You receive a one-day extract of fct_event as a DataFrame with event_id, occurred_at_utc, received_at_utc, visitor_id, user_id, account_id, event_name, is_bot_flagged and surface. Write a function returning one row per data-quality rule with the rule name, the failing row count and the failing share of the extract. Cover at minimum: duplicate event_id, received_at_utc earlier than occurred_at_utc, occurred_at_utc later than the extract's maximum received_at_utc, account_id present while user_id is NULL, and rows whose occurred_at date differs from their received_at date. Do not drop rows; report only.

Approach

1. Compute the extract's own reference clock first: max(received_at_utc). Wall-clock now() is wrong here because the extract may be replayed days later, which would turn every row into a future-dated failure.
2. Express each rule as a boolean Series over the same index so the checks compose, then aggregate with .sum() and divide by len(df). Building a list of (name, mask) pairs keeps the rule set extensible and keeps one code path for counting.
3. For the duplicate rule, decide and state the convention: df.duplicated('event_id', keep=False).sum() counts every member of a duplicated group, df.duplicated('event_id').sum() counts only the surplus copies. Either is defensible; an unstated choice is not. The rest of this item assumes keep=False.
4. Treat received_at < occurred_at as clock skew, not corruption: occurred_at is client-supplied. Separate it from the date-mismatch rule, which is the one that actually breaks a daily metric keyed on occurred_at.
5. Know which rules imply which before you read the counts. A row whose occurred_at exceeds max(received_at_utc) has its own received_at no later than that maximum, so it is necessarily a clock-skew row as well: the future-dated mask is a subset of the skew mask, always. Neither is a subset of the date-mismatch mask, because skew of a few minutes inside one UTC date mismatches nothing.
6. Return a tidy DataFrame sorted by failing_share descending, and add a boolean column saying whether the rule should block publication, so the output is a decision rather than a list of numbers.

Follow-up

• The date-mismatch count is 2.1 percent on this extract. What late-arrival rule would you write for a daily metric, and how many days would you hold the number open?
• Duplicate event_id values appear only on the 'core_action_completed' event. What upstream cause would you check before deduplicating?
• Which of these rules should fire an alert at the pipeline, and which should only appear in a weekly review?

Rebuild per-visitor ordering without groupby convenience methods

Easy · pandas · vectorisation · window logic

You have a DataFrame of 2 million fct_event rows with visitor_id, occurred_at_utc and event_id, unsorted and containing duplicate timestamps within a visitor. Produce three new columns: event_rank, the 1-based position of the event within its visitor ordered by occurred_at_utc; seconds_since_prev, the gap to that visitor's previous event, NULL for the first; and is_first_for_visitor. You may use sort_values, shift, cumsum, numpy and boolean masking. You may not use groupby.transform, groupby.apply, groupby.cumcount, groupby.rank or merge_asof. Break timestamp ties on event_id.

Approach

1. Sort once by ['visitor_id', 'occurred_at_utc', 'event_id'] and reset the index. The whole exercise reduces to row arithmetic on a sorted frame, and the tiebreak on event_id is what makes the result reproducible across runs.
2. Mark visitor boundaries with is_first = df['visitor_id'].ne(df['visitor_id'].shift()). This is the single fact every other column derives from.
3. Compute seconds_since_prev as the diff of the timestamp column, then overwrite it with NaT/NaN wherever is_first is True. The shift crosses the boundary between visitors and will otherwise hand the first row of each visitor the last event of the previous one.
4. Build event_rank from a running counter that resets at boundaries: take a global cumulative position (np.arange(len(df))) and subtract, per row, the global position at which that visitor started. Get the start position by forward-filling the positions where is_first is True, which is a cumsum-free reset and is O(n).
5. Verify against the forbidden method once, as a test rather than as the implementation, and confirm the two agree on every row.

Worked solution (20 min)

1. Sort on the three-key tuple and reset_index(drop=True).
2. Compute is_first via .ne(.shift()), which is True for row 0 because the shifted value is NaN.
3. pos = np.arange(len(df)); start = pd.Series(np.where(is_first, pos, np.nan)).ffill(); event_rank = (pos - start + 1).astype(int).
4. gap = df['occurred_at_utc'].diff().dt.total_seconds(); gap[is_first] = np.nan.
5. Assert event_rank equals df.groupby('visitor_id').cumcount() + 1 on the sorted frame.

Expected result: Three columns on the sorted frame: event_rank starting at 1 for every visitor and increasing by 1 with no gaps, seconds_since_prev null exactly where is_first_for_visitor is True, and is_first_for_visitor summing to df['visitor_id'].nunique().

Verify it yourself

• is_first_for_visitor.sum() equals the number of distinct visitor_id values.
• seconds_since_prev.isna().sum() equals that same number.
• event_rank.max() per visitor equals that visitor's row count, and event_rank is non-negative everywhere.
• A visitor with two events at the identical timestamp gets ranks 1 and 2 with a gap of 0.0, ordered by event_id.

Follow-up

• The frame does not fit in memory. How does your approach change if you can only process one visitor-partitioned chunk at a time?
• occurred_at_utc is client-supplied and sometimes runs backwards within a visitor. Does your seconds_since_prev go negative, and should it?
• How would you extend this to reset the counter at every change of surface as well as visitor?

Sessionise an event stream with gap and midnight rules

Hard · sessionisation · vectorisation · event streams

Sessionise a raw event stream. Input: a DataFrame with visitor_id, user_id (often NULL), occurred_at_utc and event_name, unsorted, up to 5 million rows. A session breaks when the gap from that visitor's previous event exceeds 30 minutes, and is force-closed at UTC midnight so no session spans two calendar dates. A gap of exactly 30 minutes does not break. Emit one row per session with session_id, visitor_id, the user_id as of the last event in the session, started_at_utc, ended_at_utc, session_date, duration_seconds and event_count. Vectorise; do not loop per visitor.

Approach

1. Sort by ['visitor_id', 'occurred_at_utc', 'event_id'] once, then express the whole problem as one boolean vector: a row starts a new session when the visitor changed, or the gap exceeds 30 minutes, or the UTC date differs from the previous row's UTC date. Cumsum that vector and you have the session key.
2. Get the comparison direction right on the gap: the rule is strictly greater than 1800 seconds, so an event at exactly 1800 seconds continues the session. Write it as gap > pd.Timedelta(minutes=30), and make the tie a test case rather than an assumption.
3. Derive the midnight break from the date change, not from inserting synthetic boundary rows. A date change implies a break even when the gap is two seconds, which is precisely the force-close rule and is why the two conditions are ORed rather than one subsuming the other.
4. Aggregate with a single groupby on the session key: min and max of occurred_at_utc, size for event_count, and last for user_id, which is correct because the frame is already sorted so 'last' is the final event in the session. That is the identity-as-of-session-end rule.
5. Compute duration_seconds as (max - min).dt.total_seconds(), which makes a single-event session 0 seconds. Say so explicitly, because a downstream mean session duration is sensitive to whether single-event sessions are 0 or excluded.

Follow-up

• Sessions are used as the denominator of a conversion rate. How does moving the inactivity gap from 30 to 45 minutes move that rate, and in which direction?
• A visitor's clock is 40 minutes ahead, so their events arrive with future occurred_at values. What does your sessioniser do, and what would you rather it did?
• The same person signs up mid-session on mobile and continues on desktop. How many sessions and how many users does your output show, and is that the right answer?

Write a query using SQL window functions to calculate the running total of vehicle listings by category.

Medium · sql and data manipulation

Approach

1. Say which table is the grain you start from, and join outward from it.
2. Compute rates by summing numerator and denominator separately, never by averaging rates.
3. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

• How does the query change if the join becomes one-to-many?
• How would you verify this result without re-running the same query?

Given a table of user sessions, how would you calculate the conversion rate per user segment?

Medium · sql and data manipulation

Approach

1. Check whether any join is one-to-many before aggregating, or the sums inflate.
2. State the window function and its partition and ordering out loud before writing it.
3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

• How would you verify this result without re-running the same query?
• How does the query change if the join becomes one-to-many?

Find reactivation gaps in account paid-period history

Hard · gaps-and-islands · window-functions · subscriptions

fct_subscription_period holds account_id, subscription_id, period_start_utc, period_end_utc, period_status and change_reason. A mid-period plan or seat change closes one row and opens another, so a single continuous paid tenure is often many rows, and an account may hold two overlapping subscriptions. Collapse rows with period_status in ('active','past_due') into continuous tenures per account, treating gaps of three days or less as continuous. Return account_id, tenure_start, tenure_end, and for every tenure after the first, the gap in days that preceded it.

Approach

1. Filter to paid rows only: period_status IN ('active','past_due'). Trialing periods are not tenure, and including them turns every trial that never converted into a one-period tenure followed by a fake churn.
2. Order by period_start_utc and take a running maximum of all prior ends: MAX(period_end_utc) OVER (PARTITION BY account_id ORDER BY period_start_utc, period_end_utc, subscription_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING). Those three columns are the only stable ordering this schema exposes, so check first that they are unique within an account; if rows tie on all three, the island numbering is order-dependent between runs and you need a real row key before the result is reproducible.
3. LAG on its own is wrong here because with overlapping or nested periods the immediately preceding row by start date is not the one that ends latest, so the running maximum is the part that cannot be shortcut.
4. Flag a new island when prior_max_end IS NULL OR period_start_utc > prior_max_end + interval '3 days', then number islands with a running SUM of the flag over the same ordering and an explicit ROWS frame.
5. Group to (account_id, island) taking MIN(period_start_utc) and MAX(period_end_utc), then LAG(tenure_end) OVER (PARTITION BY account_id ORDER BY tenure_start) to compute the preceding gap in days for every tenure after the first.
6. Sanity-check with change_reason, which is the only lineage this schema carries: list its distinct values first, then confirm that rows recording a plan or seat change sit inside a tenure rather than opening one, and that every tenure after the first opens on a row whose reason records a restart rather than an ordinary renewal. Do not reconcile against a churn timestamp on dim_account, which this schema does not define; and where such a column does exist, a cancellation timestamp records when the request was made and routinely sits weeks before the period it ends.

Worked solution (35 min)

1. Find an account with a known mid-period upgrade and dump its period rows to use as the trace case.
2. Check that (period_start_utc, period_end_utc, subscription_id) is unique per account, since the whole ordering rests on it.
3. Write the paid-rows CTE and the running MAX with the explicit frame.
4. Add the island flag and the running SUM, then verify the trace account yields one island.
5. Group to tenures and add the LAG-based gap in days.
6. List the distinct change_reason values, then count accounts with more than one tenure and compare against the count of accounts carrying a restart-flavoured reason anywhere in their history.

Expected result: One row per (account_id, tenure). An account with a continuous history returns a single row with a NULL preceding gap. Every non-first tenure has gap_days strictly greater than 3 by construction.

Verify it yourself

• No two tenures for the same account overlap or touch within the grace window.
• MIN(tenure_start) per account equals MIN(period_start_utc) per account in the filtered input.
• The account with a known mid-period upgrade produces one tenure, not two.
• (period_start_utc, period_end_utc, subscription_id) has no duplicates within an account; if it does, say how many accounts are affected before reading the output.
• The share of accounts with more than one tenure is small; if it is most accounts, the status filter or the grace window is wrong.

Follow-up

• Why three days of grace? What do 0 and 30 days each do to the count of accounts classed as reactivated?
• An account runs two concurrent subscriptions for different teams. One tenure or two, and what does the revenue reader expect?
• How would you turn these tenures into a monthly gross logo churn series without double-counting an account that churned and returned in the same month?

If we introduce a new "Price Alert" feature, what KPIs would you track to determine its effectiveness?

Medium · product sense and metrics

Approach

1. Restate the decision this analysis has to support, and who acts on the answer.
2. Decompose the metric into the rates that drive it, and say which one you would check first.
3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

• How would you detect that the metric is being gamed rather than genuinely improving?
• Which segment would you cut first, and what would that rule out?

How would you design a metric to measure the success of a new search filter?

Medium · product sense and metrics

Approach

1. Fix the population and the time window before naming any metric.
2. Name one primary metric, then the guardrail that stops it being gamed.
3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

• What would you do if the primary metric and the guardrail moved in opposite directions?
• How would you detect that the metric is being gamed rather than genuinely improving?

How do you balance user experience with business monetization goals in a marketplace?

Medium · product sense and metrics

Approach

1. Restate the decision this analysis has to support, and who acts on the answer.
2. State what result would change your recommendation, so the answer is falsifiable.
3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

• How would you detect that the metric is being gamed rather than genuinely improving?
• What would you do if the primary metric and the guardrail moved in opposite directions?

Talk about a project where you had to prioritize speed over perfection—how did you manage the trade-offs?

Medium · product sense and metrics

Approach

1. State what result would change your recommendation, so the answer is falsifiable.
2. Name one primary metric, then the guardrail that stops it being gamed.
3. Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

• Which segment would you cut first, and what would that rule out?
• What would you do if the primary metric and the guardrail moved in opposite directions?

How would you handle a situation where an A/B test shows significant results for one metric but not for another?

Medium · a/b testing and experimentation

Approach

1. Name the guardrails that would stop a launch even on a positive primary result.
2. Decide the analysis before seeing data, including how long it runs and when you look.
3. State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

• What would you conclude if the result is positive but the test is underpowered?
• What would you do if you could not randomise at all?

What are the most common experimentation pitfalls that could lead to false positives?

Medium · a/b testing and experimentation

Approach

1. Name the guardrails that would stop a launch even on a positive primary result.
2. Say whether units interfere with each other, and switch design if they do.
3. Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

• What would you conclude if the result is positive but the test is underpowered?
• What would you do if you could not randomise at all?

Size an activation test before anyone writes code

Easy · power · minimum detectable effect · proportions

A team wants to test a new onboarding checklist against the seven-day activation rate: users with is_core_action = TRUE events on at least two distinct UTC dates inside the first seven days, counted from dim_user.account_created_at_utc with is_internal = FALSE. Baseline is 22%. Weekly non-internal signups are 9,000, split evenly between two arms. State the minimum detectable effect after two weeks of enrolment, the enrolment needed to detect a 1.5 percentage point absolute lift, and the earliest date a readout would be honest.

Approach

1. Use the two-proportion sizing shortcut n = 16 p(1-p) / d^2 per arm, where d is the absolute lift. The 16 is 2(1.96 + 0.84)^2 = 15.7 rounded, i.e. two-sided alpha 0.05, 80% power, equal arms.
2. Invert it for the MDE at fixed n: d = sqrt(16 p(1-p) / n). Two weeks gives 9,000 per arm, so d = sqrt(2.746 / 9000) = 0.0175, which is 1.75 percentage points absolute or 7.9% relative on a 22% base.
3. Run it forward for d = 0.015: n = 2.746 / 0.000225 = 12,202 per arm, 24,404 total, about 2.7 weeks of enrolment. Round up to three whole weeks so the enrolment window contains no partial week, since signups are not uniform across weekdays.
4. Add the metric's own lag rather than reporting the enrolment end date. Activation is unobservable until seven days after the last enrolled signup, and the definition publishes on an eight-day lag, so the readout lands 29 days after launch.
5. Close by testing whether the ask is realistic. If the team's honest prior is a 0.5pp lift, the requirement is about 110,000 per arm and roughly 24 weeks of enrolment, and saying so before launch is the deliverable.

Worked solution (20 min)

1. Compute p(1-p) = 0.22 x 0.78 = 0.1716 and 16 x 0.1716 = 2.746.
2. MDE at two weeks: n = 4,500 per arm per week x 2 = 9,000; d = sqrt(2.746 / 9000) = 0.0175.
3. Sample for d = 0.015: n = 2.746 / 0.000225 = 12,202 per arm, which is 2.71 weeks of enrolment, so enrol three whole weeks.
4. Readout date = 21 days of enrolment + 8 days of follow-up and publication lag = 29 days from launch.
5. State the boundary of what this quarter can answer: below roughly 0.7pp the requirement passes 56,000 per arm and more than a full quarter of enrolment.

Expected result: MDE after two weeks is 1.75 percentage points absolute, 7.9% relative. Detecting 1.5pp needs 12,202 users per arm, so three weeks of enrolment and a readout 29 days after launch. Detecting 0.5pp needs about 110,000 per arm, roughly 24 weeks, which is the answer the team needs to hear before the code is written.

Verify it yourself

• MDE must fall as 1 / sqrt(n). Doubling enrolment from two weeks to four should move 1.75pp to 1.24pp, not to 0.88pp.
• Substituting d = 0.0175 back into 16 p(1-p) / d^2 must return about 9,000 per arm.
• An exact two-proportion calculator at alpha 0.05 two-sided, power 0.80, p1 = 0.22, p2 = 0.235 returns about 12,260 per arm, within half a percent of the shortcut.

Follow-up

• The team also wants trial-to-paid read from the same test. What does a second primary metric do to your alpha, and what would you actually do about it?
• Signups are heavier on weekdays. What goes wrong if enrolment runs 17 days instead of 14?
• The activation definition uses a two-distinct-days threshold. If that were loosened to one day, what happens to the baseline rate and to the sample you need?

Attribute a flow-completion drop to one client build

Medium · release-regression · instrumentation · staged-rollout

Core-flow completion rate on surface = 'ios', defined as distinct flow_instance_id reaching 'flow_completed' within 30 minutes of 'flow_started' with no 'error_shown' on the same id in between, fell from 78% to 71% over four days. A new iOS build began a staged rollout on day one. Using fct_event columns flow_id, flow_instance_id, app_version, event_name, user_id and occurred_at_utc, produce the completion rate per build per day, size the loss in absolute completions, and say whether the fix is a rollback. Treat the possibility that the instrumentation changed rather than the flow.

Approach

1. Cut the rate by app_version and day, each build against its own denominator. A pooled series during a staged rollout is a weighted average whose weights move daily, so it declines in proportion to the rollout share even when the old build is perfectly flat; confirming that proportionality is itself the evidence the regression is build-specific rather than environmental.
2. Test the denominator before believing the rate. Compute flow_started events per distinct user_id on each build: if the new build re-mints flow_instance_id on retry, or fails to carry the id from start to completion, the denominator inflates and the numerator falls with nothing changing for the user.
3. Normalise to a user-level outcome that is immune to the id question, namely completions per distinct user_id per day on each build. This is the number that says whether anyone actually failed to finish.
4. Check the error path: pull error_shown counts and their properties for flow_instance_id values on the new build. A genuine regression produces errors; an instrumentation break produces missing completions with no corresponding error volume, and the two prescribe different fixes.
5. Size the real loss as the per-user completion gap multiplied by daily users on the new build, and give the rollback recommendation conditional on the error evidence rather than on the rate.

Follow-up

• The rollout is at 45% and product wants to go to 100% tomorrow. What do you say, and what would you need by when?
• If the flow_instance_id is genuinely being re-minted, what is the correct historical treatment of the four affected days?
• How would you have caught this on day one instead of day four?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

• Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
• Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
• Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

• Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
• On the third attempt, write the grain of every CTE as a comment before writing its body.
• Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

• Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
• Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
• Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

• Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
• Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
• Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

• Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
• Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
• Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

• Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
• Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
• Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

• Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
• Re-read only your own notes from this week, and open no new material.
• Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Where your data came from

Data people depend on systems owned by other teams, and much of the job is negotiating for instrumentation, access, or a fix to a broken pipeline. Prepare an example of getting something changed upstream that you did not control. Describe what you asked for, what you traded, and how you worked while you waited.

How do you explain the concept of statistical significance to a non-technical stakeholder?

Medium · behavioural and stakeholder questions

Approach

1. Name the disagreement or constraint, and how you resolved it with evidence.
2. State the situation in two sentences and spend the rest on your reasoning.
3. Close with what you would do differently, concretely.

Follow-up

• What did you decide not to do, and why?
• How did you know the outcome was caused by your change?

Handle a request for numbers supporting a decision already made

Hard · integrity · framing · stakeholder

A senior leader has already decided to sunset a plan tier and asks you for the analysis showing it is the right call. Accounts on that tier carry 6% of MRR at constant FX and have the highest licensed-seat utilisation in the book. The leader's support matters to your next review cycle, and the decision is being presented in four days. Deliver what you produce, what you decline to produce, and the exact sentence you will say in the meeting where the number appears on a slide.

Approach

1. Recognise what is being probed: whether you can find the legitimate request inside an illegitimate framing instead of either complying or refusing on principle. The generic answer promises to push back; the strong one produces something genuinely useful and states its limits in the room, without ambushing anybody.
2. Separate the decision from the justification. Sunsetting the tier may be correct for reasons the data does not hold, such as support cost, roadmap surface area or sales motion. What you decline is a one-sided document. What you produce is the case read both ways, which also happens to be more useful to the leader.
3. Build the symmetric analysis: MRR at risk at constant FX, the share of affected accounts with a plausible migration path given seats_licensed and billing_term, the recovery rate assumed for that migration and where it came from, and the downside case in which high-utilisation accounts treat the sunset as a reason to re-evaluate the vendor entirely.
4. Surface the inconvenient fact privately and early. The highest seat utilisation in the book is a retention signal, and the leader should hold it before the room does, so they can incorporate it rather than be caught by it.
5. Agree the meeting sentence in advance with the leader, so that nobody is surprised. Something to the effect that the tier is 6% of MRR and its accounts are the most heavily used in the book, and that the case for sunsetting rests on cost and focus rather than on revenue. That is true, it supports the decision on its real grounds, and it stops the deck claiming the numbers endorse it.
6. Decide your own line before you need it: what you will not put your name to, and that the route if asked anyway is your own manager rather than a confrontation in the meeting.

Follow-up

• The deck circulates with your analysis included and the downside case removed. What do you do, and by when?
• What changes if the honest analysis says the sunset is clearly the wrong call?
• How do you write the same memo when the leader is your skip-level and the meeting is tomorrow?

Defend a flat experiment readout against a post-hoc segment

Medium · experiments · segmentation · pushback

A feature you evaluated is flat on seven-day activation: +0.05pp with a 95% interval of [-0.47pp, +0.57pp], from 61,000 exposed users per arm in fct_experiment_exposure joined to dim_user and fct_event. Baseline activation is 32%. The launch team asks you to drop every surface except mobile_web, where the point estimate is +1.1pp, and re-run. You have ten minutes in their planning meeting. Deliver a spoken position: what you will and will not do, and the decision you recommend.

Approach

1. Recognise what is being probed: whether you hold a statistical position under social pressure without becoming either rigid or apologetic. A generic answer says the segment is not significant; a strong one separates the request into a question that is answerable (is the mobile_web number real?) and one that is not (can we ship on it?), and answers both.
2. Price the multiplicity out loud. The slice was chosen after seeing the results, so its estimate is selected on favourable noise and is biased away from zero. With k independent looks at a nominal 5% level, the chance of at least one false positive is 1 - 0.95^k: 26% at six segments, 64% at twenty. Quote the k you actually inspected, not the k you reported.
3. Use the arithmetic already in front of you. On the point estimates, a +1.1pp mobile_web effect combined with a pooled +0.05pp implies the remaining surfaces average negative in proportion to mobile_web's share of exposures. State that as a testable implication of their story rather than as a rebuttal of it.
4. Ask the one question that settles the category: was mobile_web named in the analysis plan before launch? If it was, it is a planned comparison and gets a corrected reading. If it was not, it is a hypothesis, and the honest move is to size the test that would confirm it.
5. Convert the refusal into a cost. Size a mobile_web-only confirmatory test at the claimed effect, state the weeks of mobile_web traffic it needs, and close with the recommendation: do not ship this as a lift, and note that the interval already rules out anything at or above +0.6pp, which is itself a useful input to the roadmap.

Follow-up

• The confirmatory test you sized needs nine weeks of mobile_web traffic and the team has three. What do you recommend instead?
• Suppose mobile_web was pre-registered. How does your reading change, and what correction do you apply?
• Your interval excludes +0.6pp. Is that the same as saying the feature does nothing?