Interview guides · Bosch Building Technologies

Bosch Building Technologies Data Scientist Interview Questions & Guide 2026

Take-home style practice for data scientist candidates: a messy dataset, a fixed time budget, and an analysis you write up and defend.

The 60-second brief

A Data Scientist at Bosch Building Technologies plays a crucial role in transforming complex data sets into actionable insights that drive product innovation and operational efficiency. This role is pivotal in enhancing the decision-making processes across various teams by leveraging advanced analytical techniques and machine learning algorithms. With a focus on smart building technologies, you will contribute to projects that impact everything from energy management systems to security solutions, ultimately improving user experience and business outcomes.

If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.

Focus areas

Read NRR on a fixed account cohort
Measure churn only on renewal-eligible accounts
Power experiments for heavy-tailed account revenue

Overview

The significance of the Data Scientist role is underscored by the scale and complexity of the data handled at Bosch. You will work on diverse datasets generated from cutting-edge technologies and IoT devices, making your contributions vital to the development of intelligent systems that cater to customer needs. In this dynamic environment, you will collaborate with cross-functional teams, applying your expertise to solve real-world problems, thereby influencing the strategic direction of products and services at Bosch.

The Bosch Building Technologies interview process

Bosch Building Technologies candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Technical Assessments

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Behavioral Interviews

Behavioural answers from data candidates get audited in a way that answers from other roles do not. When you say a model lifted retention, the next question is the denominator, the window, and how you knew the lift was not seasonal. So attach the measurement to each claim while you tell it: what the metric was before, over what period, and against what comparison. Numbers with no baseline read as rounded-up memory, and one unsupported figure tends to make the rest of the story sound rehearsed.

What to demonstrate

Whether each impact number arrives with a baseline, a window and a comparison, or as a bare percentage
Whether you can name the method that attributed the effect to your work (an experiment, a staged rollout, a seasonal control) or concede the link was correlational
Whether the magnitudes stay internally consistent when the interviewer multiplies them against the scale you described earlier

How to prepare

For each story, write the impact line as metric, value before, value after, window, and how attribution was established. Any line missing two of those five is a follow-up you will answer badly.
Re-derive one headline number from the source table rather than the deck that reported it. Resume numbers drift upward across retellings.
Decide in advance which figures you cannot share, and prepare the ratio or relative change you can give instead, so a confidentiality limit does not read as evasion.

Where candidates lose points

Randomising an experiment at the user level when users share an account

Two problems fire at once. Colleagues in one workspace see each other's work and talk to each other, so a treated user changes the behaviour of a control user in the same account, which violates the no-interference assumption and biases the estimate toward zero. Separately, outcomes within an account are strongly correlated, so the effective sample size is roughly n / (1 + (m - 1) * rho) for m users per account and intra-class correlation rho, not n. With rho around 0.3 and twenty users per account that is a design effect near 6.7, meaning a user-level confidence interval is about two and a half times narrower than it should be and results cross significance thresholds on noise alone. Randomise the account and cluster the standard errors.

Computing monthly churn against the entire customer base when contracts are annual

An annual contract has no opportunity to churn except at its renewal date, so an account that is eleven months from renewal is in the denominator while being incapable of appearing in the numerator. The resulting rate is smaller than the real one by roughly the ratio of the base to the renewal-eligible base, and it oscillates with the seasonality of when deals were originally signed rather than with anything about the customers. The corresponding trap on the other side is counting a churn on the date the record was updated rather than on term_end_date, which shifts losses into whichever month the operations team did its paperwork.

Reaching for a model before the target metric exists

Before naming an algorithm, write down the label, the prediction time, and the action that changes when the score crosses a threshold. If you cannot say what decision the output drives, any modelling choice is guesswork dressed up as method.

Ignoring interference between units in a marketplace experiment

Ask whether one unit's treatment can change another unit's outcome through shared inventory, a matching pool, a social graph or a common budget. Where it can, randomise at a level that contains the spillover, such as region or time slice, and say explicitly what that costs you in statistical power.

Questions & practice

Describe a machine learning project you’ve worked on. What was your approach?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

If you were tasked with improving an existing predictive model, what steps would you take?

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Set a baseline first, so any model has something honest to beat.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Sessionise an API event stream with a 30-minute inactivity gap

Medium · sessionisation · event-streams · pandas

fct_api_request arrives as a DataFrame with account_id, user_id, request_at (tz-aware UTC), traffic_class and http_status, roughly 5 million rows. Assign a session_id to every human-attributable request: drop rows where user_id is null or traffic_class is in ('ci','synthetic_monitor','load_test'), then open a new session whenever the gap since that user's previous remaining request exceeds 30 minutes. Return the filtered frame plus session_id, and a per-session summary with user_id, account_id, session start, session end and request count. Do not loop over rows.

Approach

Settle the filter-then-gap ordering before writing code. Removing CI and synthetic rows changes the gaps, so sessionising the raw stream and filtering afterwards is a different answer; the definition given filters first, and the two diverge most for accounts whose CI runs every ten minutes.
Sort once by (user_id, request_at) with a stable kind, then gap = df.groupby('user_id', sort=False).request_at.diff(). The first row of each user yields NaT, which is exactly the boundary condition you want rather than a special case to patch.
new_session = gap.isna() | (gap > Timedelta(minutes=30)); session_id = new_session.cumsum(). The cumsum runs over the whole sorted frame and therefore produces globally unique ids in one pass; a per-user cumcount collides across users and forces a composite key on every downstream join.
Build the summary with a single groupby('session_id').agg(...). user_id and account_id can be carried with 'first' only because the sort key groups them — state that dependency, since it silently breaks if someone later re-sorts the frame.
Decide explicitly what a session means when one user_id holds memberships in several accounts: either add account_id to the sort and group keys, or document that sessions may cross accounts. Leaving it undecided produces sessions whose account_id is whichever row sorted first.

Worked solution (30 min)

human = df[df.user_id.notna() & ~df.traffic_class.isin(['ci','synthetic_monitor','load_test'])].sort_values(['user_id','request_at'], kind='mergesort').reset_index(drop=True)
gap = human.groupby('user_id', sort=False).request_at.diff(); human['session_id'] = (gap.isna() | (gap > pd.Timedelta(minutes=30))).cumsum()
summary = human.groupby('session_id').agg(user_id=('user_id','first'), account_id=('account_id','first'), start=('request_at','min'), end=('request_at','max'), n_requests=('request_at','size')).reset_index()
assert summary.n_requests.sum() == len(human) and human.groupby('session_id').user_id.nunique().max() == 1

Expected result: human gains a dense session_id increasing in (user_id, request_at); summary has exactly one row per session, n_requests sums to len(human), and every within-session consecutive gap is at most 30 minutes while every same-user between-session gap exceeds it.

Verify it yourself

summary.n_requests.sum() == len(human); no request is lost or double-counted.
human.groupby('session_id').user_id.nunique().max() == 1.
human.groupby('session_id').request_at.diff().max() <= pd.Timedelta(minutes=30).
Shifting every timestamp by a constant offset leaves the session count unchanged.

Follow-up

Where does 30 minutes come from, and how would you pick it from this data instead of from convention?
An engineer reused their personal key for a nightly batch job, so machine traffic carries a human user_id. How would you detect that, and should those requests form sessions?
How much does the session count change if you sessionise before dropping CI traffic rather than after?

Explain the time complexity of your solution.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Write a Python function to calculate the Fibonacci sequence.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Consecutive qualifying weeks before renewal, as a ranked worklist

Hard · gaps and islands · window functions · date boundaries

fct_api_request carries account_id, workspace_id, environment, request_at (timestamptz), http_status and traffic_class. fct_subscription_period carries account_id, term_end_date, auto_renew and is_current. A qualifying week for an account is an ISO week with at least 50 successful production requests in traffic_class ('interactive','batch'). Over the last 52 whole ISO weeks, and for accounts whose current term ends within 90 days, return the current qualifying-week streak length, the week it began, and the longest earlier streak. An account whose streak has broken must appear with a current length of zero.

Approach

Bucket weeks as date_trunc('week', request_at AT TIME ZONE 'UTC'). request_at is a timestamptz, so an unpinned date_trunc silently uses the session time zone, weeks start at a local midnight, and Monday-morning traffic lands in the previous week for part of the fleet. Pinning UTC also makes the seven-day arithmetic below exact across daylight-saving transitions.
Apply the exclusions before counting: environment = 'production', http_status < 400, traffic_class IN ('interactive','batch'). Then apply the volume floor and drop the current partial week, which can never meet a floor calibrated on whole weeks.
Build islands with the row-number anchor: ROW_NUMBER() OVER (PARTITION BY account_id ORDER BY week_start) as rn, then week_start - rn * interval '7 days' is constant inside a run of consecutive weeks. Group by that anchor to get each streak's start, end and length.
The current streak is the island whose end equals the last whole week; if none does, the account's current streak is zero and that is the interesting case. The longest earlier streak is the maximum length among the remaining islands.
Join the renewal filter from the current subscription row and LEFT JOIN the streak summary so an account with no qualifying week at all still appears, rather than vanishing from the risk list precisely because it went quiet.
Finish with an operating point. The list is worked by a team with finite capacity, so order it and cut it at that capacity, and say what happens to the accounts below the line.

Worked solution (40 min)

Build weekly_qualified: filter the fact on environment, status and traffic_class, group by account_id and date_trunc('week', request_at AT TIME ZONE 'UTC'), keep groups with count(*) >= 50, and exclude the in-progress week and anything older than 52 whole weeks.
Add rn = ROW_NUMBER() OVER (PARTITION BY account_id ORDER BY week_start) and anchor = week_start - rn * interval '7 days'.
Group by (account_id, anchor) to get streak_start = min(week_start), streak_end = max(week_start), streak_len = count(*).
Per account, take current_len as the streak_len where streak_end = the last whole week else 0, current_start from the same island, and longest_prior as max(streak_len) over the other islands.
Join to fct_subscription_period on is_current with term_end_date <= current_date + 90, LEFT JOIN the streak summary, and order by current_len ascending then term_end_date ascending.

Expected result: One row per renewal-eligible account with current_streak_weeks, current_streak_start_week and longest_prior_streak_weeks, where an account with no qualifying week in the last whole week returns current_streak_weeks = 0 and still appears, ordered so the quietest imminent renewals sit at the top.

Verify it yourself

Per account, the sum of all island lengths equals the count of qualifying weeks for that account.
current_streak_weeks > 0 if and only if the last whole week is a qualifying week for that account.
No island contains a gap: re-deriving consecutive differences inside an island returns exactly 7 days everywhere.
An account qualifying in all 52 weeks returns a single island of length 52 and a null-free current streak start.

Follow-up

The floor of 50 requests was picked for this exercise. How would you calibrate it from data, and what would force a recalibration?
A regional holiday week drops several accounts below the floor at once. How do you keep that out of the risk list?
How would you evaluate whether contacting these accounts actually changed renewal, given that coverage is assigned deliberately?

How do you prioritize tasks when working on multiple projects?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Given a dataset, how would you identify potential outliers?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you handle missing data in a dataset?

Medium

Approach

Say what you would check first and why it is the highest-information step.
Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Estimate a staggered overage-price change without a control arm

Hard · difference-in-differences · synthetic-control · staggered-adoption

On 2026-03-01 the overage rate for sku_code = 'ingest_gb' rose 18% for accounts whose dim_account.billing_country falls in six countries. Each account moves to the new rate at its own renewal, so treatment switches on across twelve months. Randomisation was never available. Using fct_usage_daily and fct_subscription_period, estimate the effect on net_amount_cents per account and on gross logo retention. Specify the estimator, the identifying assumption and how you test it, how you handle accounts that churn out of the panel, and how you do inference with six treated countries.

Approach

Fix the estimand before the estimator. Effect on revenue per account and effect on retention are different questions, and the second determines whether the first is worth having: an 18% rate rise that raises revenue while costing renewals is a loss. Define both on a cohort frozen before 2026-03-01, carrying churned accounts at zero revenue rather than dropping them, so attrition cannot masquerade as a revenue effect.
Do not run a single two-way fixed-effects regression. With treatment switching on at different renewal dates and effects that vary with time since treatment, the two-way fixed-effects coefficient is a variance-weighted average that includes comparisons of later-treated accounts against already-treated ones, and those comparisons can carry negative weights. Use a cohort-by-cohort estimator such as Callaway-Sant'Anna, or a stacked event study using only not-yet-treated controls, and report the event-study path rather than one number.
Treat parallel trends as a maintained assumption to be probed, not a result to be shown. Plot pre-treatment lead coefficients with intervals and state that flat leads are consistent with the assumption rather than proof of it. Handle anticipation explicitly: accounts notified of the increase before their renewal can pull ingest volume forward, so exclude a pre-renewal anticipation window and check whether the leads move when you do.
When the six treated countries are dominated by a few large accounts, the group mean is not a stable object and difference-in-differences on it will be noisy in a way its standard error will not reflect. Build a synthetic control at the country level from a donor pool of untreated countries, weighted to match the pre-period trajectory of net revenue per account, and report the placebo distribution across donor countries instead of a conventional standard error.
Do inference at the level treatment was assigned, which is the country. Six treated clusters is far too few for cluster-robust standard errors, which are badly downward-biased below roughly forty clusters. Use a wild cluster bootstrap, and report a randomisation-inference p-value from placebo assignments of treated status alongside it.
Apply the domain's data hygiene or the whole estimate is contaminated: exclude is_internal accounts, exclude the trailing metering settling window so the final months are not artificially low, and join as-of to the fct_subscription_period version live on each date rather than to is_current, which would price last year's usage at this year's contract.

Worked solution (45 min)

Build a balanced monthly account panel from fct_usage_daily joined as-of to fct_subscription_period, excluding is_internal accounts and carrying churned accounts forward at zero revenue.
Define treatment cohorts by renewal month, estimate group-time average treatment effects, and aggregate to an event-study path with leads from -6 to -1 and lags from 0 to +11.
Fit a country-level synthetic control on pre-period net revenue per account and generate the placebo distribution by refitting on each donor country in turn.
Compute a wild cluster bootstrap p-value clustered on country and report the randomisation-inference p-value from the placebo distribution beside it.
Repeat the full path for gross logo retention, restricted each month to the renewal-eligible base plus the 45-day grace window.

Expected result: An event-study path whose lead coefficients are statistically indistinguishable from zero and whose lags are reported by months since renewal, with country-level inference from a wild cluster bootstrap and a placebo-based p-value from the synthetic control. A single pooled two-way fixed-effects number is not reported, and the retention path is computed only on the renewal-eligible base.

Verify it yourself

Treated and synthetic pre-period trajectories must track within the pre-period fit error. A poor pre-period fit cannot be rescued by a longer post-period and invalidates the comparison.
Gross logo retention must be computed on accounts whose term_end_date fell in the month, not on the whole panel. On annual contracts only about a twelfth of accounts can churn in any month, so the whole-base rate understates churn by roughly an order of magnitude.
At six treated clusters the wild cluster bootstrap p-value should be noticeably larger than the naive cluster-robust one. If they agree closely, the bootstrap was run on the wrong clustering variable.

Follow-up

Your event study shows a significant lead coefficient two months before renewal. Is that anticipation, a violation of parallel trends, or a coding error, and what distinguishes them?
Revenue per account rose 6% and gross logo retention fell 1.8 points. How do you combine those into a single recommendation, and over what horizon?
One treated country contains an account holding 20% of that country's revenue. What does that do to the synthetic control fit, and what would you do about it?

Blended gross margin fell while every segment improved

Medium · mix shift · simpsons paradox · as-of join

Blended gross margin across paying accounts fell from 71 percent to 66 percent over two quarters, yet margin improved inside every plan_tier and every deployment_model. You have fct_usage_daily (account_id, sku_code, usage_date, net_amount_cents, cogs_cents), fct_subscription_period (account_id, plan_tier, term_start_date, term_end_date) and dim_account (account_id, deployment_model, is_internal, effective_from, effective_to). Produce an exact decomposition of the five-point move into mix, rate and interaction terms, and name the segment shift that carries it.

Approach

Write blended margin as a revenue-weighted average of segment margins, m = sum over i of w_i * m_i, where w_i is the segment's share of net_amount_cents. Margin is a ratio of sums, so only revenue weights reproduce the blended figure.
Apply the exact three-term decomposition: delta_m = sum(delta_w_i * m_i0) + sum(w_i0 * delta_m_i) + sum(delta_w_i * delta_m_i), read as mix, rate and interaction. It is exact by construction, so the three terms must sum to the observed change with no residual.
Build segments from as-of joins: take the fct_subscription_period version whose term brackets each usage_date and the dim_account version whose effective_from and effective_to bracket it, rather than joining on is_current and backdating today's attributes over last year's usage.
Run the decomposition on more than one segmentation, at minimum plan_tier, deployment_model and sku_code, since the mix that actually moved may not be the one anybody already suspected.
Once the driving segment is identified, determine whether the shift came from acquisition landing new accounts in a lower-margin segment or from existing accounts migrating, because those have different owners and different remedies.

Follow-up

The interaction term is sometimes large. What does it mean operationally, and when would you prefer a log-mean decomposition that distributes it across the other two terms?
Revenue-weighted average margin is one construction. What changes if leadership wants margin weighted by account count, and which question does each version answer?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
On the third attempt, write the grain of every CTE as a comment before writing its body.
Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
Re-read only your own notes from this week, and open no new material.
Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

Discuss a time when you had to optimize a model for better performance.

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Describe a challenging project you worked on. How did you ensure success?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Pick a story where you drove the decision, not one where you observed it.
Close with what you would do differently, concretely.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Report an underpowered consumption test to a non-technical executive

Medium · communication · uncertainty · experimentation

An account-randomised packaging change ran six weeks across 900 paying accounts. The effect on billable units per account per month is plus 4.1 percent, with a 95 percent interval from minus 3.2 to plus 11.8 after clustering standard errors at the account and applying the pre-registered winsorisation at the 99th percentile. An executive with no statistical background wants one number this week to decide a full rollout. Produce a three-sentence spoken answer, one chart, and an explicit recommendation of ship, stop or keep running, with the cost of each option stated.

Approach

The interviewer is probing whether you can be decision-useful without either hiding the uncertainty or hiding behind it. Start from the decision rather than the statistics: establish what the executive would do differently at plus 4 percent versus zero, because if the action is identical the interval does not matter.
Translate the interval into consequences in units the executive already reasons about. Multiply both endpoints by the cohort's baseline consumption and contracted rates to give an annualised revenue range, so the answer is a range of dollars rather than a range of percentages.
Price the option to wait. Using the observed variance, state roughly how many additional account-weeks halve the interval width, so keep running becomes a quantified choice instead of a stall.
Offer a cheaper path to the same decision: a lower-variance proximate outcome such as successful billable units on the new SKU, or CUPED using each account's pre-period consumption, quoting the expected variance reduction as one minus the squared pre-post correlation.
Give a recommendation and name the single observation that would reverse it. A strong answer commits; a generic one recites the interval and leaves the decision on the table.

Follow-up

The executive says it clearly works and is just not provable, so ship it. What is your answer?
How much of the interval width comes from clustering and how much from the revenue tail, and what would you do about each?
If you had to ship this week with no more data, which guardrail would you watch for the first fortnight and at what threshold would you roll back?