cswork / libraryOpen practice workspace

Interview guides · Hach

Hach Data Scientist Interview Questions & Guide 2026

Modeling drills for a Hach data scientist interview: baselines, regression interpretation, class imbalance, model defense.

The 60-second brief

As a Data Scientist at Hach, you will play a pivotal role in transforming data into actionable insights that contribute to our commitment to providing innovative water quality solutions. This position is crucial for enhancing our products and services, ensuring that our users can make informed decisions based on accurate data analysis. You'll engage with complex datasets, leveraging statistical methodologies and machine learning techniques to drive improvements in our offerings and operational efficiencies.

Most of the loop measures decision-making under uncertainty rather than recall. You are scored on whether you state your assumptions, commit to an estimate you can defend, and say explicitly what evidence would change it.

Focus areas

  • Define numerator, denominator and window precisely
  • Separate novelty effects from durable behaviour change
  • Pick a randomisation unit that respects interference

Overview

The impact of your work extends beyond numbers; you will influence product development, enhance user experience, and support our mission to provide superior water quality monitoring solutions. By collaborating with cross-functional teams, you will address real-world challenges, contributing to products that help ensure safe and sustainable water resources. Expect to be part of a dynamic environment where your analytical skills will inform strategic directions and foster continuous innovation.

The Hach interview process

Hach candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Application Review

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

  • Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
  • Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
  • Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

  • Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
  • Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
  • Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Assessments

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

  • Whether the query you write matches the plan you just described
  • What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
  • Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

  • Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
  • Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
  • Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Behavioral Interviews

Most of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.

What to demonstrate

  • Whether you can state the other side's argument accurately before you explain why you disagreed
  • What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
  • Whether you distinguish being overruled from being wrong, and can give an example of each

How to prepare

  • Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
  • For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
  • Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.

Cross-Functional Discussions

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

  • Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
  • Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
  • Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

  • For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
  • Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
  • For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Where candidates lose points

Comparing cohort retention curves of different maturities, or building the curve from users who are still present

A cohort four weeks old has no week-8 value, so an average taken across cohorts silently drops young cohorts from the later columns and keeps them in the earlier ones. The curve then bends upward at the tail, and the reading that 'retention is improving over time' is an artefact of which cohorts survived to be measured. The same error appears in the denominator when retention is computed over users active in the current period rather than over the full original cohort, which conditions on survival and guarantees a flattering number. The fix is a triangle: fix the cohort at signup, bound every window on both sides, and only compare cells where every cohort has had the full elapsed time, publishing the rest as blank rather than as a partial average.

Counting on an identity key that changes underneath the metric

visitor_id is per browser and per device, and it resets on cookie clearance, private browsing and platform privacy changes, so the distinct-visitor count drifts upward for reasons unrelated to reach. Any rate with visitors in the denominator therefore decays over time even when behaviour is constant, and any rate with visitors in the numerator inflates. The stitching at signup makes it worse in both directions: a user who signed up on mobile and returns on desktop is two visitors and one user, while a shared device is one visitor and several users. Decide which key each metric is counted on, write it into the definition, and when comparing a period before and after a platform privacy change, expect a level shift in every visitor-keyed metric and do not attribute it to the product.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Questions & practice

What are p-values, and how do they influence hypothesis testing?

Medium · statistics and probability

Approach

  1. Translate the result into the decision it informs, in one plain sentence.
  2. Write down the assumption the method needs before you use the method.
  3. Say what the estimate is of, and over what population it generalises.

Follow-up

  • How would you explain this result to someone who does not know statistics?
  • What sample size would you need to detect an effect half this size?

Write a function to calculate the mean and standard deviation of a list of numbers.

Medium · statistics and probability

Approach

  1. Translate the result into the decision it informs, in one plain sentence.
  2. Quantify uncertainty explicitly rather than reporting a point estimate alone.
  3. Write down the assumption the method needs before you use the method.

Follow-up

  • What sample size would you need to detect an effect half this size?
  • How would you explain this result to someone who does not know statistics?

Discuss the Poisson distribution and a scenario where it is applicable.

Medium · statistics and probability

Approach

  1. Write down the assumption the method needs before you use the method.
  2. Translate the result into the decision it informs, in one plain sentence.
  3. Sanity-check the answer against a simple bound or a simulated case.

Follow-up

  • What sample size would you need to detect an effect half this size?
  • Which assumption here is most likely to be violated in practice?

Can you walk us through your process for developing a predictive model?

Medium · machine learning and modelling

Approach

  1. Set a baseline first, so any model has something honest to beat.
  2. Pick an evaluation metric that matches the cost of each error type, not a default.
  3. Say how the offline result would be validated online before it is trusted.

Follow-up

  • Where could label leakage enter this setup?
  • What would you monitor after launch to know the model is still valid?

Can you describe the K-means clustering algorithm and its applications?

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • How would you choose the decision threshold, and who owns that choice?

Implement seven-day activation from its written definition

Medium · metric definition · cohorts · pandas · joins

Implement the seven-day activation rate. Inputs: dim_user with user_id, account_created_at_utc and is_internal; fct_event with user_id, occurred_at_utc and is_core_action. A user activates when core-action events carrying a non-NULL user_id fall on at least two distinct UTC dates inside [account_created_at_utc, account_created_at_utc + 7 days). The denominator is every non-internal user whose account_created_at_utc lands in the cohort week, including users with no events at all. Return one row per cohort week with numerator, denominator and rate, publishing only weeks whose last signup is at least eight days old.

Approach

  1. Build the denominator first, from dim_user alone, filtered on is_internal = False. Deriving it from the join is the standard way to lose every user who never fired an event, which is exactly the population the metric is about.
  2. Join events to users on user_id with a left join from the user side, then apply the window as a half-open interval: occurred_at >= created AND occurred_at < created + 7 days. The right bound is exclusive, so an event at exactly created + 7 days does not count.
  3. Count distinct UTC dates per user, not distinct events. Floor occurred_at_utc to date before the nunique, and do it in UTC rather than local time so the threshold does not move with the user's country.
  4. Apply the >= 2 threshold, aggregate to cohort week, and compute the rate by re-summing numerator and denominator per week rather than averaging any per-user or per-day rate. Fix the week anchor explicitly: cohort_week is the Monday of the signup week in UTC, which is what Postgres DATE_TRUNC('week') returns and what any SQL version of this metric will produce. In pandas, subtract dt.weekday days from the floored timestamp. If you reach for periods instead, the anchor that matches is to_period('W') (equivalently 'W-SUN'), whose weeks end Sunday and therefore start Monday; to_period('W-MON') labels weeks that end on Monday, so it runs Tuesday through Monday and its start_time is a Tuesday. Mixing the two shifts every cohort label by one day and silently moves Mondays into the previous week.
  5. Suppress immature weeks: drop any cohort week whose maximum account_created_at_utc is within 8 days of the data cut, and return them as absent rather than as a partial number.

Worked solution (30 min)

  1. users = dim_user[~dim_user.is_internal].copy(); created = users['account_created_at_utc']; users['cohort_week'] = created.dt.floor('D') - pd.to_timedelta(created.dt.weekday, unit='D'), which is the Monday-start week. The period spelling that agrees with it is created.dt.to_period('W').dt.start_time; 'W-MON' does not agree and is off by a day.
  2. ev = events[events.is_core_action & events.user_id.notna()]; merge onto users on user_id with how='inner' for the numerator side only.
  3. Filter to the half-open window, add ev_date = occurred_at_utc.dt.date, group by user_id and count distinct dates, keep users with >= 2.
  4. numer = users.merge(activated_user_ids, how='left', indicator=True) then group by cohort_week and sum the indicator; denom = users.groupby('cohort_week').size().
  5. rate = numer / denom; drop weeks where users.groupby('cohort_week')['account_created_at_utc'].max() > data_max - 8 days.

Expected result: One row per mature cohort week with numerator, denominator and rate, where denominator equals the count of non-internal signups in that week regardless of activity, numerator is less than or equal to denominator, and the most recent 1 to 2 weeks are absent rather than reported low.

Verify it yourself

  • For a synthetic user with two core actions 90 minutes apart on the same UTC date, activation is False.
  • For a synthetic user with one core action on day 0 and one at exactly created + 7 days, activation is False; move the second to created + 6 days 23 hours and it becomes True.
  • Pin the week anchor with two users created at Sunday 23:59 UTC and the following Monday 00:01 UTC: they must land in different cohort weeks, and the Sunday user's cohort_week must be the Monday six days before them. to_period('W-MON') puts both in one period, which is the off-by-one this check exists to catch.
  • The denominator summed across all published weeks equals the count of non-internal users created in that date range.
  • Re-summing numerator and denominator to a monthly figure gives a different answer from averaging the weekly rates; the re-summed one is the metric.

Follow-up

  • The threshold is 2 distinct days. What changes in the reported history if someone moves it to 3, and how would you publish that change?
  • Invited seats and SSO-provisioned users have no pre-signup session. Should they be in this denominator at all, and what does including them do to the rate for sales-assisted accounts?
  • How would you produce the same metric at account grain, and which of the two would you put on the dashboard?

Find reactivation gaps in account paid-period history

Hard · gaps-and-islands · window-functions · subscriptions

fct_subscription_period holds account_id, subscription_id, period_start_utc, period_end_utc, period_status and change_reason. A mid-period plan or seat change closes one row and opens another, so a single continuous paid tenure is often many rows, and an account may hold two overlapping subscriptions. Collapse rows with period_status in ('active','past_due') into continuous tenures per account, treating gaps of three days or less as continuous. Return account_id, tenure_start, tenure_end, and for every tenure after the first, the gap in days that preceded it.

Approach

  1. Filter to paid rows only: period_status IN ('active','past_due'). Trialing periods are not tenure, and including them turns every trial that never converted into a one-period tenure followed by a fake churn.
  2. Order by period_start_utc and take a running maximum of all prior ends: MAX(period_end_utc) OVER (PARTITION BY account_id ORDER BY period_start_utc, period_end_utc, subscription_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING). Those three columns are the only stable ordering this schema exposes, so check first that they are unique within an account; if rows tie on all three, the island numbering is order-dependent between runs and you need a real row key before the result is reproducible.
  3. LAG on its own is wrong here because with overlapping or nested periods the immediately preceding row by start date is not the one that ends latest, so the running maximum is the part that cannot be shortcut.
  4. Flag a new island when prior_max_end IS NULL OR period_start_utc > prior_max_end + interval '3 days', then number islands with a running SUM of the flag over the same ordering and an explicit ROWS frame.
  5. Group to (account_id, island) taking MIN(period_start_utc) and MAX(period_end_utc), then LAG(tenure_end) OVER (PARTITION BY account_id ORDER BY tenure_start) to compute the preceding gap in days for every tenure after the first.
  6. Sanity-check with change_reason, which is the only lineage this schema carries: list its distinct values first, then confirm that rows recording a plan or seat change sit inside a tenure rather than opening one, and that every tenure after the first opens on a row whose reason records a restart rather than an ordinary renewal. Do not reconcile against a churn timestamp on dim_account, which this schema does not define; and where such a column does exist, a cancellation timestamp records when the request was made and routinely sits weeks before the period it ends.

Worked solution (35 min)

  1. Find an account with a known mid-period upgrade and dump its period rows to use as the trace case.
  2. Check that (period_start_utc, period_end_utc, subscription_id) is unique per account, since the whole ordering rests on it.
  3. Write the paid-rows CTE and the running MAX with the explicit frame.
  4. Add the island flag and the running SUM, then verify the trace account yields one island.
  5. Group to tenures and add the LAG-based gap in days.
  6. List the distinct change_reason values, then count accounts with more than one tenure and compare against the count of accounts carrying a restart-flavoured reason anywhere in their history.

Expected result: One row per (account_id, tenure). An account with a continuous history returns a single row with a NULL preceding gap. Every non-first tenure has gap_days strictly greater than 3 by construction.

Verify it yourself

  • No two tenures for the same account overlap or touch within the grace window.
  • MIN(tenure_start) per account equals MIN(period_start_utc) per account in the filtered input.
  • The account with a known mid-period upgrade produces one tenure, not two.
  • (period_start_utc, period_end_utc, subscription_id) has no duplicates within an account; if it does, say how many accounts are affected before reading the output.
  • The share of accounts with more than one tenure is small; if it is most accounts, the status filter or the grace window is wrong.

Follow-up

  • Why three days of grace? What do 0 and 30 days each do to the count of accounts classed as reactivated?
  • An account runs two concurrent subscriptions for different teams. One tenure or two, and what does the revenue reader expect?
  • How would you turn these tenures into a monthly gross logo churn series without double-counting an account that churned and returned in the same month?

Weekly visit-to-signup conversion split by acquisition channel

Easy · funnel · conversion · aggregation

From fct_session (session_id, visitor_id, started_at_utc, referrer_channel, is_bot_flagged, consent_state) and fct_event (visitor_id, occurred_at_utc, event_name), compute visit-to-signup conversion for one ISO week, split by channel. session_id is the unique key of fct_session. Denominator: distinct visitor_id with a session starting in the week, is_bot_flagged = FALSE and consent_state <> 'denied'. Numerator: those visitors with a 'signup_completed' event in the same week. Label each visitor with the referrer_channel of their first session in the window. Return channel, visitors, signups and rate, plus one all-channel total row.

Approach

  1. Build a visitor spine that is one row per visitor: filter sessions to the week, drop is_bot_flagged and consent_state = 'denied', then take the first session per visitor with ROW_NUMBER() OVER (PARTITION BY visitor_id ORDER BY started_at_utc, session_id) = 1 to carry the channel label. Collapsing to one row here is what makes the channel buckets mutually exclusive and the totals additive.
  2. session_id is the unique key, so that ordering is total and the label is reproducible. If the table carried no unique key you would have to write an explicit tie rule instead, because two sessions on different channels at the identical timestamp would otherwise label the visitor differently between runs.
  3. Attach the outcome as a semi-join (EXISTS on a signup_completed event for that visitor inside the same week) rather than a join to the event table, so a visitor who fires the event twice does not count twice and inflate the numerator past the denominator.
  4. Aggregate with COUNT() as visitors and COUNT() FILTER (WHERE signed_up) as signups, and compute the rate as signups::numeric / NULLIF(visitors, 0) so an empty channel returns NULL rather than a division error.
  5. Produce the total with GROUP BY GROUPING SETS ((channel), ()), which re-sums numerator and denominator for the total row. Averaging the channel rates gives a different and wrong number whenever channel volumes differ, which they always do.
  6. Verify the spine before trusting the output: COUNT(*) must equal COUNT(DISTINCT visitor_id), and the per-channel visitor counts must sum to the total row.

Follow-up

  • The denominator is distinct visitors. If a browser release shortens cookie lifetime, what happens to this rate, and how would you tell that apart from a genuine drop?
  • A visitor's first session is direct and their signup session is paid search. Your label says direct. When is that the wrong answer for the decision being made?
  • How do you roll four weeks into a month, and why is averaging the four weekly rates wrong?

How would you approach a dataset with missing values?

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

Discuss how you would communicate complex data insights to a non-technical audience.

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

Explain the concept of a t-test and when you would use it.

Medium

Approach

  1. Clarify what is being asked and what a complete answer would contain.
  2. Say what you would check first and why it is the highest-information step.
  3. State your assumptions explicitly before working the problem.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Size an activation test before anyone writes code

Easy · power · minimum detectable effect · proportions

A team wants to test a new onboarding checklist against the seven-day activation rate: users with is_core_action = TRUE events on at least two distinct UTC dates inside the first seven days, counted from dim_user.account_created_at_utc with is_internal = FALSE. Baseline is 22%. Weekly non-internal signups are 9,000, split evenly between two arms. State the minimum detectable effect after two weeks of enrolment, the enrolment needed to detect a 1.5 percentage point absolute lift, and the earliest date a readout would be honest.

Approach

  1. Use the two-proportion sizing shortcut n = 16 p(1-p) / d^2 per arm, where d is the absolute lift. The 16 is 2(1.96 + 0.84)^2 = 15.7 rounded, i.e. two-sided alpha 0.05, 80% power, equal arms.
  2. Invert it for the MDE at fixed n: d = sqrt(16 p(1-p) / n). Two weeks gives 9,000 per arm, so d = sqrt(2.746 / 9000) = 0.0175, which is 1.75 percentage points absolute or 7.9% relative on a 22% base.
  3. Run it forward for d = 0.015: n = 2.746 / 0.000225 = 12,202 per arm, 24,404 total, about 2.7 weeks of enrolment. Round up to three whole weeks so the enrolment window contains no partial week, since signups are not uniform across weekdays.
  4. Add the metric's own lag rather than reporting the enrolment end date. Activation is unobservable until seven days after the last enrolled signup, and the definition publishes on an eight-day lag, so the readout lands 29 days after launch.
  5. Close by testing whether the ask is realistic. If the team's honest prior is a 0.5pp lift, the requirement is about 110,000 per arm and roughly 24 weeks of enrolment, and saying so before launch is the deliverable.

Worked solution (20 min)

  1. Compute p(1-p) = 0.22 x 0.78 = 0.1716 and 16 x 0.1716 = 2.746.
  2. MDE at two weeks: n = 4,500 per arm per week x 2 = 9,000; d = sqrt(2.746 / 9000) = 0.0175.
  3. Sample for d = 0.015: n = 2.746 / 0.000225 = 12,202 per arm, which is 2.71 weeks of enrolment, so enrol three whole weeks.
  4. Readout date = 21 days of enrolment + 8 days of follow-up and publication lag = 29 days from launch.
  5. State the boundary of what this quarter can answer: below roughly 0.7pp the requirement passes 56,000 per arm and more than a full quarter of enrolment.

Expected result: MDE after two weeks is 1.75 percentage points absolute, 7.9% relative. Detecting 1.5pp needs 12,202 users per arm, so three weeks of enrolment and a readout 29 days after launch. Detecting 0.5pp needs about 110,000 per arm, roughly 24 weeks, which is the answer the team needs to hear before the code is written.

Verify it yourself

  • MDE must fall as 1 / sqrt(n). Doubling enrolment from two weeks to four should move 1.75pp to 1.24pp, not to 0.88pp.
  • Substituting d = 0.0175 back into 16 p(1-p) / d^2 must return about 9,000 per arm.
  • An exact two-proportion calculator at alpha 0.05 two-sided, power 0.80, p1 = 0.22, p2 = 0.235 returns about 12,260 per arm, within half a percent of the shortcut.

Follow-up

  • The team also wants trial-to-paid read from the same test. What does a second primary metric do to your alpha, and what would you actually do about it?
  • Signups are heavier on weekdays. What goes wrong if enrolment runs 17 days instead of 14?
  • The activation definition uses a two-distinct-days threshold. If that were loosened to one day, what happens to the baseline rate and to the sample you need?

Size every candidate cause of a trial-to-paid decline

Hard · cohort-maturity · experiment-readout · mix-shift · contribution-sizing

Trial-to-paid conversion on weekly trial-start cohorts from fct_subscription_period reads 3.1 points below the trailing eight-week mean for the three most recent cohorts. Three things happened in that window: a pricing experiment reached 50% of new trials, a payment processor migration added settlement delay, and paid_search spend tripled. Using fct_subscription_period, fct_experiment_exposure and dim_user, rank the causes by their contribution in points of the headline, state the remainder, and give the decision you would take on Monday.

Approach

  1. Kill the immature cohorts first, because everything downstream is computed on them. The metric is lagged by the trial length plus a 14-day conversion window plus a settlement allowance, and a processor migration lengthens exactly that last term; recompute each cohort at a fixed cohort age rather than as of today, and confirm the newest cohort's value is still climbing day over day.
  2. Hold the experiment analysis to the exposed population. Join fct_experiment_exposure on unit_id with is_in_analysis_population = TRUE rather than reading an assignment log, then check the variant split for a sample-ratio mismatch before believing any effect at all. Contribution to the headline is the variant effect multiplied by the exposed share, which is not the same number as the variant effect.
  3. Decompose the cohort mix by dim_user.first_touch_channel using the same weight-times-rate arithmetic as any other mix question, so the paid_search increase is sized as a weight change at a measured conversion rate rather than asserted from the spend figure.
  4. Convert all three to points of the headline, sum them, and print the residual against the historical week-to-week standard deviation of the metric. If the residual is inside that band, say so and stop looking; if it is outside, name what you would investigate next rather than leaving it implied.
  5. Land the decision. Only one of the three is actionable on Monday, so state whether the experiment has accrued enough exposed units to stop at the pre-declared horizon, and state separately what the settlement-lag correction does to the published series and its lag rule.

Follow-up

  • How do you choose the fixed cohort age, and what do you lose by choosing it too long?
  • If the pricing variant is genuinely 1.2 points worse, does that settle whether to stop it? What else is on the other side of that decision?
  • The trailing eight-week mean spans the processor migration. What is the right baseline instead?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

  • Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
  • Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
  • Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

  • Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
  • Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
  • Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

  • Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
  • Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
  • Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

  • Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
  • Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
  • Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

  • Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
  • Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
  • Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

  • Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
  • Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
  • Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

  • Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
  • Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
  • Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Can you provide an example of how you have influenced a team decision?

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. Pick a story where you drove the decision, not one where you observed it.
  3. Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

  • How did you know the outcome was caused by your change?
  • What would you do differently if you ran that project again?

Explain a wide interval to a non-technical executive

Medium · communication · uncertainty · pricing

A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.

Approach

  1. Recognise what is being probed: whether you can carry uncertainty into a decision instead of either hiding it or hiding behind it. The generic answer promises to explain the interval in plain English; the strong one replaces the question 'what is the number' with 'across this range, where does the decision change'.
  2. Find the threshold before you draft anything. Ask what the pricing case assumes, then compute the conversion drop at which the higher price stops adding revenue: price uplift on the conversions kept against the revenue lost from conversions forgone. That single figure is what makes the range legible.
  3. Restate the estimate and both bounds in the unit the audience already reasons in. Convert percentage points into monthly first-paid conversions at current trial volume, then into mrr_cents_constant_fx, so the slide reads as money per month rather than as statistics.
  4. Place the range against the break-even and say which part of it sits on each side. If most of the range clears the threshold, that is a recommendation to proceed with a monitoring plan; if the range straddles it, that is a recommendation to narrow the range first.
  5. Name what would narrow it and what that costs in weeks, then give one recommendation with an explicit condition for revisiting it. Uncertainty stated without a next step is read as indecision and the midpoint gets used anyway.

Follow-up

  • The executive says to give the midpoint and they will manage the risk. What do you do?
  • How does the slide change if the interval were a 4.6pp to 0.2pp drop, with no positive outcomes in range?
  • Why is a geo holdout the credible read here rather than the attributed channel numbers you already have?

Handle a request for numbers supporting a decision already made

Hard · integrity · framing · stakeholder

A senior leader has already decided to sunset a plan tier and asks you for the analysis showing it is the right call. Accounts on that tier carry 6% of MRR at constant FX and have the highest licensed-seat utilisation in the book. The leader's support matters to your next review cycle, and the decision is being presented in four days. Deliver what you produce, what you decline to produce, and the exact sentence you will say in the meeting where the number appears on a slide.

Approach

  1. Recognise what is being probed: whether you can find the legitimate request inside an illegitimate framing instead of either complying or refusing on principle. The generic answer promises to push back; the strong one produces something genuinely useful and states its limits in the room, without ambushing anybody.
  2. Separate the decision from the justification. Sunsetting the tier may be correct for reasons the data does not hold, such as support cost, roadmap surface area or sales motion. What you decline is a one-sided document. What you produce is the case read both ways, which also happens to be more useful to the leader.
  3. Build the symmetric analysis: MRR at risk at constant FX, the share of affected accounts with a plausible migration path given seats_licensed and billing_term, the recovery rate assumed for that migration and where it came from, and the downside case in which high-utilisation accounts treat the sunset as a reason to re-evaluate the vendor entirely.
  4. Surface the inconvenient fact privately and early. The highest seat utilisation in the book is a retention signal, and the leader should hold it before the room does, so they can incorporate it rather than be caught by it.
  5. Agree the meeting sentence in advance with the leader, so that nobody is surprised. Something to the effect that the tier is 6% of MRR and its accounts are the most heavily used in the book, and that the case for sunsetting rests on cost and focus rather than on revenue. That is true, it supports the decision on its real grounds, and it stops the deck claiming the numbers endorse it.
  6. Decide your own line before you need it: what you will not put your name to, and that the route if asked anyway is your own manager rather than a confrontation in the meeting.

Follow-up

  • The deck circulates with your analysis included and the downside case removed. What do you do, and by when?
  • What changes if the honest analysis says the sunset is clearly the wrong call?
  • How do you write the same memo when the leader is your skip-level and the meeting is tomorrow?