cswork / libraryOpen practice workspace

Interview guides · Exl

Exl Data Scientist Interview Questions & Guide 2026

Exl Data Scientist interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.

The 60-second brief

A Data Scientist at Exl operates at the intersection of advanced analytics, domain-specific business strategy, and scalable technology. You are not just building models; you are solving complex, high-stakes problems for global clients, often translating raw data into actionable insights that drive operational efficiency and revenue growth. Your work directly impacts how organizations leverage Artificial Intelligence, Machine Learning, and Predictive Modeling to navigate competitive markets.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

  • Separate contracted seats from actively used seats
  • Power experiments for heavy-tailed account revenue
  • Analyse at the account grain, cluster errors

Overview

This role requires a unique blend of technical rigor and business acumen. Whether you are working on Generative AI applications, Natural Language Processing (NLP), or building robust Recommendation Systems, you will be expected to own the end-to-end lifecycle of your solutions. You will collaborate with cross-functional teams, including engineering, product, and client management, to ensure that the analytical frameworks you build are not only statistically sound but also deliver tangible value to the end user.

Exl values candidates who can demonstrate a clear "solutioning" mindset. It is not enough to know the math; you must be able to explain how your model solves a specific business challenge.

The Exl interview process

Exl candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Screening Phase

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

  • Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
  • The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
  • Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

  • Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
  • Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
  • Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Technical Assessments

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

  • Whether the query you write matches the plan you just described
  • What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
  • Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

  • Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
  • Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
  • Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Case Studies

This round runs as a working session, so part of what it decides is whether you are useful to think with. The interviewer will interrupt: a hint that the data you assumed does not exist, a challenge to your metric, a nudge toward a branch you skipped. Treating those as interference is the common failure. Reason out loud while your thinking is still provisional so there is something to react to, and when a redirect arrives, take it instead of defending the path you had already started down.

What to demonstrate

  • Whether your reasoning is audible while it is still unsettled, or only after you have privately decided
  • What you do with a hint: absorb it and adjust, or argue for the original route
  • Whether your clarifying questions have answers that would change your approach, as opposed to filling silence
  • Whether you can be wrong about something in the middle of the case and keep moving without restarting

How to prepare

  • Run practice cases with a partner instructed to interrupt twice: once to remove a data source you assumed existed, once to reject the metric you chose. Practise absorbing both without going back to the start.
  • Before each practice case, write down the clarifying questions you plan to ask, then check afterwards whether any answer actually changed what you did. Drop the ones that did not.
  • Explain an analysis you already know well to someone outside the field and have them stop you at every point where the reasoning jumped a step.

Final Round

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

  • Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
  • Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
  • Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
  • Whether you ask what decision is actually on the table before explaining anything

How to prepare

  • Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
  • For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
  • Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Computing monthly churn against the entire customer base when contracts are annual

An annual contract has no opportunity to churn except at its renewal date, so an account that is eleven months from renewal is in the denominator while being incapable of appearing in the numerator. The resulting rate is smaller than the real one by roughly the ratio of the base to the renewal-eligible base, and it oscillates with the seasonality of when deals were originally signed rather than with anything about the customers. The corresponding trap on the other side is counting a churn on the date the record was updated rather than on term_end_date, which shifts losses into whichever month the operations team did its paperwork.

Treating raw request or usage volume as engagement

Most traffic in this domain is emitted by machines. Continuous-integration pipelines, scheduled batch jobs, synthetic monitors, backfills and client retries can all grow by an order of magnitude from one configuration change made by one engineer, and none of it represents a new decision to use the product. The inversion is what makes it dangerous: when the platform degrades, clients retry, so error-driven retry volume rises at the exact moment the customer is most likely to leave, and an engagement dashboard built on raw counts shows growth immediately before a churn. Filter on traffic_class and on successful status before anything else, and keep failed-request volume as its own separate series.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Ending an analysis without a recommendation or next step

Close with what you would do and what would change your mind, stated as a condition you can check later. If the evidence is genuinely inconclusive, recommend the specific next measurement and say what it costs in time or exposure.

Questions & practice

Explain the concept of autocovariance and its role in time-series forecasting.

Medium · statistics and probability

Approach

  1. Write down the assumption the method needs before you use the method.
  2. Say what the estimate is of, and over what population it generalises.
  3. Sanity-check the answer against a simple bound or a simulated case.

Follow-up

  • Which assumption here is most likely to be violated in practice?
  • What sample size would you need to detect an effect half this size?

How would you handle class imbalance in a classification model?

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Say how the offline result would be validated online before it is trusted.
  3. Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • How would you choose the decision threshold, and who owns that choice?

If a client’s model performance suddenly drops in production, what is your diagnostic approach?

Medium · machine learning and modelling

Approach

  1. Say how the offline result would be validated online before it is trusted.
  2. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • How would you choose the decision threshold, and who owns that choice?
  • Where could label leakage enter this setup?

Bootstrap a confidence interval for net revenue retention

Medium · bootstrap · retention · resampling

You have one row per account with arr_start_cents (ARR twelve months ago) and arr_end_cents (ARR today, zero if churned), covering the fixed cohort of accounts that had ARR twelve months ago. Net revenue retention is sum(arr_end_cents) / sum(arr_start_cents). Write a nonparametric bootstrap from scratch, without scipy.stats.bootstrap: resample accounts with replacement, recompute the ratio of sums on each resample, and return the point estimate with a 95% percentile interval from 10,000 resamples. Also report the interval you would get from the mean of per-account ratios, and explain the difference.

Approach

  1. Resample the account, because the account is the unit the estimand is defined over. One bootstrap draw is a vector of account indices and both numerator and denominator are recomputed from that same draw; resampling the two sides independently destroys the within-account correlation that makes a ratio estimator stable.
  2. Vectorise the draws: idx = rng.integers(0, n, size=(B, n)), then end[idx].sum(axis=1) / start[idx].sum(axis=1). A 10,000 by n index matrix is usually far cheaper than a Python loop; if the matrix is too large for memory, chunk over B rather than reverting to a loop.
  3. Take the interval from np.quantile(ratios, [0.025, 0.975]). The percentile interval differs from estimate +/- 1.96 * bootstrap SE whenever the resample distribution is skewed, which it will be here, and the skew is the thing you want represented.
  4. Compute the mean-of-ratios version on the same resamples, and state the exact relationship rather than guessing which of the two is larger. With r_i = arr_end_i / arr_start_i, the ratio of sums is the arr_start-weighted mean of exactly those r_i, so sum(end)/sum(start) - mean(r) = Cov(arr_start, r) / mean(arr_start) using the population covariance. The gap is positive when larger accounts retain and expand better than smaller ones, and negative when they do not; a cohort whose small accounts churn at a higher rate has positive covariance, which puts the mean of per-account ratios BELOW the ratio of sums. Requires arr_start_i > 0 for every account, which the fixed-cohort definition guarantees; r_i is floored at 0 and unbounded above, so a handful of 4x expansions among small accounts can flip the sign. Compute the covariance and report it instead of asserting a direction.
  5. Report the interval width beside the concentration of the cohort. If the largest account is 12% of starting ARR, a narrow interval is evidence that the resampling unit is wrong rather than evidence that the estimate is precise.

Worked solution (30 min)

  1. start = df.arr_start_cents.to_numpy(float); end = df.arr_end_cents.to_numpy(float); n = len(start); point = end.sum() / start.sum()
  2. rng = np.random.default_rng(7); idx = rng.integers(0, n, size=(10_000, n)); ratios = end[idx].sum(1) / start[idx].sum(1)
  3. lo, hi = np.quantile(ratios, [0.025, 0.975]); return point, lo, hi
  4. per_acct = end / start; mean_point = per_acct.mean(); mean_boot = per_acct[idx].mean(1); compare np.quantile(mean_boot, [0.025, 0.975]) against (lo, hi), and report np.cov(start, per_acct, ddof=0)[0,1] / start.mean() as the quantity that accounts for the gap between the two centres.

Expected result: A point estimate equal to sum(end)/sum(start) on the unresampled data — the bootstrap mean is close but must not be substituted for it — and a percentile interval that is asymmetric around the point when expansion is skewed. The mean-of-ratios interval is centred on a different estimand, and which side it falls on is decided by Cov(arr_start, r): it sits below the ratio of sums in the common case where small accounts churn at a higher rate than large ones, and above it when the large accounts are the ones shrinking. Its width is governed by the spread of per-account growth, while the ratio-of-sums width is governed by how concentrated starting ARR is, so neither interval is reliably the wider one.

Verify it yourself

  • point equals sum(end)/sum(start) computed directly on the original arrays, and point - per_acct.mean() equals np.cov(start, per_acct, ddof=0)[0,1] / start.mean() to floating-point tolerance. That is an algebraic identity, not an empirical claim: a mismatch means the per-account ratios or the weights are wrong, and the sign of it tells you which of the two numbers is larger before you look.
  • np.median(ratios) sits within a small fraction of a point of the point estimate; a large gap indicates a wrong resampling unit or a transposed index matrix.
  • Raising B from 10,000 to 20,000 moves the endpoints by far less than the interval width.
  • Setting every arr_end equal to its arr_start gives point == 1.0, per_acct.mean() == 1.0, a zero covariance term and a zero-width interval.

Follow-up

  • The cohort has 800 accounts and the largest is 12% of starting ARR. How much do you trust a percentile interval here?
  • How would you extend this to an interval on the year-over-year change in NRR?
  • Two accounts merged mid-window and one contract was co-termed into the other. How do you keep the cohort fixed?

Given two tables, how would you perform a pivot operation or a complex merge in Python/Pandas?

Medium · sql and data manipulation

Approach

  1. Compute rates by summing numerator and denominator separately, never by averaging rates.
  2. Check whether any join is one-to-many before aggregating, or the sums inflate.
  3. State the window function and its partition and ordering out loud before writing it.

Follow-up

  • How would you verify this result without re-running the same query?
  • How does the query change if the join becomes one-to-many?

Write a SQL query to identify the top N users based on specific activity metrics.

Medium · sql and data manipulation

Approach

  1. Compute rates by summing numerator and denominator separately, never by averaging rates.
  2. State the window function and its partition and ordering out loud before writing it.
  3. Say which table is the grain you start from, and join outward from it.

Follow-up

  • How would you verify this result without re-running the same query?
  • What breaks if events arrive late or out of order?

Collapse retried requests into logical operations per account

Medium · window functions · deduplication · null semantics

fct_api_request carries request_id, account_id, api_key_id, idempotency_key (null when the caller supplied none), is_retry, request_at and http_status. For one ISO week, count logical operations rather than HTTP calls: requests sharing (account_id, api_key_id, idempotency_key) collapse to the earliest one, while every request with a null idempotency_key is its own operation. Return per account the raw request count, the logical operation count and the ratio between them, and state which direction a rising 5xx rate pushes that ratio.

Approach

  1. Split the stream before deduplicating. GROUP BY and PARTITION BY treat NULLs as equal to one another, while the = operator does not, so a single grouping pass folds every null-key request in an account into one 'operation' and can cut the count by orders of magnitude.
  2. Deduplicate the keyed rows with ROW_NUMBER() OVER (PARTITION BY account_id, api_key_id, idempotency_key ORDER BY request_at, request_id) and keep rn = 1. Including request_id in the ORDER BY makes the result deterministic when two rows share a millisecond, which they will.
  3. UNION ALL the null-key rows back in unchanged; they need no dedup and must not pass through the partition.
  4. Aggregate per account: count() over the raw week, count() over the union, and the ratio of the two. Report the ratio, not the difference, so accounts of very different sizes are comparable.
  5. Cross-check against is_retry, which flags a resend of the same idempotency key: raw minus logical should sit close to the count of is_retry rows, and a large gap means clients are resending without a key and the dedup is under-counting duplicates.
  6. Name the inversion explicitly: 5xx responses provoke client retries, so amplification rises exactly when reliability falls, and any engagement metric built on raw request counts will show growth during an outage.

Worked solution (25 min)

  1. Filter to the ISO week with request_at >= :week_start AND request_at < :week_start + interval '7 days', and record the raw count per account as the baseline.
  2. Build keyed_dedup: SELECT ... ROW_NUMBER() OVER (PARTITION BY account_id, api_key_id, idempotency_key ORDER BY request_at, request_id) AS rn FROM the week WHERE idempotency_key IS NOT NULL, then keep rn = 1.
  3. Build unkeyed: the same week's rows WHERE idempotency_key IS NULL, taken as-is.
  4. UNION ALL the two, then GROUP BY account_id selecting the logical count; join back to the raw counts and compute raw::numeric / logical.
  5. Validate on one account by comparing raw - logical against COUNT(*) FILTER (WHERE is_retry) for that account and explaining any gap.

Expected result: One row per account with raw_requests, logical_operations and an amplification ratio at or above 1.0, where logical_operations counts each distinct (account_id, api_key_id, idempotency_key) once and each null-key request once.

Verify it yourself

  • logical_operations is never greater than raw_requests, and the two are equal for accounts whose keys are all distinct.
  • For accounts with no null idempotency_key, raw_requests - logical_operations is close to the count of is_retry rows.
  • Deliberately grouping on the key without the null split reproduces a visibly lower count, confirming the NULL collapse is the mechanism.
  • Re-running the dedup returns the same surviving request_id set, proving the ORDER BY tiebreak made it deterministic.

Follow-up

  • Which of these two counts belongs in the billable-units metric, and which in an engagement metric?
  • An account's amplification ratio jumps from 1.05 to 3.0 in a day. Name three causes and the query that separates them.

How would you design a Recommendation System for a retail client with sparse user data?

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

Perform a guesstimate on the number of transactions processed by a specific platform in one hour.

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • Which segment would you cut first, and what would that rule out?

How do you optimize Python code for large-scale data processing?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. Fix the population and the time window before naming any metric.
  3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

Explain the role of embeddings in modern LLM architectures.

Medium

Approach

  1. Work from the decision backwards to the evidence you would need.
  2. State your assumptions explicitly before working the problem.
  3. Clarify what is being asked and what a complete answer would contain.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Measure named coverage at an assignment threshold you cannot randomise

Hard · regression-discontinuity · instrumental-variables · selection-bias

Named customer-success coverage is assigned to accounts whose arr_cents at fiscal-year start is at least $50,000. About 88% of accounts above the line receive a named owner and about 9% below it do, through documented exceptions. Roughly 200 accounts sit within $15,000 of the cutoff, and gross logo retention on the renewal-eligible base runs near 0.88. Estimate the effect of coverage on renewal, name the design, state its identifying assumptions and the test for each, and say whether this data can answer the question at all.

Approach

  1. Say first why the naive comparison is unusable, because the answer turns on it. Coverage is assigned to the largest accounts because they are valuable and to distressed accounts because they are at risk, so selection runs in both directions at once and the naive estimate can come out with either sign depending on which rule dominated. Matching on observed size does not fix it: the risk signal that triggered coverage is the same signal that predicts renewal.
  2. Name the design and the estimand together. This is a fuzzy regression discontinuity with the $50,000 threshold as an instrument for coverage. The estimator is the Wald ratio, the jump in renewal at the cutoff divided by the jump in coverage probability of 0.88 - 0.09 = 0.79, and the estimand is a local average treatment effect for accounts near $50,000 whose coverage is actually decided by the rule, not the average effect of coverage across the book.
  3. Fit it as local linear regression on each side with a triangular kernel and a data-driven bandwidth (Imbens-Kalyanaraman, or Calonico-Cattaneo-Titiunik with bias-corrected robust intervals). A global polynomial fitted across the whole ARR range is not a substitute and is well documented to manufacture spurious jumps at interior points.
  4. Test the assumptions instead of asserting them. Run a density test for bunching just above $50,000, which is exactly what a sales team structuring a deal at $50,001 to win coverage would produce and which invalidates the design outright. Check continuity at the cutoff of employee_band, acquisition_channel, deployment_model and pre-period usage. Run placebo cutoffs at other ARR values, and plot the estimate against bandwidth.
  5. Do the power arithmetic before promising an answer. With about 100 accounts per side and a renewal base rate of 0.88 (standard deviation 0.325), the sharp-design MDE at 80% power and two-sided 0.05 is 2.80 * 0.325 * sqrt(2/100) = about 12.9 percentage points, and dividing by the 0.79 first stage gives about 16 points on the fuzzy estimand. No plausible effect of assigning an account manager is that large.
  6. Give the honest verdict with a route forward attached. This design cannot answer the question on one fiscal year. Pool several years at the same cutoff, accepting the added assumption that the coverage model and the threshold's meaning were stable across them, or ask for a randomised holdout among accounts near the threshold, which is cheap to justify precisely because the rule is already arbitrary there.

Worked solution (45 min)

  1. Plot the density of arr_cents in bins around $50,000 and run the manipulation test; stop and report the design as invalid if bunching is present.
  2. Plot the first stage, share of accounts with a named owner against ARR, and confirm the jump is close to 0.79 and locally flat on each side.
  3. Fit local linear regressions of renewal on ARR either side of the cutoff with a triangular kernel and data-driven bandwidth, take the Wald ratio, and report bias-corrected robust intervals.
  4. Run covariate-continuity checks, placebo cutoffs, and a bandwidth-sensitivity curve for the point estimate.
  5. Compute the MDE at the realised effective sample and write the verdict, powered or not, with the alternative design named.

Expected result: A fuzzy RD local average treatment effect at the $50,000 cutoff with a first stage of about 0.79, and an MDE of roughly 16 percentage points on the fuzzy estimand at about 100 accounts per side. The honest finding is that the design is underpowered by an order of magnitude relative to any credible coverage effect, so the recommendation is pooled fiscal years or a randomised holdout at the margin.

Verify it yourself

  • The first-stage jump and the outcome jump must be estimated on the same bandwidth. A Wald ratio assembled from two different bandwidths is not the regression discontinuity estimator.
  • Covariates that coverage could not have affected must be continuous at the cutoff. A jump in employee_band or acquisition_channel at $50,000 indicates either manipulation of the running variable or a second rule sharing the same threshold.
  • Point estimates should vary smoothly with bandwidth. An estimate that flips sign between a $10,000 and a $20,000 bandwidth is a fitting artefact rather than a finding.

Follow-up

  • The density test shows clear bunching just above $50,000. Is any part of the design salvageable, and what would you report instead?
  • Coverage also correlates with support ticket severity. Does adding severity as a control help, hurt, or do nothing in a regression discontinuity?
  • If you pool five fiscal years to gain power, what new identifying assumption have you taken on, and how would you check it?

Error rate halves while severe support tickets double

Hard · missing data · aggregation bias · reconciliation

The fleet-wide customer-visible error rate fell from 1.8 percent to 0.9 percent, while sev1 and sev2 tickets and reopened_count rose across the same fortnight. Using fct_api_request (account_id, environment, sdk_name, traffic_class, http_status, request_at, billable_units), fct_usage_daily (account_id, sku_code, usage_date, billable_quantity) and fct_support_ticket (account_id, severity, opened_at, reopened_count, linked_incident_id), determine whether reliability improved, and if not, identify precisely which rows are missing and from when. Deliverable: a diagnosis backed by an independent corroborating source.

Approach

  1. Distrust an improvement that contradicts an independent operational signal. Two sources disagreeing is itself the finding; decide which one is more likely to be broken before explaining either.
  2. Recompute the rate as the metric tree defines it, per account first and then as the share of accounts above the reliability target. A single global average is dominated by whichever account sends the most traffic, so a fleet number can fall while a quarter of accounts get worse.
  3. Audit for missingness rather than for badness: count fct_api_request rows per hour split by environment, sdk_name and status class, indexed against the trailing same-hour baseline. A partial ingestion failure shows as a step drop confined to one slice, not a uniform decline.
  4. Reconcile against a source the request pipeline does not feed, such as implied request volume from fct_usage_daily for the same accounts and dates. If the usage table is flat while request rows fell, rows are missing rather than traffic.
  5. Test whether the missingness is differential by status, which is the mechanism that fakes an improvement: if 5xx rows are written on a path that stopped while 2xx rows were unaffected, the numerator falls faster than the denominator and the rate drops with nothing improving.
  6. Close with the affected window, the affected slice and a restated series marked unreliable across that window, rather than a silently patched number.

Follow-up

  • The missing rows are unrecoverable. How do you present that fortnight in a series people compare week over week?
  • What monitor would have caught this within an hour, and what is its false-positive cost on a normal quiet weekend?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

  • Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
  • Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
  • Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

  • Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
  • On the third attempt, write the grain of every CTE as a comment before writing its body.
  • Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

  • Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
  • Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
  • Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

  • Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
  • Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
  • Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

  • Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
  • Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
  • Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

  • Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
  • Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
  • Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

  • Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
  • Re-read only your own notes from this week, and open no new material.
  • Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

Walk me through a project where you had to explain a complex model to a non-technical stakeholder.

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Pick a story where you drove the decision, not one where you observed it.

Follow-up

  • What did you decide not to do, and why?
  • What would you do differently if you ran that project again?

Report an underpowered consumption test to a non-technical executive

Medium · communication · uncertainty · experimentation

An account-randomised packaging change ran six weeks across 900 paying accounts. The effect on billable units per account per month is plus 4.1 percent, with a 95 percent interval from minus 3.2 to plus 11.8 after clustering standard errors at the account and applying the pre-registered winsorisation at the 99th percentile. An executive with no statistical background wants one number this week to decide a full rollout. Produce a three-sentence spoken answer, one chart, and an explicit recommendation of ship, stop or keep running, with the cost of each option stated.

Approach

  1. The interviewer is probing whether you can be decision-useful without either hiding the uncertainty or hiding behind it. Start from the decision rather than the statistics: establish what the executive would do differently at plus 4 percent versus zero, because if the action is identical the interval does not matter.
  2. Translate the interval into consequences in units the executive already reasons about. Multiply both endpoints by the cohort's baseline consumption and contracted rates to give an annualised revenue range, so the answer is a range of dollars rather than a range of percentages.
  3. Price the option to wait. Using the observed variance, state roughly how many additional account-weeks halve the interval width, so keep running becomes a quantified choice instead of a stall.
  4. Offer a cheaper path to the same decision: a lower-variance proximate outcome such as successful billable units on the new SKU, or CUPED using each account's pre-period consumption, quoting the expected variance reduction as one minus the squared pre-post correlation.
  5. Give a recommendation and name the single observation that would reverse it. A strong answer commits; a generic one recites the interval and leaves the decision on the table.

Follow-up

  • The executive says it clearly works and is just not provable, so ship it. What is your answer?
  • How much of the interval width comes from clustering and how much from the revenue tail, and what would you do about each?
  • If you had to ship this week with no more data, which guardrail would you watch for the first fortnight and at what threshold would you roll back?

Walk through an analysis you later discovered was wrong

Easy · data quality · error ownership · metering

Six weeks ago you reported that consumption fell 9 percent in the last week of the month, and a team spent a sprint investigating the cause. The fall was an artefact: rows in fct_usage_daily land late and are restated in place, and you queried before the tail had settled. Describe how you found the error, what you told the people who acted on it, and the control you put in place so this class of mistake cannot reach a dashboard again. Be specific about how the settling window was measured.

Approach

  1. The interviewer is probing whether you self-report errors before someone else finds them, and whether your fix is structural rather than a promise to be more careful. Say plainly that the number was wrong and that a sprint was spent on it, before describing any diagnosis.
  2. Establish the artefact quantitatively instead of asserting that data lands late. For each usage_date, compare the total as of first_written_at against the settled total and read the settling time off that curve, for example 97 percent of final by day three and 99.5 percent by day five.
  3. Correct the record the same day, in the channel the original number went out in, to the same audience. The cost of the wasted sprint belongs in the correction, not in a footnote.
  4. Make the fix structural: exclude a trailing lag window from every reportable figure, and make the reporting view return no rows inside that window rather than returning partial ones. A dashboard that shades unsettled days still gets read as a decline.
  5. State what generalises. Any fact table restated in place has this failure mode, so the guard belongs at the source rather than on the one dashboard that embarrassed you. A strong answer ends with the class of error closed; a generic one ends with a lesson learned.

Follow-up

  • How did you choose the completeness threshold behind the lag window, and what would make you recalibrate it?
  • What did you say to the team that lost the sprint, and what did they say back?
  • Is there a legitimate case for showing the unsettled tail at all, and to whom?