Interview guides · SAS

SAS Data Scientist Interview Questions & Guide 2026

SAS data scientist prep: probability, sampling distributions, and stating the assumption behind every test you run.

The 60-second brief

As a Data Scientist at SAS, you will stand at the forefront of advanced analytics, enterprise software development, and cutting-edge machine learning innovation. This role is crucial for driving the intelligence behind industry-leading data management and analytics solutions that empower global enterprises to transform complex data into actionable decisions. You will collaborate closely with software engineers, product managers, and domain experts to design, test, and scale sophisticated analytical models and data-driven product features.

If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.

Focus areas

Read NRR on a fixed account cohort
Separate contracted seats from actively used seats
Analyse at the account grain, cluster errors

Overview

Your day-to-day impact involves translating ambiguous business requirements into robust technical specifications, executing end-to-D-to-end data experiments, and deploying high-performance machine learning pipelines. Whether you are optimizing core analytics engines, exploring generative AI applications, or refining enterprise-grade statistical models, your work directly influences how millions of users interact with enterprise data platforms. The role demands a rare combination of rigorous statistical foundations, production-ready coding capabilities, and sharp product intuition.

Navigating the interview loop at requires you to demonstrate technical mastery balanced with a clear understanding of enterprise software ecosystems. You will encounter a structured process designed to evaluate both your theoretical knowledge and your practical execution capabilities. Expect a supportive yet rigorous environment where clarity of thought and precision in problem-solving matter more than memorizing trivia.

The SAS interview process

SAS candidates report 6 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Online Application

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Video Assessment

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Recruiter Consultation

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Screening

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Deep-Dive Technical Discussions

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Panel Presentation

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Reporting a mean over accounts when account revenue is heavy-tailed

When a small number of accounts hold most of the revenue, the sample mean is dominated by whichever of them happens to be in the sample, and the sample variance keeps growing as more data arrives instead of stabilising. In that regime the usual central-limit-based confidence interval understates uncertainty, and a single renewal or a single large account's batch job can flip the sign of a measured effect. The fixes are to pre-register a winsorisation or capping rule before looking at the outcome, to report account counts crossing a threshold alongside the revenue figure, or to define the estimand on a bounded transform. Choosing the cap after seeing the result is a separate and worse problem, because the cap then encodes the answer.

Randomising an experiment at the user level when users share an account

Two problems fire at once. Colleagues in one workspace see each other's work and talk to each other, so a treated user changes the behaviour of a control user in the same account, which violates the no-interference assumption and biases the estimate toward zero. Separately, outcomes within an account are strongly correlated, so the effective sample size is roughly n / (1 + (m - 1) * rho) for m users per account and intra-class correlation rho, not n. With rho around 0.3 and twenty users per account that is a design effect near 6.7, meaning a user-level confidence interval is about two and a half times narrower than it should be and results cross significance thresholds on noise alone. Randomise the account and cluster the standard errors.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Naming a model class before naming the deployment constraints

Set out the latency budget, the label delay, the retraining cadence, the interpretability requirement and the number of labelled examples, then pick the model that fits them. A boosted-tree answer to a problem where each decision must be explained to the affected user is a well-executed answer to the wrong question.

Questions & practice

Explain how Central Limit Theorem applies when building confidence intervals for sparse datasets.

Medium · statistics and probability

Approach

Translate the result into the decision it informs, in one plain sentence.
Sanity-check the answer against a simple bound or a simulated case.
Say what the estimate is of, and over what population it generalises.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

How would you design an A/B testing framework for a major backend algorithm update?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Set a baseline first, so any model has something honest to beat.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Permutation-test a consumption experiment randomised at account level

Hard · permutation-test · heavy-tails · cuped · experiment-readout

An experiment randomised 900 accounts into two arms. You have one row per account: account_id, arm, consumption_28d (billable units after launch) and consumption_pre (the 28 days before). Consumption is heavy-tailed and the largest account is several percent of the total. Write a permutation test from scratch: winsorise at the pooled 99th percentile as a pre-registered rule, use the difference in arm means of the winsorised outcome as the statistic, and obtain a two-sided p-value from 20,000 relabellings of the account-level arm vector. Report the observed effect, the p-value, and the same test on a CUPED-adjusted outcome.

Approach

Be precise about what the permutation test needs. Under the sharp null of no effect for any account, the outcomes are exchangeable across arm labels, and the test is valid for ANY statistic T(outcomes, labels) provided the identical function is applied to the observed labels and to all 20,000 relabellings. The pooled 99th percentile is a function of the outcome vector alone, so recomputing it inside the loop returns the same number 20,000 times: that is wasted CPU, not a bias, and hoisting it out is an optimisation rather than a correctness fix. Say plainly that capping at all changes the estimand from mean consumption to mean capped consumption; it is not a neutral cleaning step.
The mistake that does invalidate the test is an asymmetry between the observed statistic and the permuted ones, and the easiest way to create it is to derive the cleaning rule from the observed arm labels and then freeze it — winsorise each arm at its own observed 99th percentile, hold those two caps fixed, and permute. The observed value is then computed with caps matched to its own partition while every relabelling is scored with caps belonging to a different one, so the null distribution no longer answers the question the p-value claims to answer. A per-arm cap recomputed consistently inside every permutation is a valid test, but it estimates a contrast whose two sides are capped at different thresholds, so prefer the pooled cap on estimand grounds and pre-register it.
Permute the account-level arm vector, because the account is the randomisation unit. Relabelling anything finer — users, workspaces, requests — generates a null distribution narrower than the design actually supports and returns p-values that are anti-conservative.
Vectorise the null: tile the treatment indicator into a (B, n) matrix and permute along axis 1 with rng.permuted(..., out=...). The statistic is a difference of means, so the treated sum alone determines it and the whole null is one matrix-vector product. Use the two-sided p-value (1 + count(|stat_perm| >= |stat_obs|)) / (B + 1); the plus-one on each side is not cosmetic, it keeps the p-value away from exactly zero and keeps the test valid at finite B.
For CUPED, fit theta = cov(y, x) / var(x) on the pooled data and use that same theta for the observed statistic and every relabelling. Pooled theta, like the pooled cap, carries no label information, so where in the loop you compute it is again only a performance question; fitting theta within arms is what goes wrong, because the adjusted outcome then depends on the labels and an observed-label fit frozen across all 20,000 relabellings breaks the match between observed and permuted statistics. x must be measured entirely before launch, which consumption_pre is. Expected variance reduction is about 1 - corr(y, x)^2; measure the achieved reduction from the two null distributions rather than asserting it.

Worked solution (45 min)

cap_y = np.quantile(df.consumption_28d, 0.99); y = np.minimum(df.consumption_28d.to_numpy(float), cap_y); cap_x = np.quantile(df.consumption_pre, 0.99); x = np.minimum(df.consumption_pre.to_numpy(float), cap_x)
t = (df.arm == 'treatment').to_numpy(); n1 = int(t.sum()); n0 = len(t) - n1; obs = y[t].mean() - y[~t].mean()
rng = np.random.default_rng(11); L = np.tile(t.astype(np.int8), (20_000, 1)); rng.permuted(L, axis=1, out=L); s1 = L @ y; stats = s1/n1 - (y.sum() - s1)/n0
p = (1 + int(np.sum(np.abs(stats) >= abs(obs)))) / (20_000 + 1)
theta = np.cov(y, x, ddof=1)[0,1] / np.var(x, ddof=1); y_adj = y - theta*(x - x.mean()); repeat steps 2 to 4 on y_adj and compare stats.std(ddof=1) between the two runs.

Expected result: A two-sided p-value strictly between 1/20001 and 1, an observed effect expressed in capped billable units per account, and a CUPED null whose standard deviation is smaller than the unadjusted one by roughly sqrt(1 - corr(y, x)^2). The CUPED point estimate stays close to the unadjusted one, since the adjustment removes variance rather than shifting the effect.

Verify it yourself

The permutation null is centred near zero: |mean(stats)| is a small fraction of std(stats).
An A/A run, relabelling arm at random before the test and repeating over seeds, gives roughly uniform p-values with about 5% below 0.05.
std(stats_cuped) < std(stats) whenever corr(y, x) is materially non-zero, and the ratio matches sqrt(1 - r^2) to within simulation noise.
Moving the pooled cap computation inside the permutation loop reproduces the same p-value bit for bit at the same seed — the direct demonstration that a pooled cap carries no label information — while setting the cap to infinity reproduces the uncapped test exactly, so the winsorisation switch is independently verifiable.

Follow-up

The p-value is 0.04 with the cap and 0.31 without it. What do you report, and what did you pre-register?
Colleagues in a shared workspace can see the treated behaviour. How does that change the design and the estimate?
How many accounts would you need to detect a 5% lift given this outcome's distribution?

Write a query to identify the top three highest-spending enterprise accounts per region using ranking functions.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

This category tests your ability to query, join, aggregate, and transform large datasets efficiently using standard relational database concepts and advanced analytical functions.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Running commitment burn-down and the date consumption crosses it

Medium · window functions · running totals · committed consumption

fct_subscription_period gives committed_amount_cents, term_start_date and term_end_date for each account's current version where pricing_model = 'committed_consumption'. fct_usage_daily gives account_id, workspace_id, sku_code, usage_date and net_amount_cents, at one row per workspace and SKU per day. Inside each account's current term, return the running total of net_amount_cents by usage_date, the first usage_date on which that running total reaches committed_amount_cents, and the fraction of the term elapsed at that point. Accounts that have not reached their commitment must still appear, with a null crossing date.

Approach

Collapse usage to one row per (account_id, usage_date) first. The fact is grained by workspace and SKU, so a raw running total leaves several rows per date and the first-crossing date becomes dependent on the arbitrary order of rows inside that day.
Restrict to the term with usage_date BETWEEN term_start_date AND term_end_date on the account's current row, so no prior term's consumption leaks into this term's burn-down.
Compute SUM(net_cents) OVER (PARTITION BY account_id ORDER BY usage_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW). Name ROWS explicitly: the default frame is RANGE, which includes all peer rows at the same ORDER BY value and would hand back the whole day's total on each of that day's rows.
Extract the crossing with MIN(usage_date) FILTER (WHERE running_cents >= committed_amount_cents) grouped per account, which returns NULL for accounts still under commitment instead of dropping them.
Elapsed fraction is (crossing_date - term_start_date)::numeric / NULLIF(term_end_date - term_start_date, 0); Postgres date subtraction yields whole days, so the guard matters for same-day terms.
Exclude the trailing days still inside the metering settling window, measured from first_written_at against restated_at, and say how many days you cut and why.

Worked solution (30 min)

Build terms: SELECT account_id, committed_amount_cents, term_start_date, term_end_date FROM fct_subscription_period WHERE is_current AND pricing_model = 'committed_consumption' AND committed_amount_cents IS NOT NULL.
Build daily: join fct_usage_daily to terms on account_id with usage_date inside the term, then GROUP BY account_id, usage_date summing net_amount_cents.
Add running_cents with SUM(...) OVER (PARTITION BY account_id ORDER BY usage_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW).
Aggregate per account with MIN(usage_date) FILTER (WHERE running_cents >= committed_amount_cents) AS crossing_date, then compute the elapsed fraction with the NULLIF guard.
LEFT JOIN that summary back onto terms so every committed account appears, and verify the last running_cents per account equals a plain SUM over the term.

Expected result: A per-account-per-day series of running_cents inside the current term, plus one summary row per committed account carrying crossing_date and term_elapsed_fraction, with crossing_date NULL exactly for accounts whose final running total is below committed_amount_cents.

Verify it yourself

The maximum running_cents per account equals SUM(net_amount_cents) over that account's term computed independently.
crossing_date is NULL if and only if the final running total is below the commitment.
term_elapsed_fraction falls in [0, 1] for every non-null crossing date.
running_cents is non-decreasing within each account, which fails immediately if a negative credit row was included without comment.

Follow-up

Turn this into an end-of-term overage forecast. What breaks if you extrapolate a linear run rate on a consumption product?
An account crosses its commitment at 40 percent of the term. Is that an expansion signal or a billing-surprise risk, and what would you check to tell them apart?

How do you determine whether a drop in user retention is seasonal or caused by a recent product release?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Your primary product metric dropped by fifteen percent overnight; walk through your diagnosis framework.

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize competing requests from product managers and engineering leads?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Extract month-over-month retention cohorts using conditional aggregation and self-joins.

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you compute statistical significance when dealing with heavily skewed conversion metrics?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

What steps do you take when your experiment lacks sufficient statistical power due to low user traffic?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

Cut variance with pre-period usage before the test starts

Medium · cuped · variance-reduction · stratification

You are planning an account-randomised test on 2,800 paying accounts. The outcome is a 28-day sum of billable_quantity for sku_code = 'compute_hours' from fct_usage_daily, and the same account's 28-day pre-period sum correlates 0.80 with it. 420 accounts were created inside the pre-period and have partial or no history. Specify the variance-reduction plan: the adjusted estimator and where its coefficient comes from, how assignment is stratified, how the 420 incomplete accounts are handled, and what must change in the pre-period window given that fct_usage_daily rows are restated after first write.

Approach

Write the estimator explicitly: Y_adj = Y - theta * (X - mean(X)), with theta = Cov(X, Y) / Var(X). Estimate theta from pre-experiment history or pooled across arms, never separately by arm and never from post-treatment outcomes. Fitting theta on treatment-arm outcomes folds the effect being measured into the adjustment and biases the result toward whatever the treatment did.
Quantify the gain and convert it into the currency the team cares about. The residual variance multiplier is 1 - 0.80^2 = 0.36, so the standard error falls to 0.60 of its unadjusted value and the MDE falls with it. That is the same precision as running with 1 / 0.36, about 2.8 times as many accounts, which matters because the account population is fixed and cannot be bought with a longer run.
Stratify assignment on pre-period usage decile crossed with the three paid plan tiers, giving 30 cells at roughly 93 accounts each, and collapse any cell below about 20. Use the same strata in the analysis through strata fixed effects or post-stratification: stratified assignment analysed pooled discards much of the gain, and stratified analysis without stratified assignment risks empty cells in the top decile, which is precisely where the revenue sits.
Handle the 420 incomplete accounts by imputing X at the stratum mean and adding a binary indicator for missing pre-period, rather than dropping them. Dropping silently redefines the population to established accounts, which is usually the opposite of the segment a new feature targets, and it makes the result non-generalisable in a way the readout will not disclose.
Fix the window against restatement. Measure the empirical settling time by comparing a usage_date's total at first_written_at against its value after restated_at has stopped moving, then end the pre-period that many days before assignment. A pre-period whose last days are still settling carries recency-correlated measurement error in the covariate, which both weakens rho and can correlate with assignment date.
Pre-register the whole plan before assignment: the estimation set for theta, the strata definition and collapsing rule, the imputation rule, the winsorisation cap and the trailing exclusion. Every one of these can be tuned after the fact to move a p-value, which is why they are worth nothing if decided afterwards.

Worked solution (30 min)

Estimate rho on a historical pair of adjacent 28-day windows, applying exactly the traffic-class and is_internal filters the live experiment will use.
Compute the variance multiplier 1 - 0.80^2 = 0.36 and translate it to a standard-error multiplier of 0.60 and an effective-sample multiplier of about 2.8.
Build 30 strata from ten pre-period usage deciles crossed with three paid plan tiers, inspect the minimum cell count and collapse cells below about 20 accounts.
Measure the metering settling time from first_written_at against restated_at and shift the pre-period window back by that many days.
Write the pre-registration: theta source, strata, imputation for the 420 incomplete accounts, winsorisation cap, trailing exclusion.

Expected result: Standard-error multiplier 0.60 and effective-sample multiplier about 2.8 at rho = 0.80, with 30 strata collapsed where thin, the 420 incomplete accounts retained through stratum-mean imputation plus a missingness indicator, and the pre-period ending one measured settling window before assignment.

Verify it yourself

Adjusted and unadjusted point estimates must be close under valid randomisation, because this reduces variance rather than moving the estimate. A large shift means theta was fitted on post-treatment data or that randomisation itself failed.
The multiplier 1 - rho^2 is monotone in the absolute correlation and equals 1 at rho = 0, so at rho = 0.30 the standard error can only fall by about 5%. Any larger claimed reduction at that correlation is arithmetically wrong.
Computing the pre-period sum twice, a week apart, should return the same number once the settling window is excluded. If it does not, the exclusion window is too short.

Follow-up

Once continuous-integration and synthetic traffic are excluded, rho turns out to be 0.45 rather than 0.80. What is the revised variance reduction, and is the added complexity still worth it?
How would you extend this to more than one covariate, and what stops you from adding twenty?
Does this adjustment repair an imbalance you discover after assignment, or only reduce variance? Be precise about the difference.

Monthly logo churn triples with no change in satisfaction

Hard · renewal-eligible base · censoring · timestamp semantics

Monthly logo churn, computed as churned accounts divided by all paying accounts, tripled last month. Contracts are annual. From fct_subscription_period (account_id, term_start_date, term_end_date, amendment_type, auto_renew, booked_at, superseded_by_id, is_current) and dim_account (account_id, churned_at, account_status, employee_band, acquisition_channel), rebuild churn on the renewal-eligible base, separate calendar effects from customer behaviour, and state whether retention actually changed. Note that churned_at is sometimes set when the record was updated rather than at term end.

Approach

Rebuild the denominator as accounts whose term_end_date falls in the month. An account eleven months from renewal sits in the current denominator while being structurally incapable of entering the numerator, so on annual contracts the published rate understates the truth by roughly the reciprocal of the annual renewal fraction and moves with the signing calendar.
Plot the renewal-eligible base by month across two years. A signing surge twelve months earlier reproduces itself as an eligibility surge now, and a rate whose denominator ignores that tracks the sales calendar rather than customer sentiment.
Date each churn by term_end_date, never by churned_at or an update timestamp. Inspect the distribution of churned_at minus term_end_date: a backlog cleared in one batch appears as a mass at a single date and shifts losses into whichever month the operations team did its paperwork.
Apply the 45-day grace for late renewal paperwork so the most recent month and a half is marked not reportable, rather than printing a number that will rise once the paperwork lands.
Compare corrected gross logo retention against its own trailing distribution, and if a real change survives, cut it by employee_band, acquisition_channel and plan_tier before proposing any cause.

Follow-up

Some contracts in the window have not reached their renewal date. When does this require a survival estimator rather than a simple rate, and which one would you use?
How do you report churn to an audience that wants a monthly number when the underlying event is annual and lumpy?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

What the behavioural round is actually scoring

Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.

This category explores how you collaborate with cross-functional teams, manage ambiguity, and drive projects to completion.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Walk through an analysis you later discovered was wrong

Easy · data quality · error ownership · metering

Six weeks ago you reported that consumption fell 9 percent in the last week of the month, and a team spent a sprint investigating the cause. The fall was an artefact: rows in fct_usage_daily land late and are restated in place, and you queried before the tail had settled. Describe how you found the error, what you told the people who acted on it, and the control you put in place so this class of mistake cannot reach a dashboard again. Be specific about how the settling window was measured.

Approach

The interviewer is probing whether you self-report errors before someone else finds them, and whether your fix is structural rather than a promise to be more careful. Say plainly that the number was wrong and that a sprint was spent on it, before describing any diagnosis.
Establish the artefact quantitatively instead of asserting that data lands late. For each usage_date, compare the total as of first_written_at against the settled total and read the settling time off that curve, for example 97 percent of final by day three and 99.5 percent by day five.
Correct the record the same day, in the channel the original number went out in, to the same audience. The cost of the wasted sprint belongs in the correction, not in a footnote.
Make the fix structural: exclude a trailing lag window from every reportable figure, and make the reporting view return no rows inside that window rather than returning partial ones. A dashboard that shades unsettled days still gets read as a decline.
State what generalises. Any fact table restated in place has this failure mode, so the guard belongs at the source rather than on the one dashboard that embarrassed you. A strong answer ends with the class of error closed; a generic one ends with a lesson learned.

Follow-up

How did you choose the completeness threshold behind the lag window, and what would make you recalibrate it?
What did you say to the team that lost the sprint, and what did they say back?
Is there a legitimate case for showing the unsettled tail at all, and to whom?

State the measured impact of your own work honestly

Hard · impact · causal inference · self-assessment

You are asked for the business impact of a renewal-risk worklist you shipped nine months ago. Customer success used it, and renewals in the covered segment came in 6 points above the prior year. Coverage was assigned by the team itself: they worked the top of your list and also the accounts they were already worried about. Produce the impact claim you are willing to defend, the number you refuse to claim, and the design you would have asked for at the start.

Approach

The interviewer is probing whether you can separate what you shipped from what you caused, and whether you would have built the measurement in rather than reconstructing it afterwards. Both halves are being scored.
Name the confound precisely. Coverage assignment is doubly selected: the largest accounts get an owner because they are valuable, and distressed accounts get one because they are at risk. The naive covered versus uncovered comparison mixes a strong positive selection with a strong negative one and can come out with either sign depending on which rule dominated. Matching on account size does not fix it, because the risk signal that triggered coverage is the same signal that predicts the outcome.
Split the claims by what each needs to be true. Ranking quality is defensible from precision at k on out-of-time renewals. Adoption is defensible from timestamps showing what share of listed accounts were contacted. The outcome claim is not defensible without a design, and saying so is the point of the exercise.
Look for identification before giving up on it. A capacity cut-off, a territory boundary, or a period in which the list existed but was unstaffed can assign coverage for reasons unrelated to account health, and any of those supports a bounded estimate.
State the design you would ask for now and its price: a randomly withheld slice of the list, held for two renewal quarters, with the expected cost in renewals stated openly. That cost is what it takes to be able to answer this question at all.
Give a bounded number rather than none. Six points with an explicit statement of how much of it you can attribute is more useful than either claiming the whole figure or declining to quantify anything.

Follow-up

Your manager wants the 6 points in a promotion packet. What wording do you accept, and what do you strike?
What would have had to be true for the naive covered versus uncovered comparison to be valid?
If the holdout costs the team real renewals, how do you justify asking for it, and to whom?