Interview guides · Celebal Technologies

Celebal Technologies Data Scientist Interview Questions & Guide 2026

Case practice for Celebal Technologies data scientist candidates: scope a vague prompt into a measurable question before you analyze it.

The 60-second brief

As a Data Scientist at Celebal Technologies, you play a pivotal role in transforming data into actionable insights that drive strategic decision-making. Your expertise in analyzing complex datasets and applying advanced analytical techniques is crucial for developing models that enhance products and improve user experiences. This role is not just about crunching numbers; it involves working on diverse projects that directly impact the business's bottom line and contribute to the company's innovative product offerings.

Scope your preparation by the data you would actually touch, because the title will not tell you. A seat that lives in event logs and weekly readouts rewards fluency in aggregation and metric definitions; a seat that owns a model in production rewards fluency in train/serve skew, retraining cadence and drift monitoring. The fastest way to find out which one you are interviewing for is to ask what the team shipped last quarter and what it gets paged about.

Focus areas

Strip CI, retry and synthetic traffic first
Power experiments for heavy-tailed account revenue
Measure churn only on renewal-eligible accounts

Overview

You will collaborate with cross-functional teams, including engineers, product managers, and business analysts, to identify opportunities for leveraging data to solve critical business problems. Your work will contribute to various domains, including machine learning, artificial intelligence, and big data analytics, making it an exciting and dynamic environment. Expect to tackle challenging problems that require both technical acumen and creative problem-solving skills, ensuring that your contributions have a visible and significant impact on the organization.

The Celebal Technologies interview process

Celebal Technologies candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Technical Screening

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Technical Interviews

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

HR Interview

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Where candidates lose points

Randomising an experiment at the user level when users share an account

Two problems fire at once. Colleagues in one workspace see each other's work and talk to each other, so a treated user changes the behaviour of a control user in the same account, which violates the no-interference assumption and biases the estimate toward zero. Separately, outcomes within an account are strongly correlated, so the effective sample size is roughly n / (1 + (m - 1) * rho) for m users per account and intra-class correlation rho, not n. With rho around 0.3 and twenty users per account that is a design effect near 6.7, meaning a user-level confidence interval is about two and a half times narrower than it should be and results cross significance thresholds on noise alone. Randomise the account and cluster the standard errors.

Reading consumption metrics before the metering lag window has closed

Usage pipelines land late and correct themselves, which is exactly what is_restated and restated_at record. A dashboard queried on day T sees a partially populated tail for the last several days, so the most recent points always slope downward and always look like a regression. Analysts then explain the artefact, and sometimes ship a change to fix it. Establish the empirical settling time by measuring how much a given usage_date's total moves between first_written_at and its final value, exclude that many trailing days from every reportable figure, and never compare a fresh period against a settled one.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Analysing at a different unit than the one randomised

Say out loud what was randomised (user, device, account, cluster) and make the analysis unit match, or account for the clustering with cluster-robust standard errors, the delta method, or aggregation up to the randomised unit. Randomising users and then running a test over sessions understates variance and inflates the false-positive rate.

Questions & practice

Describe the bias-variance tradeoff.

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Sanity-check the answer against a simple bound or a simulated case.
Quantify uncertainty explicitly rather than reporting a point estimate alone.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

Explain how a random forest model works.

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Permutation-test a consumption experiment randomised at account level

Hard · permutation-test · heavy-tails · cuped · experiment-readout

An experiment randomised 900 accounts into two arms. You have one row per account: account_id, arm, consumption_28d (billable units after launch) and consumption_pre (the 28 days before). Consumption is heavy-tailed and the largest account is several percent of the total. Write a permutation test from scratch: winsorise at the pooled 99th percentile as a pre-registered rule, use the difference in arm means of the winsorised outcome as the statistic, and obtain a two-sided p-value from 20,000 relabellings of the account-level arm vector. Report the observed effect, the p-value, and the same test on a CUPED-adjusted outcome.

Approach

Be precise about what the permutation test needs. Under the sharp null of no effect for any account, the outcomes are exchangeable across arm labels, and the test is valid for ANY statistic T(outcomes, labels) provided the identical function is applied to the observed labels and to all 20,000 relabellings. The pooled 99th percentile is a function of the outcome vector alone, so recomputing it inside the loop returns the same number 20,000 times: that is wasted CPU, not a bias, and hoisting it out is an optimisation rather than a correctness fix. Say plainly that capping at all changes the estimand from mean consumption to mean capped consumption; it is not a neutral cleaning step.
The mistake that does invalidate the test is an asymmetry between the observed statistic and the permuted ones, and the easiest way to create it is to derive the cleaning rule from the observed arm labels and then freeze it — winsorise each arm at its own observed 99th percentile, hold those two caps fixed, and permute. The observed value is then computed with caps matched to its own partition while every relabelling is scored with caps belonging to a different one, so the null distribution no longer answers the question the p-value claims to answer. A per-arm cap recomputed consistently inside every permutation is a valid test, but it estimates a contrast whose two sides are capped at different thresholds, so prefer the pooled cap on estimand grounds and pre-register it.
Permute the account-level arm vector, because the account is the randomisation unit. Relabelling anything finer — users, workspaces, requests — generates a null distribution narrower than the design actually supports and returns p-values that are anti-conservative.
Vectorise the null: tile the treatment indicator into a (B, n) matrix and permute along axis 1 with rng.permuted(..., out=...). The statistic is a difference of means, so the treated sum alone determines it and the whole null is one matrix-vector product. Use the two-sided p-value (1 + count(|stat_perm| >= |stat_obs|)) / (B + 1); the plus-one on each side is not cosmetic, it keeps the p-value away from exactly zero and keeps the test valid at finite B.
For CUPED, fit theta = cov(y, x) / var(x) on the pooled data and use that same theta for the observed statistic and every relabelling. Pooled theta, like the pooled cap, carries no label information, so where in the loop you compute it is again only a performance question; fitting theta within arms is what goes wrong, because the adjusted outcome then depends on the labels and an observed-label fit frozen across all 20,000 relabellings breaks the match between observed and permuted statistics. x must be measured entirely before launch, which consumption_pre is. Expected variance reduction is about 1 - corr(y, x)^2; measure the achieved reduction from the two null distributions rather than asserting it.

Worked solution (45 min)

cap_y = np.quantile(df.consumption_28d, 0.99); y = np.minimum(df.consumption_28d.to_numpy(float), cap_y); cap_x = np.quantile(df.consumption_pre, 0.99); x = np.minimum(df.consumption_pre.to_numpy(float), cap_x)
t = (df.arm == 'treatment').to_numpy(); n1 = int(t.sum()); n0 = len(t) - n1; obs = y[t].mean() - y[~t].mean()
rng = np.random.default_rng(11); L = np.tile(t.astype(np.int8), (20_000, 1)); rng.permuted(L, axis=1, out=L); s1 = L @ y; stats = s1/n1 - (y.sum() - s1)/n0
p = (1 + int(np.sum(np.abs(stats) >= abs(obs)))) / (20_000 + 1)
theta = np.cov(y, x, ddof=1)[0,1] / np.var(x, ddof=1); y_adj = y - theta*(x - x.mean()); repeat steps 2 to 4 on y_adj and compare stats.std(ddof=1) between the two runs.

Expected result: A two-sided p-value strictly between 1/20001 and 1, an observed effect expressed in capped billable units per account, and a CUPED null whose standard deviation is smaller than the unadjusted one by roughly sqrt(1 - corr(y, x)^2). The CUPED point estimate stays close to the unadjusted one, since the adjustment removes variance rather than shifting the effect.

Verify it yourself

The permutation null is centred near zero: |mean(stats)| is a small fraction of std(stats).
An A/A run, relabelling arm at random before the test and repeating over seeds, gives roughly uniform p-values with about 5% below 0.05.
std(stats_cuped) < std(stats) whenever corr(y, x) is materially non-zero, and the ratio matches sqrt(1 - r^2) to within simulation noise.
Moving the pooled cap computation inside the permutation loop reproduces the same p-value bit for bit at the same seed — the direct demonstration that a pooled cap carries no label information — while setting the cap to infinity reproduces the uncapped test exactly, so the winsorisation switch is independently verifiable.

Follow-up

The p-value is 0.04 with the cap and 0.31 without it. What do you report, and what did you pre-register?
Colleagues in a shared workspace can see the treated behaviour. How does that change the design and the estimate?
How many accounts would you need to detect a 5% lift given this outcome's distribution?

Write a Python function to find the longest common prefix in a list of strings.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Write a function to calculate the Fibonacci sequence.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Seven-day activation rate by signup cohort week

Easy · funnel · cohorts · censoring

dim_account carries account_id, created_at, is_internal and is_current; fct_api_request carries account_id, request_at, http_status, api_key_id and traffic_class. Build a signup cohort by the ISO week of created_at over accounts with is_internal = false. An account counts as activated when it issues a request with http_status < 400, a non-null api_key_id and traffic_class <> 'synthetic_monitor' within 168 hours of its own created_at. Return cohort accounts, activated accounts and the rate per week, and exclude any week that has not yet fully elapsed its 168-hour window.

Approach

Collapse dim_account to one row per account_id before anything else. It is a type 2 dimension, so a plan or status change gives the same account several rows; filtering to is_current = true is the cheapest correct choice here because created_at does not change across versions.
Express the window as interval arithmetic on the timestamptz column: request_at >= created_at AND request_at < created_at + interval '168 hours'. A date-difference of 7 days is a different and wrong condition for accounts created mid-day.
Test activation with EXISTS rather than a join to MIN(request_at). EXISTS short-circuits, keeps the cohort at one row per account, and cannot fan out.
Aggregate by date_trunc('week', created_at AT TIME ZONE 'UTC'), counting accounts and activated accounts, and divide as a ratio of counts.
Drop unreportable weeks: the last account in a cohort week is created just under week_start + 7 days, so the week is only complete once now() >= week_start + interval '14 days'. Without that filter the newest week always looks like a regression.

Worked solution (20 min)

Write the cohort CTE: SELECT account_id, created_at FROM dim_account WHERE is_current AND NOT is_internal AND created_at >= <window start>.
Add the activation predicate as a correlated EXISTS over fct_api_request on account_id with the four conditions: http_status < 400, api_key_id IS NOT NULL, traffic_class <> 'synthetic_monitor', and the 168-hour bracket.
Group by date_trunc('week', created_at AT TIME ZONE 'UTC'); select count() AS cohort_accounts, count() FILTER (WHERE activated) AS activated_accounts, and the ratio cast to numeric.
Add HAVING or an outer WHERE that keeps only weeks where week_start + interval '14 days' <= now().
Spot-check one account that activated on hour 167 and one that activated on hour 169 to confirm the boundary is exclusive at the top.

Expected result: One row per ISO cohort week with cohort_accounts, activated_accounts and activation_rate between 0 and 1; the newest row is at least 14 days old, and accounts with no rows at all in fct_api_request appear in the denominator with activated = 0 rather than disappearing.

Verify it yourself

Summing cohort_accounts across weeks equals the count of non-internal current-version accounts created in the same window.
Re-running the query a day later leaves every previously reported week byte-identical; any movement means the censoring filter is wrong.
activated_accounts is never greater than cohort_accounts, and the count of distinct account_id in the activated set matches a separate COUNT DISTINCT over the fact table.

Follow-up

The median time-to-first-successful-call is more informative than a fixed-window rate. Why can you not compute it from this query, and what estimator does it need?
How would you separate accounts that never called from accounts that called and got only 4xx responses, and which of those is a product problem?

How would you approach a project to predict customer churn?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

What steps would you take to clean and preprocess data before analysis?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you evaluate the success of a new feature in a product?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you prioritize your tasks when working on multiple projects?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Pick the randomisation unit when workspaces and queues are shared

Medium · interference · randomisation-unit · clustering · switchback

A collaboration feature was tested by randomising 18,000 users 50/50. dim_user_membership shows a median of 9 activated, non-service users per paying account, and weekly successful interactive requests per user carries an intra-class correlation of 0.25 within an account. The reported lift is 4% at p = 0.05. Separately, the same team wants to evaluate a change to the admission-control queue, which is shared by every account in a region. Give the correct randomisation unit for each test, the effective sample size and corrected p-value for the first, and the design for the second.

Approach

Separate the two failures of user-level randomisation, because they need different fixes and only one of them is a variance problem. Interference: a treated colleague changes a control colleague's behaviour inside a shared workspace, which biases the estimate toward zero. Correlated outcomes: users inside an account are not independent draws, which understates the variance. Clustering the standard errors fixes the second and leaves the first entirely intact.
Compute the design effect: 1 + (m - 1) * rho = 1 + 8 * 0.25 = 3.0. Effective sample is 18,000 / 3 = 6,000 users, corresponding to roughly 2,000 accounts. Standard errors are understated by sqrt(3) = 1.73, so a reported t of 1.96 is really 1.13 and the honest two-sided p is about 0.26, not 0.05.
Re-run the first test randomised on account_id with inference clustered on account_id, or equivalently as a regression on account-level means. The account-means version is the conservative one and is usually easier to defend to a non-specialist audience, at the cost of some efficiency when cluster sizes are very unequal.
For the queue change, account randomisation fails too, and for a different reason: treated and control accounts contend for the same finite capacity, so the control arm is mechanically affected by the treated arm and the contrast estimates a property of a mixed system rather than the effect of the policy. Randomise time instead, with a switchback alternating the admission policy at the region-by-30-minute-slot level.
Specify the switchback so it is actually valid. Impose a washout at the start of each slot at least as long as the p99 queue drain time and discard requests enqueued before the switch; randomise slot order rather than strictly alternating, because a fixed alternation aliases with the hourly and weekday traffic cycle; block on hour-of-day so both policies see peak and trough; and cluster inference on the slot, which is the unit that was randomised.
State the power consequence plainly: the cluster count is slots, not requests. Fourteen days of 30-minute slots gives 672 slots, and the variance to plan against is between-slot variance in the outcome, which is far larger than between-request variance and is what makes switchbacks expensive.

Worked solution (30 min)

Compute the design effect 1 + (9 - 1) * 0.25 = 3.0 and the effective sample 18,000 / 3 = 6,000 users, about 2,000 accounts.
Inflate the standard error by sqrt(3) = 1.732, convert the reported t of 1.96 to 1.13, and read off a two-sided p of about 0.26.
Recompute the same contrast on account-level means as an independent check that does not depend on the pilot's rho estimate.
Lay out the switchback: region-by-30-minute slots, randomised order, washout equal to the p99 drain time, blocking on hour-of-day, inference clustered on slot, 672 slots over fourteen days.

Expected result: Design effect 3.0, effective sample about 6,000 users or 2,000 accounts, corrected two-sided p about 0.26. The 4% lift is not significant once clustering is respected and is additionally biased toward zero by within-workspace spillover. The queue change is run as a randomised-order switchback with slot-level clustering and a drain-time washout, not as an account-level split.

Verify it yourself

The design effect depends only on cluster size and intra-class correlation: it must collapse to 1 when rho = 0 and when m = 1. If your expression does not, it is not the design effect.
An account-means regression and a user-level regression with cluster-robust standard errors should give similar standard errors. A large gap indicates very unequal cluster sizes, in which case the means version needs weighting.
The switchback's sample size is slots, not requests. If your power calculation cites a sample in the millions, you have counted requests and will badly overstate precision.

Follow-up

The intra-class correlation was estimated at 0.25 from a 300-account pilot. If it is really 0.40 the design effect becomes 4.2. How do you plan under that uncertainty rather than betting on the point estimate?
Under what conditions is user-level randomisation still the right choice even though users share accounts?
The queue change is expected to leave a four-hour retry backlog. What does carryover of that length do to a 30-minute switchback, and what would you run instead?

Activation drops six points starting at a deploy hour

Medium · instrumentation · release analysis · metric inputs

Seven-day activation, defined as an account's first request with http_status < 400, api_key_id not null and traffic_class <> 'synthetic_monitor' within 168 hours of created_at, fell six points for sign-up cohorts after a Tuesday. A client SDK major version shipped that morning. From fct_api_request (account_id, api_key_id, sdk_name, sdk_version, http_status, traffic_class, request_at) and dim_account (account_id, created_at, is_internal), decide whether activation actually fell or the metric's inputs changed, and state in advance what evidence would convince you of each.

Approach

Decompose the definition and recompute activation under each relaxation: status only, status plus traffic_class, then the full definition. If the entire drop lives in the api_key_id clause, this is an instrumentation question rather than a behavioural one.
Measure the null rate of api_key_id by sdk_version and by hour. A stamping change appears as a step at the deploy boundary confined to the new version; a behavioural change appears as a ramp that grows with adoption and leaves old-version traffic untouched.
Hold the cohort's SDK mix fixed before comparing. New sign-ups adopt the newest version first, so a cohort-level drop can be pure composition even when no individual version changed at all.
Corroborate with a source the release did not touch: whether the same cohorts appear in fct_usage_daily with non-zero billable_quantity, and whether their fct_support_ticket rows with category = 'onboarding' rose.
Write the decision rule down before looking at the answer. An instrumentation artefact predicts unchanged downstream usage and a version-confined null step; a real regression predicts falling downstream usage and more onboarding tickets in the same cohorts.

Follow-up

Old-version and new-version populations are not exchangeable, because new accounts adopt the new version first. How would you build a comparison that is not confounded by cohort age?
What backfill or metric-versioning policy keeps the historical series interpretable once you fix the stamping?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Bring numbers, including the ones you are unsure of

An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.

Give an example of how you’ve worked effectively in a team.

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.
Close with what you would do differently, concretely.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Allocate one analyst week across three competing requests

Easy · prioritisation · stakeholders · scoping

Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.

Approach

The interviewer is probing whether you prioritise on decision timing and reversibility or on who asked most forcefully. Sort by the date each decision is actually taken and by what the default outcome is if nothing arrives.
Apply that sort concretely. The Thursday readout has a hard irreversible deadline and no value afterwards. The pricing review has three weeks of slack. The renewal list has a rolling deadline set by term_end_date, so part of it is urgent this week and the rest is not, which means it can be split rather than deferred whole.
Find the cheapest sufficient version of each request rather than the full version. The readout goes in full. The renewal list ships as a filtered query over renewal-eligible accounts ranked by two inspectable signals rather than as a model. The margin work is scoped to the accounts that dominate the pricing decision, since revenue is heavily skewed and the tail will not change the conclusion.
Make the trade visible in one written note to all three at once, with dates. Telling each person separately that they are the priority is how an allocation becomes a credibility problem.
Refuse something explicitly and say why. The model version of the renewal list is the usual candidate, because it cannot be evaluated without a holdout nobody has agreed to yet, and building it this week forecloses that.
Leave slack. A plan with none is a plan to miss the one deadline that cannot move.

Follow-up

The sales leader escalates to your manager. What did you already do that makes that a short conversation?
Which of the three deadlines would you push back on, and what exactly would you ask for?
What would you change about how these requests reach you so next week is not the same?

Walk through an analysis you later discovered was wrong

Easy · data quality · error ownership · metering

Six weeks ago you reported that consumption fell 9 percent in the last week of the month, and a team spent a sprint investigating the cause. The fall was an artefact: rows in fct_usage_daily land late and are restated in place, and you queried before the tail had settled. Describe how you found the error, what you told the people who acted on it, and the control you put in place so this class of mistake cannot reach a dashboard again. Be specific about how the settling window was measured.

Approach

The interviewer is probing whether you self-report errors before someone else finds them, and whether your fix is structural rather than a promise to be more careful. Say plainly that the number was wrong and that a sprint was spent on it, before describing any diagnosis.
Establish the artefact quantitatively instead of asserting that data lands late. For each usage_date, compare the total as of first_written_at against the settled total and read the settling time off that curve, for example 97 percent of final by day three and 99.5 percent by day five.
Correct the record the same day, in the channel the original number went out in, to the same audience. The cost of the wasted sprint belongs in the correction, not in a footnote.
Make the fix structural: exclude a trailing lag window from every reportable figure, and make the reporting view return no rows inside that window rather than returning partial ones. A dashboard that shades unsettled days still gets read as a decline.
State what generalises. Any fact table restated in place has this failure mode, so the guard belongs at the source rather than on the one dashboard that embarrassed you. A strong answer ends with the class of error closed; a generic one ends with a lesson learned.

Follow-up

How did you choose the completeness threshold behind the lag window, and what would make you recalibrate it?
What did you say to the team that lost the sprint, and what did they say back?
Is there a legitimate case for showing the unsettled tail at all, and to whom?