Interview guides · Alight Solutions

Alight Solutions Data Scientist Interview Questions & Guide 2026

Data scientist prep for Alight Solutions: practice defining a consulting and client services metric, picking its guardrails and defending the tradeoff.

The 60-second brief

A Data Scientist at Alight Solutions serves as a vital bridge between complex data architecture and actionable business strategy. In an organization dedicated to cloud-based human capital and financial solutions, your work directly influences how millions of users navigate their benefits, health, and retirement options. You are not just building models; you are solving high-stakes problems that impact the financial well-being and healthcare outcomes of a massive, diverse user base.

How much statistics you need depends on the flavour of the seat. Experiment-facing work wants you deep enough to notice that repeated looks at accumulating data inflate the false positive rate of a fixed-sample test; modelling-facing work wants estimation and honest uncertainty intervals.

Focus areas

Reconstruct pipeline stage history from mutable rows
Segment margin by pricing model before comparing
Cluster inference at the account, not the engagement

Overview

The role is inherently product-focused, requiring you to translate ambiguous business challenges into structured analytical frameworks. Whether you are analyzing engagement patterns in healthcare navigation or optimizing value engineering models, you will work cross-functionally to ensure that data-driven insights are woven into the fabric of Alight Solutions products. It is a position of significant influence, where your ability to communicate complex findings to non-technical stakeholders is just as critical as your technical proficiency.

The Alight Solutions interview process

Alight Solutions candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Video Interviews

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Where candidates lose points

Treating accounts as independent observations

Revenue is concentrated: a small number of client_ids typically carries a large share of fees, and engagements within one account share a partner, a rate card and a delivery team. Ordinary standard errors computed over engagements therefore understate uncertainty badly. Cluster at client_id, and with fewer than roughly 40 clusters use a wild cluster bootstrap or a CR2 correction, because cluster-robust standard errors are downward-biased in that regime and will manufacture significance that a replication will not reproduce.

Trending utilisation or revenue on work_date without accounting for timesheet backfill

Time entries are created days to weeks after the work happens, and the backfill tail often runs two to six weeks. A dashboard keyed on work_date therefore shows the most recent weeks as a decline that reverses on every refresh. The fix is either to hold the reporting window back past the observed backfill tail (measure the tail with the timesheet submission lag metric rather than guessing) or to report an as-of-entered_at snapshot so the series is internally consistent, and to state which one you used.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Crediting a treatment for regression to the mean

Selecting a group because it is extreme (lowest-engagement users, accounts having their worst month, the bottom decile of a score) moves that group's expected next-period value back toward the average even under no treatment, by exactly as much as the selecting measure is imperfectly correlated with its own later value. Compare against units that met the same selection rule and went untreated, or use two pre-periods so the bounce-back is visible before the intervention starts. A pre-post number on a group chosen for being extreme measures the selection rule, not the treatment.

Questions & practice

Kaplan-Meier days to payment with unpaid invoices censored

Hard · survival analysis · censoring · kaplan-meier

invoice_lines has invoice_line_id, engagement_id, line_type, issued_at, due_date, paid_at (null when unpaid), amount_usd and status in draft, issued, partially_paid, paid, disputed, written_off. At a given snapshot_date, estimate the median days from issue to full payment. Implement Kaplan-Meier yourself; no lifelines or equivalent. Treat issued, partially_paid and disputed as right-censored at snapshot_date, and decide and justify what to do with written_off. Report the naive mean over paid lines alongside your estimate and state the sign of its bias.

Approach

Build the duration and event table explicitly. Drop draft lines, which have no clock. For status paid, duration = (paid_at - issued_at).days with event = 1. For issued, partially_paid and disputed, duration = (snapshot_date - issued_at).days with event = 0.
Handle written_off as a competing event rather than a censor. Censoring it makes the estimator answer 'time to payment if written-off invoices could still pay', which overstates collection. Either report a cumulative-incidence version alongside, or censor them and say plainly that the result is conditional on eventual collection.
Implement the estimator directly: sort unique event times, at each t take n_i as the count with duration >= t and d_i as the payments at exactly t, and accumulate S(t) = product of (1 - d_i / n_i). Censored rows leave the risk set without causing a drop, which is the whole mechanism and the reason the answer differs from any completed-case average.
Read the median as min{t : S(t) <= 0.5}. If S never reaches 0.5 within observed follow-up, report 'not reached'; interpolating past the last observation invents data that the snapshot does not contain.
Add Greenwood's formula for Var(S(t)) to put a band on the curve, then invert the band at 0.5 for an interval on the median rather than quoting a point estimate alone.
Compare against the mean over paid lines only and name the direction: at any snapshot the paid set over-represents fast payers, so the naive mean is biased low, and the bias widens exactly when collections deteriorate.

Worked solution (40 min)

Assemble a frame of (duration_days, event) per non-draft line, with snapshot-based durations for the three open statuses and an explicit, documented rule for written_off.
times = np.sort(df.loc[df.event == 1, 'duration_days'].unique()); for each t, n = (df.duration_days >= t).sum() and d = ((df.duration_days == t) & df.event).sum().
S = np.cumprod(1 - d / n) aligned to times; prepend S(0) = 1.0 so the curve starts where it should.
median = times[np.argmax(S <= 0.5)] if (S <= 0.5).any() else 'not reached'; compute Greenwood variance as S(t)^2 * cumsum(d / (n * (n - d))).
naive = df.loc[df.event == 1, 'duration_days'].mean(); report both, plus the censored share, since the gap between the two scales with it.

Expected result: A step function starting at S(0) = 1.0 that drops only at observed payment days, an integer median or an explicit 'not reached', a Greenwood-based interval, and a naive completed-case mean that sits below the Kaplan-Meier median on data with material censoring.

Verify it yourself

S is non-increasing, starts at 1.0, and never goes below 0; the number at risk at the first event time equals the count of all non-draft lines.
Re-running with every censored row deleted makes the Kaplan-Meier median equal the empirical median of paid durations, since with no censoring the estimator reduces to the empirical survival function.
A six-row toy example worked by hand reproduces the implementation's S values exactly.
The gap between the naive mean and the Kaplan-Meier median grows when you move snapshot_date earlier, which raises the censored share.

Follow-up

A large account moved to a monthly payment run. Is administrative censoring still independent of payment time, and what would you check?
Finance wants one DSO number against a target. What do you give them, and what do you refuse to give them?
Stratify by line_type. Do milestone lines behave like fees lines, and what would it mean for the firm if they do not?

Simulate the chance a capped engagement crosses its cap

Medium · monte carlo · bootstrap resampling · pricing caps

A capped time-and-materials engagement has not_to_exceed_usd = 400000, has billed 250000 to date, and bills at a blended bill_rate_usd of 250. Fourteen delivery weeks remain before planned_end_date. You have that engagement's last twenty weekly totals of approved billable hours as a pandas Series. Estimate the probability that cumulative billable value crosses the cap before the planned end, and the expected unbillable hours if it does. Resample the observed weeks; do not assume normality. Report a Monte Carlo standard error and justify the number of draws.

Approach

Reduce the deterministic part first: the cap still affords (400000 - 250000) / 250 = 600 hours, so the whole question is the distribution of the sum of fourteen future weekly hour totals against a fixed threshold of 600.
Before resampling, check the twenty observed weeks for trend and lag-1 autocorrelation. Independent resampling is defensible only if the weeks are exchangeable; on a ramping engagement use a moving-block bootstrap or model the ramp, because i.i.d. draws understate the upper tail, which is exactly the tail the question is about.
Draw one (B, 14) array with np.random.default_rng().choice(observed, size=(B,14), replace=True), sum along axis 1, and compute p_hat = mean(total > 600) in one vectorised pass.
Report expected unbillable hours two ways: unconditional mean(maximum(total - 600, 0)) for expected loss, and the mean conditional on a breach for how bad a breach is when it happens. The second is the number that drives a change-order conversation.
Attach se(p_hat) = sqrt(p_hat(1 - p_hat)/B) and pick B from the precision you need: plus or minus one point at 95% confidence needs about 1.96^2 * 0.25 / 0.01^2, roughly 9600 draws at the worst case p = 0.5.
State the assumptions that would flip the answer: constant blended rate, no scope change, no holiday weeks inside the fourteen, and a cap that applies to fees rather than to fees plus expenses.

Follow-up

Hours past the cap still cost money. Restate the result as expected gross margin rather than a probability.
Two weeks of time are entered but not yet approved. How do you fold them in without double-counting?
A change order that raises the cap is judged 60% likely. How does that change the number you present, and to whom?

Collapse time entries into contiguous staffing spells

Medium · sessionisation · gap and island · vectorisation

From approved delivery time entries (consultant_id, engagement_id, work_date, hours, charge_code, status, with engagement_id not null), build staffing spells. Per consultant and engagement, collapse weeks containing any logged hours into contiguous runs, where three or more consecutive zero-hour weeks end a spell. Output one row per spell with consultant_id, engagement_id, start_week, end_week, active_weeks, gap_weeks and total_hours. Consultants sit on several engagements at once, so spells from different engagements may overlap in time and must not be merged. No Python loop over rows.

Approach

Aggregate to (consultant_id, engagement_id, week_start) with summed hours and keep only weeks with positive hours. The absent weeks are the signal, so materialising zeros here would destroy the thing you are detecting.
Convert week_start into an integer week index, ((week_start - epoch_monday).dt.days // 7), so gap detection is integer subtraction rather than calendar arithmetic that breaks over month and year boundaries.
Sort by [consultant_id, engagement_id, week_index], diff the week index inside each pair, and mark a spell start where the diff is null (first row of the pair) or greater than 3. A diff of 1 is adjacent weeks and a diff of 3 is two empty weeks, which the tolerance permits.
Take spell_id = the cumulative sum of that boolean over the whole frame so ids are globally unique, then a single groupby on [consultant_id, engagement_id, spell_id] yields min and max week, active week count and summed hours; gap_weeks = (end - start + 1) - active_weeks.
Do not deduplicate overlapping spells across engagements. A consultant on two engagements in the same week is the normal case, and that overlap is the fact any capacity or context-switching question needs.

Follow-up

Re-run with a one-week and a four-week tolerance. What happens to the spell count, and which tolerance would you defend to a staffing lead?
Using these spells, how would you measure how many engagements a consultant is split across in a given week, and why is that not just a count of rows?

Explain the difference between various join types and when you would use each in a production environment.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Write a query using SQL window functions to calculate a rolling average of user interactions over the last 30 days.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Cost hours at the rate in force on the work date

Medium · type-2 scd · effective dating · date boundaries

dim_consultant is a type-2 dimension with consultant_sk, consultant_id, level, practice_area, standard_cost_rate_usd, effective_from TIMESTAMPTZ, effective_to TIMESTAMPTZ NULL and is_current. fct_time_entry holds consultant_id, work_date DATE, hours, status and a cost_rate_usd snapshotted when the row was written. Recompute last year's delivery cost per practice_area by joining each approved entry to the dimension row valid on its work_date, then reconcile against the snapshotted rate and list entries where the two differ by more than one cent. Say what a session timezone change does to your join predicate.

Approach

Join on a half-open interval: te.work_date >= d.effective_from AND (d.effective_to IS NULL OR te.work_date < d.effective_to). This assumes effective_to holds the next row's effective_from. Verify that on one consultant with three versions before committing, because if effective_to instead stores the last valid instant the upper bound must be inclusive, and getting it backwards either double counts or drops a boundary day.
Refuse is_current = TRUE as a join predicate. It prices last year's hours at today's rate, so a promotion or an annual rate change silently restates history, and the restatement is invisible in the output.
Be explicit about the DATE to TIMESTAMPTZ comparison. Postgres casts work_date to midnight in the session TimeZone while effective_from is an absolute instant, so a version that opens at 2026-03-01 00:00 UTC lands on one side of a 2026-03-01 work_date under a UTC session and the other side under a negative-offset session. Normalise both to one convention, for example comparing work_date against (effective_from AT TIME ZONE 'UTC')::date, and write down which convention the warehouse uses.
Validate one-to-one before trusting any sum: group the joined result by time_entry_id and assert no id appears twice. Overlapping validity intervals in the dimension are a real defect and present as inflated cost, not as an error message.
Reconcile the recomputed rate against the snapshotted cost_rate_usd. Differences are either late-corrected dimension rows or entries snapshotted at the wrong moment; both are findings with a named cause, not rounding noise to be smoothed over.

Worked solution (30 min)

Count approved time entries in the window; this is the exact row count the join must preserve.
Write the half-open interval join and immediately compare the post-join row count against that number.
Check for duplicates by grouping on time_entry_id and, if any appear, inspect that consultant's dimension rows for overlap.
Aggregate cost by practice_area using the dimension rate, then again using the snapshotted rate, and diff the two totals.
List the individual entries whose two rates differ by more than a cent, with both rates and the dimension row's validity interval.

Expected result: A cost total per practice_area plus an exception list. The joined row count equals the approved-entry count exactly, and the two cost totals agree except for the entries on the exception list.

Verify it yourself

COUNT() after the join equals COUNT() before it.
No time_entry_id appears more than once in the joined set.
Recomputed total cost is within the exception list's aggregate difference of the snapshot-based total.
Re-running after SET TIME ZONE to a non-UTC zone assigns every entry to the same dimension row, or the query documents exactly where it does not.

Follow-up

How would you detect overlapping or gapped validity intervals across the whole dimension in one query?
Which of the two rates belongs in a client-facing margin number, and which in an internal cost review?
A consultant changes practice_area mid-engagement. Which practice_area owns the hours, and what breaks either way?

How do you prioritize your work when you have multiple competing requests from different product teams?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you balance trade-offs between short-term conversion metrics and long-term user satisfaction?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

What product metrics would you track to measure the effectiveness of a personalized benefits recommendation engine?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you design a success metric for a new healthcare navigation feature?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How do you determine the required sample size for an A/B test to ensure statistical significance?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

What are the most common experimentation pitfalls you have encountered, and how do you mitigate them?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

Cut variance with pre-period margin before buying more accounts

Medium · cuped · variance reduction · stratification · ancova

You are designing an account-randomised pilot of a delivery-review process, measured on engagement gross margin over one quarter. The design is underpowered at the account count available. Each account has two prior quarters of margin history reconstructible from fct_invoice_line and fct_time_entry, correlated 0.6 with the pilot-quarter outcome. Specify a variance-reduction plan: the estimator, the coefficient it uses, how you handle accounts whose first engagement starts inside the pilot window, and the sample-size equivalent of the reduction you expect. State the precondition that makes the adjustment safe.

Approach

Define the adjusted outcome Y* = Y - theta * (X - Xbar) with X the pre-period margin and theta = Cov(Y, X) / Var(X). The precondition is that X is measured strictly before assignment, so its arm means agree in expectation and subtracting it removes the account-level variance component without touching the treatment contrast.
State the payoff exactly: Var(Y*) = Var(Y) * (1 - rho^2). At rho = 0.6 that is a 36% variance reduction, a 20% standard-error reduction since sqrt(0.64) = 0.8, and a sample-size equivalent of 1 / (1 - rho^2) = 1.56 times the accounts. Real, but it will not rescue a design that is five times short.
Implement it as the ANCOVA regression of Y on treatment and X, which is numerically the same estimator when theta is the OLS coefficient and which hands you cluster-robust inference for free. Estimate theta on pooled data, never separately per arm, which reintroduces bias.
Keep accounts with no history rather than dropping them: impute X at the sample mean and add a no-history indicator. Because X is pre-treatment, imputation cannot bias the treatment coefficient, while dropping those accounts silently changes the estimand to accounts with history.
Balance at assignment as well as adjusting after it. Block on account_tier and pre-period margin tercile, randomise within block, and carry block fixed effects into the model. At roughly 120 accounts, blocking guarantees balance where post-hoc adjustment only achieves it on average.

Worked solution (25 min)

Build the account-quarter margin panel: fees from fct_invoice_line where line_type in (fees, milestone, credit_note), cost as approved fct_time_entry hours times cost_rate_usd, rolled up through parent_client_id.
Estimate rho between the pre-period window and a historical quarter that stands in for the pilot, and confirm it is near the assumed 0.6 rather than assuming it.
Fit Y on treatment, X and block fixed effects with standard errors clustered at client_id; read theta off the X coefficient.
Compare the standard error on the treatment coefficient with and without X in the model.
Recompute the MDE from the reduced standard error and test it against the decision threshold before committing to the pilot.

Expected result: At rho = 0.6 the adjusted standard error should come in about 20% below the unadjusted one, equivalent to 1.56 times the accounts, and the two treatment-effect point estimates should agree within about one standard error since both are unbiased.

Verify it yourself

The treatment coefficient with and without the covariate must be close; a large shift means X is imbalanced across arms, which is either a randomisation failure or a covariate that is not actually pre-treatment.
Realised variance reduction measured on the pilot data should be near 1 - rho^2; a much smaller reduction means rho was estimated on a period that does not resemble the pilot.
Dropping the no-history accounts entirely should move the point estimate only within noise; a large move means those accounts differ systematically and the estimand has changed.

Follow-up

Why is pre-period outcome usually a better covariate than account_tier alone, and when does that ranking flip?
A colleague wants to add hours worked during the pilot quarter as a second covariate. What happens to the estimate?
Your pre-period window is two quarters. What sets the optimal length, and what degrades if you extend it to eight?

Firm margin rose while every pricing model's margin fell

Easy · mix shift · margin · decomposition

Firm-wide engagement gross margin rose from 31% to 34% quarter over quarter. Split by fct_engagement.pricing_model, margin fell inside time_and_materials, fixed_fee, retainer and outcome_based. Sources are fct_invoice_line (engagement_id, line_type, amount_usd, period_start_date, period_end_date) and fct_time_entry (engagement_id, hours, cost_rate_usd, status). Explain the aggregate move, quantify how much of the 3-point change is mix and how much is within-model, and state which number you put in front of practice leadership and why.

Approach

Write the aggregate as a weighted mean: M = sum over pricing models p of w_p * m_p, where m_p is margin within p and w_p is p's share of the fee base. The weights must be the fee base, because margin is a fees-weighted ratio; weighting by engagement count answers a different question.
Apply the exact three-term decomposition: dM = sum (w_p1 - w_p0) * m_p0 + sum w_p0 * (m_p1 - m_p0) + sum (dw_p)(dm_p). Verify the three components sum to the reported +3.0 points before interpreting any of them.
Identify which weights moved and in which direction; the aggregate can only rise this way if fee share shifted toward the structurally higher-margin models, typically fixed_fee delivered under budget or retainer.
Guard against composition inside each stratum: repeat the same decomposition within pricing_model by practice_area and account_tier, so a within-model decline is not itself an unexamined mix effect.
Confirm both sides of the ratio are on the same clock and the same scope: numerator includes line_type IN ('fees','milestone','credit_note') with credits signed negative; the cost side includes all approved delivery hours, billable and non-billable alike.
Report within-model margin as the headline delivery signal and mix as a separate, named line with its own owner, since mix is a sales decision and margin is a delivery one.

Follow-up

Next quarter the mix reverts. What happens to the aggregate, and what will you have already told leadership so this is not a surprise?
Which of the four pricing models should not be pooled with the others even for a mix-adjusted number, and why?
How would you present this if the mix shift were deliberate strategy rather than accident?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

What the behavioural round is actually scoring

Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.

What are some of the most challenging data quality or integration issues you have dealt with in your career?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Your utilisation dashboard caused a staffing decision on incomplete data

Medium · late-arriving data · postmortem · reporting windows

Six weeks ago you shipped a weekly billable-utilisation dashboard keyed on fct_time_entry.work_date. It showed a six-point drop across the three most recent weeks. A practice lead pulled two consultants off an engagement in response. The drop reversed on the next refresh, because time entries are created days to weeks after the work happens and entered_at trails work_date. Describe what you do now: the diagnosis, the change to the artifact, and the conversation with the person who acted on your number.

Approach

Name the probe: whether you own a reporting-design error rather than reclassifying it as someone else's timesheet compliance problem, and whether you fix the class of bug instead of the single week.
Quantify before explaining. Measure the backfill curve directly from entered_at: for work_date D, the share of final hours that existed as of D plus k, for k from 1 to 45. That gives an observed tail length instead of a guessed cutoff.
Change the artifact so the incomplete region cannot be read as a trend. Either end the trended series at snapshot minus the measured tail, or publish an as-of-entered_at series that is internally consistent, and label which one is on screen.
Tell the person who acted, first and directly, with the corrected series and the specific decision to revisit. A correction that arrives after they notice costs more than the original error.
Add a standing completeness tile: timesheet submission lag, the share of entries where entered_at date minus work_date exceeds seven days, so the dashboard shows its own reliability rather than depending on you remembering.

Follow-up

Leadership still wants to see the current week. What do you show, and how do you label it?
One practice runs a three-day lag and another twenty days. Do you set one firm-wide cutoff or one per practice, and what does that cost in comparability?

Scoping a one-line question about practice profitability in twenty minutes

Medium · scoping · metric definition · exclusions

A practice leader asks, in one line, whether the data_and_ai engagements are profitable. You have fct_engagement, fct_time_entry and fct_invoice_line, and twenty minutes before they travel. Some engagements are retainer or outcome_based, so contracted_hours is NULL; some invoices are unpaid; pursuit hours sit on charge_code = 'client_pursuit' with engagement_id NULL. Produce the scoping conversation: the questions you ask, the metric you commit to in writing, the exclusions you will have to make, and what you deliver by when.

Approach

Name the probe: whether you convert an ambiguous one-liner into a decision with a metric attached, or start querying and return a number nobody can act on.
Ask what changes based on the answer. Repricing the rate card, changing the staffing pyramid, and deciding whether to keep selling fixed_fee are three different questions, and they need different cuts of the same data. Pick the one the leader actually has authority over.
Commit the metric in writing on the spot: fees plus milestone plus credit_note lines from fct_invoice_line as the revenue side, the sum of hours times cost_rate_usd over all approved entries, billable and non-billable alike, as the cost side, inception to date, reported separately by pricing_model.
List what you will not answer and why: overrun on retainer and outcome_based work, because contracted_hours is NULL there and needs an explicit predicate rather than a silent NULL drop; cash collected, because unpaid invoices are censored; and pursuit cost, which has no engagement_id and must either be allocated on a stated rule or reported as a separate line.
Commit two deliverables with dates rather than one vague one: the stratified margin table within two days, the mix decomposition within a week. Name the single assumption that, if wrong, flips the conclusion.

Follow-up

They insist on one number for a partner meeting. Which single number do you give, and what sentence goes with it?
If you had to allocate client_pursuit hours to engagements, what rule would you use and how would you show the answer is not sensitive to it?