Interview guides · Ernst & Young Advisory Services Sdn Bhd

Ernst & Young Advisory Services Sdn Bhd Data Scientist Interview Questions & Guide 2026

SQL sets for data scientist prep ahead of Ernst & Young Advisory Services Sdn Bhd, covering window functions, funnel and retention queries, and the joins that quietly drop the rows you needed.

The 60-second brief

A Data Scientist at Ernst & Young Advisory Services Sdn Bhd plays a pivotal role in transforming data into actionable insights that drive strategic decision-making. Your work will directly influence client outcomes, shaping the way organizations leverage data to mitigate risks, enhance operational efficiency, and innovate solutions. This position is critical not just for the technical expertise it brings, but also for its strategic influence on the firm's advisory capabilities across various sectors.

Coding rounds for this role are usually data-manipulation shaped rather than data-structure shaped: group-bys, joins, time windows, ranking within a partition. Confirm the format before spending a week on graph traversal.

Focus areas

Cluster inference at the account, not the engagement
Separate bookings, recognised revenue and collected cash
Compute utilisation against a defended availability denominator

Overview

In this role, you will engage with complex datasets, employing advanced analytics and machine learning techniques to address client-specific challenges. Whether you are developing predictive models, conducting risk assessments, or creating data visualization tools, your contributions will enhance the overall value proposition of Ernst & Young Advisory Services Sdn Bhd. The work environment is dynamic, filled with opportunities to collaborate with multidisciplinary teams, including consultants, engineers, and business analysts, to deliver comprehensive solutions that meet client needs.

Expect to be at the forefront of innovative projects that leverage cutting-edge technologies, such as artificial intelligence and natural language processing, all while working within a culture that emphasizes continuous learning and professional growth.

The Ernst & Young Advisory Services Sdn Bhd interview process

Ernst & Young Advisory Services Sdn Bhd candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

HR Screening

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Interviews

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Coding Assessments

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Case Studies

This round runs as a working session, so part of what it decides is whether you are useful to think with. The interviewer will interrupt: a hint that the data you assumed does not exist, a challenge to your metric, a nudge toward a branch you skipped. Treating those as interference is the common failure. Reason out loud while your thinking is still provisional so there is something to react to, and when a redirect arrives, take it instead of defending the path you had already started down.

What to demonstrate

Whether your reasoning is audible while it is still unsettled, or only after you have privately decided
What you do with a hint: absorb it and adjust, or argue for the original route
Whether your clarifying questions have answers that would change your approach, as opposed to filling silence
Whether you can be wrong about something in the middle of the case and keep moving without restarting

How to prepare

Run practice cases with a partner instructed to interrupt twice: once to remove a data source you assumed existed, once to reject the metric you chose. Practise absorbing both without going back to the start.
Before each practice case, write down the clarifying questions you plan to ask, then check afterwards whether any answer actually changed what you did. Drop the ones that did not.
Explain an analysis you already know well to someone outside the field and have them stop you at every point where the reasoning jumped a step.

Project Discussions

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Where candidates lose points

Pooling margin, realisation or overrun across pricing models

Fixed-fee margin falls with hours worked; uncapped time-and-materials margin rises with hours worked; retainer margin depends on neither. A quarter in which the firm sells more fixed-fee work will show a margin change caused entirely by mix, not by delivery performance, and the aggregate can move in the opposite direction to every individual pricing model. Always stratify by fct_engagement.pricing_model before comparing periods, and report the mix shift alongside the within-stratum change.

Treating accounts as independent observations

Revenue is concentrated: a small number of client_ids typically carries a large share of fees, and engagements within one account share a partner, a rate card and a delivery team. Ordinary standard errors computed over engagements therefore understate uncertainty badly. Cluster at client_id, and with fewer than roughly 40 clusters use a wild cluster bootstrap or a CR2 correction, because cluster-robust standard errors are downward-biased in that regime and will manufacture significance that a replication will not reproduce.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Questions & practice

What are some common algorithms used in predictive modeling?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Explain the steps involved in feature selection for a machine learning model.

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

Build a dense consultant-week hours spine with zero weeks

Easy · calendar spine · groupby · missing rows

You are given time_entries (time_entry_id, consultant_id, engagement_id, charge_code, work_date as datetime64, hours, is_billable, status) and a list of consultant_ids in scope plus a start and end date. Produce a DataFrame with exactly one row per consultant per Monday-anchored week in that range, with columns consultant_id, week_start, billable_hours, total_hours. Weeks in which a consultant logged nothing must appear as zeros rather than be absent. Count only rows with status == 'approved'. Do not loop over consultants and do not use resample.

Approach

Anchor weeks arithmetically: week_start = work_date - pd.to_timedelta(work_date.dt.weekday, unit='D'). Avoid dt.isocalendar().week, whose numbers restart each January and do not sort across a year boundary.
Construct the spine from the inputs, not from the data: pd.date_range(start, end, freq='W-MON') crossed with the in-scope consultant_ids via pd.MultiIndex.from_product, so its size is fixed before any aggregation runs.
Aggregate approved entries once with a single groupby on [consultant_id, week_start], producing total hours and billable hours (hours.where(is_billable, 0).sum()) in the same pass.
Left-merge the aggregate onto the spine, fillna(0.0) on both hour columns, and keep float dtypes so later ratios do not integer-divide.
Assert the shape before returning: len(out) == n_consultants * n_weeks, and out.total_hours.sum() equals the sum of hours over the filtered input inside the window.

Worked solution (20 min)

Filter to status == 'approved' and to work_date within [start, end], then derive week_start by subtracting the weekday offset.
Build weeks = pd.date_range(start_monday, end, freq='W-MON') and spine = pd.MultiIndex.from_product([consultant_ids, weeks], names=['consultant_id','week_start']).to_frame(index=False).
agg = filtered.assign(billable_hours=filtered.hours.where(filtered.is_billable, 0.0)).groupby(['consultant_id','week_start'], as_index=False)[['hours','billable_hours']].sum().
out = spine.merge(agg, how='left', on=['consultant_id','week_start']).fillna({'hours':0.0,'billable_hours':0.0}).rename(columns={'hours':'total_hours'}).
Sort by consultant_id then week_start and run the two assertions on row count and hour conservation.

Expected result: A frame of exactly len(consultant_ids) x len(weeks) rows, sorted by consultant then week, in which total_hours sums to the approved hours inside the window and zero-hour weeks are present as 0.0 rather than missing.

Verify it yourself

Row count equals n_consultants * n_weeks exactly, for a consultant list that includes someone with no entries at all.
Total hours in the output equal total approved hours in the filtered input, to the cent.
billable_hours <= total_hours on every row.
A week containing a year boundary lands in one week bucket only, which is the test that catches isocalendar-based bucketing.

Follow-up

Utilisation gets computed on this spine. What breaks for a consultant hired mid-window or terminated mid-window, and where would you fix it?
How would you extend this to consultant x engagement x week without the spine exploding to a row count nobody can hold in memory?

Write a function to calculate the mean and median of a list of numbers.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

How would you optimize a SQL query to retrieve data efficiently?

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Dense consultant-week series with a trailing four-week average

Medium · window functions · calendar spine · running aggregates

fct_time_entry holds consultant_id, engagement_id, charge_code, work_date, hours, is_billable and status. dim_consultant holds consultant_id, is_billable_role, hire_date, termination_date and is_current. Produce one row per billable-role consultant per ISO week over the last 26 weeks with approved billable hours and the trailing four-week average including the current week. Weeks with no time entries must appear as 0, not disappear. No consultant may appear before hire_date or after termination_date. State where you put the right-hand edge of the window and why.

Approach

Build the week spine with generate_series over date_trunc('week', ...), which is Monday-aligned in Postgres, and CROSS JOIN it to the eligible consultant set, bounding each consultant by hire_date and termination_date. The spine is what converts an absent week into a zero week; no amount of COALESCE on a query driven by fct_time_entry can recover a row that was never there.
Aggregate time entries to (consultant_id, week_start) in a CTE first, then LEFT JOIN that onto the spine and COALESCE(hours, 0). Aggregating before the join keeps it one-to-one and removes any chance of fan-out.
Average with AVG(billable_hours) OVER (PARTITION BY consultant_id ORDER BY week_start ROWS BETWEEN 3 PRECEDING AND CURRENT ROW). Use ROWS rather than RANGE: on a dense spine they agree, but ROWS states the intent in row terms and does not change meaning if the spine ever develops a hole.
Decide what the first three weeks of each consultant's series mean. That window averages over fewer than four rows, so either label them as partial with a COUNT(*) OVER the same frame, or suppress them. Pick one and say so.
Hold the right edge of the window back past the measured timesheet backfill tail. Entries are created days to weeks after work_date, so the most recent weeks are systematically incomplete and the series will show a decline that reverses on the next refresh. Measure the tail rather than guessing a round number.

Worked solution (30 min)

Generate the 26 week_start values and confirm the count and the boundary dates before joining anything.
CROSS JOIN to billable-role consultants and apply the hire_date and termination_date bounds, then count rows per consultant.
Aggregate approved billable entries to consultant-week, LEFT JOIN onto the spine, COALESCE to 0.
Add the windowed average with an explicit ROWS frame, plus a COUNT over the same frame so partial windows are visible.
Hand-verify one consultant: print their 26 weeks and check that week 5's average equals the mean of weeks 2 through 5.

Expected result: 26 rows per eligible consultant, fewer only where hire_date or termination_date truncates the series. Zero-hour weeks are present with 0.00 hours. The trailing average on each row equals the arithmetic mean of that week and the three preceding weeks in the same consultant's series.

Verify it yourself

SUM(billable_hours) over the dense series equals SUM(hours) over approved billable entries in the same window, proving the LEFT JOIN added no rows and lost none.
Row count per consultant equals 26 minus any weeks outside their hire and termination bounds.
The hand-checked consultant's week-5 average matches the mean computed by hand from weeks 2 to 5.
No week_start in the output falls inside the held-back backfill tail.

Follow-up

Extend to a 13-week average. Does your choice between ROWS and RANGE change, and why?
Add the availability denominator so this becomes utilisation. What has to be subtracted from calendar hours, and what has to be prorated?
Reproduce this series as it would have looked 30 days ago. Which column do you filter on, and why do the two versions differ?

How do you prioritize multiple tasks and deadlines?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you approach a situation where a client is skeptical about data-driven decisions?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

Given a dataset with sales information, how would you identify trends and anomalies?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Describe how you would design an experiment to test the effectiveness of a new marketing strategy.

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Refuse an early stop on a weekly-monitored pilot

Medium · peeking · sequential testing · alpha spending · novelty effect

A 26-week pilot is running and the sponsor reads a dashboard every Monday. At week 6 the primary metric is up with a nominal p of 0.03 and the sponsor wants to roll out. Quantify the type I error those weekly looks have already spent, give the boundary the week-6 look would have had to clear under a pre-registered four-look O'Brien-Fleming design, and explain why a week-6 estimate of a newly introduced delivery process is a poor estimate of its steady-state effect in either direction. Say what you would do instead.

Approach

Correct the naive arithmetic first. Looks at cumulative data are positively correlated, so the inflation is not 1 - 0.95^26 = 0.74. The classical repeated-significance result gives about 0.19 at ten equally spaced looks and about 0.25 at twenty, so 26 weekly looks at a nominal 0.05 sit near a quarter, roughly five times the advertised rate.
Give the pre-registered alternative concretely. An O'Brien-Fleming boundary with four equally spaced looks uses z_k = z_K * sqrt(K/k), which yields 4.05, 2.86, 2.34 and 2.02, corresponding to nominal two-sided p of roughly 0.00005, 0.004, 0.019 and 0.043. Week 6 is about a quarter of the information, so the boundary is 4.05 and the observed p of 0.03 is z = 2.17.
Offer the option that actually fits continuous monitoring: an always-valid confidence sequence or a mixture sequential probability ratio test, valid at every stopping time at the cost of wider intervals, rather than a fixed-horizon test read 26 times.
Separate the statistical objection from the measurement objection. Six weeks in, treated teams are six weeks into a changed way of working. Novelty inflates early estimates through attention to anything new, primacy depresses them through resistance to a process change, and in delivery-process pilots primacy usually dominates, so the early estimate is as likely too low as too high.
Diagnose rather than assert. Plot the effect by weeks since exposure rather than by calendar week and compare cohorts that adopted at different calendar dates. If the curve is still moving at the pre-specified end of the window, the window was too short and the estimand should be restated.

Worked solution (20 min)

Convert the reported p of 0.03 to a test statistic: two-sided, so z = 2.17.
Construct the four-look O'Brien-Fleming boundary: choose z_4 = 2.024 so the overall two-sided level is 0.05, then z_k = 2.024 * sqrt(4/k), giving 4.05, 2.86, 2.34, 2.02.
Place week 6 on the information fraction: 6/26 = 0.23, which is the first look, boundary 4.05.
Compare 2.17 against 4.05: the look does not cross, so continue to the next pre-specified look rather than stopping.
Plot the weekly effect estimate against weeks since exposure with its confidence band and check whether the curve has flattened before the pre-specified end.

Expected result: Roughly 25% type I error already spent by unstructured weekly looks, a required week-6 boundary of z at least 4.05 against an observed z of 2.17, and a recommendation to continue to the next pre-specified look, with the separate warning that a week-6 estimate of a process change is not its steady-state effect.

Verify it yourself

O'Brien-Fleming boundaries must fall across looks and the final one must sit just above the fixed-horizon 1.96; a final boundary below 1.96 means the spending function is wrong.
Simulating the null with 26 sequential looks at a fixed nominal 0.05 should reproduce a rejection rate near 0.25, not near 0.74.
Information fraction at each look must be built from accumulated observations or events, not elapsed calendar time, whenever enrolment is uneven.

Follow-up

The sponsor argues the cost of a false positive here is low. Does that change the boundary, and how would you set alpha from a cost ratio instead of convention?
You have four pre-registered looks. What does adding a futility boundary buy, and what does it cost?
How would you distinguish a novelty effect from a real effect that genuinely decays?

Fees per billable head fell nine percent; rank the causes

Hard · decomposition · seasonality · clustered inference

Recognised fees per billable head fell about 9% quarter over quarter and leadership wants a ranked cause by Friday. You have fct_time_entry, fct_invoice_line, fct_engagement and dim_consultant, plus a workday and regional holiday calendar. Decompose the move into available hours per head, billable utilisation and realised dollars per billable hour, separate calendar effects from behaviour, split within-segment change from mix, and deliver a ranked attribution with uncertainty. Name the one further cut you would run if the leading term is ambiguous.

Approach

Write the identity before computing anything: fees per head = (available hours per head) x (billable utilisation) x (realised dollars per billable hour), where the last term is recognised fees divided by approved billable hours. Verify the product reproduces the reported 9% exactly; if it does not, the scope of the three inputs disagrees and nothing downstream is trustworthy.
Decompose in logs: d ln(fees per head) = d ln(available per head) + d ln(utilisation) + d ln(realised rate). The log form is exactly additive for a product, so contributions sum to the total without an interaction residual and cannot attribute more than the whole move.
Strip the calendar first, because it is the term nobody can act on: available hours per head moves with scheduled workdays, regional public holidays, leave and any change in fte_fraction mix. Quantify how many of the nine points are calendar and quote the remainder as the behavioural move.
Split each surviving term by practice_area, level, region and pricing_model, and within each split separate within-segment change from mix using the weighted three-term decomposition, since a shift toward lower-rate levels reads as a rate decline that no individual rate produced.
Attach uncertainty at the unit that generates it: anything fee-side clusters at client_id because engagements in one account share a partner, a rate card and a team. With fewer than roughly forty clusters use a wild cluster bootstrap or a CR2 correction, since cluster-robust standard errors are downward-biased in that regime.
If realised rate leads and is ambiguous, the disambiguating cut is discount versus write-off versus fixed-fee overrun: compare invoiced fees to the standard-rate value of hours worked, and read written_off_hours and scope overrun by pricing_model.

Follow-up

Two of the nine points are calendar. How do you say that to an executive without sounding like you are explaining the number away?
Utilisation is up and realised rate is down. What single operational behaviour produces both, and how would you confirm it?
What would you need to see to claim this is a demand problem rather than a delivery or pricing one?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

What the behavioural round is actually scoring

Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.

Can you provide an example of how you resolved a conflict within a team?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

Your utilisation dashboard caused a staffing decision on incomplete data

Medium · late-arriving data · postmortem · reporting windows

Six weeks ago you shipped a weekly billable-utilisation dashboard keyed on fct_time_entry.work_date. It showed a six-point drop across the three most recent weeks. A practice lead pulled two consultants off an engagement in response. The drop reversed on the next refresh, because time entries are created days to weeks after the work happens and entered_at trails work_date. Describe what you do now: the diagnosis, the change to the artifact, and the conversation with the person who acted on your number.

Approach

Name the probe: whether you own a reporting-design error rather than reclassifying it as someone else's timesheet compliance problem, and whether you fix the class of bug instead of the single week.
Quantify before explaining. Measure the backfill curve directly from entered_at: for work_date D, the share of final hours that existed as of D plus k, for k from 1 to 45. That gives an observed tail length instead of a guessed cutoff.
Change the artifact so the incomplete region cannot be read as a trend. Either end the trended series at snapshot minus the measured tail, or publish an as-of-entered_at series that is internally consistent, and label which one is on screen.
Tell the person who acted, first and directly, with the corrected series and the specific decision to revisit. A correction that arrives after they notice costs more than the original error.
Add a standing completeness tile: timesheet submission lag, the share of entries where entered_at date minus work_date exceeds seven days, so the dashboard shows its own reliability rather than depending on you remembering.

Follow-up

Leadership still wants to see the current week. What do you show, and how do you label it?
One practice runs a three-day lag and another twenty days. Do you set one firm-wide cutoff or one per practice, and what does that cost in comparability?

Defending your own impact claim without randomisation or clean units

Hard · self-selection · clustered inference · impact measurement

At your review you plan to claim that the realisation dashboard you built recovered 1.4 million dollars. The evidence is that engagement-month realisation rose four points over two quarters among engagements whose leads used it. Adoption was voluntary. There are about sixty client accounts and the top five carry most fees. A new rate card shipped in the same quarter. Write the claim you can defend, the estimate you would actually produce, and what you say when asked for a causal number you cannot get.

Approach

Name the probe: whether you can separate the number you want from the number the data supports, under review pressure, without either inflating it or retreating to saying nothing can be known.
State both identification problems concretely. Voluntary adoption means adopting leads are plausibly the ones who already manage realisation, so the comparison is confounded at the person level. The rate card changes bill_rate_usd, which sits in the realisation denominator, so part of the four-point move is arithmetic rather than behavioural.
Neutralise what you can. Recompute realisation with bill rates snapshotted on work_date, or hold the denominator at the old rate card, so the rate-card change cannot move the metric by construction. Then rerun the comparison.
Get the inference right for the unit count. Cluster at client_id, not engagement, because engagements in one account share a partner, a rate card and a team. With sixty accounts and five carrying most fees, the effective cluster count is far below sixty, so report a wild cluster bootstrap interval rather than plain cluster-robust standard errors, which are biased downward in that regime.
Report both weightings and explain the divergence: an account-weighted estimate describes the typical account, a value-weighted one describes the revenue, and if they disagree a small number of accounts is carrying the result. Then give the decision-relevant sentence: the defensible range, whether its lower bound still clears the build cost, and what a proper staggered rollout would have bought.

Follow-up

The pre-period trends for adopters and non-adopters are not parallel. What do you report then?
You get to design the next rollout. What do you change so the same question is answerable, without randomising individual accounts?