Interview guides · Aubay Spain

Aubay Spain Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Aubay Spain: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

Welcome to the interview preparation guide for the Data Scientist role at Aubay Spain. As a leading European IT and integration consulting company, Aubay partners with top-tier clients across banking, insurance, telecommunications, and energy to drive their digital transformations. In this role, you are not just building models; you are delivering data solutions that solve complex, real-world business challenges for Aubay's enterprise partners.

How much statistics you need depends on the flavour of the seat. Experiment-facing work wants you deep enough to notice that repeated looks at accumulating data inflate the false positive rate of a fixed-sample test; modelling-facing work wants estimation and honest uncertainty intervals.

Focus areas

Cluster inference at the account, not the engagement
Correct late-entered timesheets before trending recent weeks
Reconstruct pipeline stage history from mutable rows

Overview

Your impact as a Data Scientist here is immediate and visible. You will be stepping into dynamic project environments, working closely with client stakeholders, and leveraging data to optimize processes, predict trends, and build intelligent products. The role demands a unique blend of analytical rigor and consultative agility. You must be as comfortable explaining a machine learning concept to a non-technical business leader as you are writing production-level Python code.

Expect a role that is highly collaborative, fast-paced, and varied. Because Aubay operates on a consulting model, the specific products, data scales, and problem spaces you tackle will evolve based on your project assignments. This makes the position incredibly rewarding for adaptable professionals who thrive on continuous learning and want to see their data strategies shape major corporate initiatives across Spain and Europe.

The Aubay Spain interview process

Aubay Spain candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Initial Outreach

An extra round usually exists because something is still open after the standard loop: a skill the earlier interviews did not sample, a level decision, or two interviewers who disagreed. It is rarely a rerun of what you already did well. Ask the recruiter who you are meeting, what function they sit in, and how long the session runs. That is an ordinary scheduling question, and the answer changes what you should prepare. What separates a strong candidate here is treating the round as a fresh evaluation with its own bar, rather than assuming earlier performance carries you through or sinks you.

What to demonstrate

Whether you can answer well on ground the earlier rounds did not cover, without leaning on what you already said to someone else
Consistency of the facts in your stories: the same sample size, timeframe, team size and scope of your own role as in earlier conversations
How you handle an unfamiliar format live, including whether you ask what kind of answer is wanted before producing one

How to prepare

Ask the recruiter for the interviewer's function, the length, and whether to expect a coding surface, a discussion, or a presentation. Preparing for a 30 minute conversation with a partner team is not the same work as preparing for a 60 minute technical block.
Write out what each earlier round actually covered, then list the two or three areas nobody probed. That gap is the most likely subject of the extra round.
Re-read the numbers in the project stories you have already told, so a second telling does not quietly contradict the first.

In-Depth Interview

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Discussion of Experiences

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Potential Technical Assessment

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Final Offer Stage

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Modelling win rate on proposals with a recorded outcome, using fields written after the decision

Two failures compound here. First, stage IN ('withdrawn','no_decision') is not missing at random: those are disproportionately deals that were going to be lost, so training on won-plus-lost only inflates apparent win rate and distorts the coefficients. Second, fields like engagement_id, final scope and revised pricing are populated after the outcome is known, so including them leaks the label and produces a model with excellent backtest accuracy and no forward value. Restrict features to values knowable at submitted_at.

Computing days-to-pay or proposal cycle time over completed records only

At any snapshot date, invoices that have already been paid are disproportionately the fast ones, and proposals that already have a decision are disproportionately the quick ones. Averaging over the completed set alone biases both numbers downward, and the bias grows exactly when the business is deteriorating, because the slow cases are the ones still open. Unpaid and undecided records are right-censored; use Kaplan-Meier or a restricted mean up to a fixed horizon, and never fill paid_at with a placeholder.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Building features from data that postdates the prediction time

Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.

Questions & practice

Explain the bias-variance tradeoff and how you manage it in your models.

Medium · statistics and probability

Approach

Quantify uncertainty explicitly rather than reporting a point estimate alone.
Say what the estimate is of, and over what population it generalises.
Write down the assumption the method needs before you use the method.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

How would you explain the results of a complex random forest model to a marketing director who has no technical background?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Build a dense consultant-week hours spine with zero weeks

Easy · calendar spine · groupby · missing rows

You are given time_entries (time_entry_id, consultant_id, engagement_id, charge_code, work_date as datetime64, hours, is_billable, status) and a list of consultant_ids in scope plus a start and end date. Produce a DataFrame with exactly one row per consultant per Monday-anchored week in that range, with columns consultant_id, week_start, billable_hours, total_hours. Weeks in which a consultant logged nothing must appear as zeros rather than be absent. Count only rows with status == 'approved'. Do not loop over consultants and do not use resample.

Approach

Anchor weeks arithmetically: week_start = work_date - pd.to_timedelta(work_date.dt.weekday, unit='D'). Avoid dt.isocalendar().week, whose numbers restart each January and do not sort across a year boundary.
Construct the spine from the inputs, not from the data: pd.date_range(start, end, freq='W-MON') crossed with the in-scope consultant_ids via pd.MultiIndex.from_product, so its size is fixed before any aggregation runs.
Aggregate approved entries once with a single groupby on [consultant_id, week_start], producing total hours and billable hours (hours.where(is_billable, 0).sum()) in the same pass.
Left-merge the aggregate onto the spine, fillna(0.0) on both hour columns, and keep float dtypes so later ratios do not integer-divide.
Assert the shape before returning: len(out) == n_consultants * n_weeks, and out.total_hours.sum() equals the sum of hours over the filtered input inside the window.

Worked solution (20 min)

Filter to status == 'approved' and to work_date within [start, end], then derive week_start by subtracting the weekday offset.
Build weeks = pd.date_range(start_monday, end, freq='W-MON') and spine = pd.MultiIndex.from_product([consultant_ids, weeks], names=['consultant_id','week_start']).to_frame(index=False).
agg = filtered.assign(billable_hours=filtered.hours.where(filtered.is_billable, 0.0)).groupby(['consultant_id','week_start'], as_index=False)[['hours','billable_hours']].sum().
out = spine.merge(agg, how='left', on=['consultant_id','week_start']).fillna({'hours':0.0,'billable_hours':0.0}).rename(columns={'hours':'total_hours'}).
Sort by consultant_id then week_start and run the two assertions on row count and hour conservation.

Expected result: A frame of exactly len(consultant_ids) x len(weeks) rows, sorted by consultant then week, in which total_hours sums to the approved hours inside the window and zero-hour weeks are present as 0.0 rather than missing.

Verify it yourself

Row count equals n_consultants * n_weeks exactly, for a consultant list that includes someone with no entries at all.
Total hours in the output equal total approved hours in the filtered input, to the cent.
billable_hours <= total_hours on every row.
A week containing a year boundary lands in one week bucket only, which is the test that catches isocalendar-based bucketing.

Follow-up

Utilisation gets computed on this spine. What breaks for a consultant hired mid-window or terminated mid-window, and where would you fix it?
How would you extend this to consultant x engagement x week without the spine exploding to a row count nobody can hold in memory?

Build a client cohort revenue table from invoice lines

Easy · cohorts · date truncation · revenue recognition

dim_client holds client_id and first_engagement_date. fct_engagement maps engagement_id to client_id. fct_invoice_line holds engagement_id, line_type, period_start_date, amount_usd and status; recognised fees are lines with line_type IN ('fees','milestone','credit_note') and status <> 'draft', and credit notes carry negative amount_usd. Return one row per (cohort_month, months_since_first) giving distinct clients billed and net recognised fees. cohort_month is the month of first_engagement_date; months_since_first is the whole-month gap between period_start_date and first_engagement_date. Long format, no pivot. Decide explicitly what to do with clients whose first_engagement_date is NULL.

Approach

Filter fct_invoice_line to recognised lines first (line_type, status <> 'draft'), and key recognition on period_start_date rather than issued_at or paid_at. Billing period, issue date and payment date are three different clocks, and mixing them produces a cohort table nobody can reconcile against finance.
Join invoice lines to fct_engagement to reach client_id, since invoice lines carry engagement_id only, then to dim_client for first_engagement_date.
Truncate both dates to month, then compute months_since_first as 12 * (year(bill_month) - year(cohort_month)) + (month(bill_month) - month(cohort_month)). Doing it on truncated months is what keeps a client billed on day 2 of the third month in bucket 2 instead of bucket 1.
Aggregate to (cohort_month, months_since_first) with SUM(amount_usd) and COUNT(DISTINCT client_id), keeping credit_note lines inside the sum so the figure is net rather than gross.
Handle NULL first_engagement_date as an explicit exclusion and report how many clients and how much revenue that removes, rather than letting the join drop them silently. That exclusion is why the output's total is smaller than the filtered line set's total, so carry the excluded figure as a named number rather than discovering the gap later.

Follow-up

How would you turn this table into a 12-month net revenue retention ratio, and what exactly becomes the denominator?
One account contributes 40 percent of a cohort's month-13 fees. How do you present the cohort so a reader sees that before they act on the ratio?
If subsidiaries roll up to parent_client_id, which first_engagement_date defines the cohort, and does the rollup change the cohort a subsidiary lands in?

Longest consecutive bench spell per consultant from weekly activity

Hard · gaps and islands · window functions · run length

You have a dense series (consultant_id, week_start, billable_hours, full_week_leave BOOLEAN) covering 26 weeks for every billable-role consultant, with zero-hour weeks present as 0. Return every maximal run of consecutive weeks where billable_hours = 0, as consultant_id, spell_start_week, spell_end_week and week_count, then each consultant's longest run. Weeks flagged full_week_leave must neither break a run nor count toward its length. Mark runs still open at the last week of the series rather than reporting them as finished.

Approach

Remove full_week_leave weeks from the sequence first, then re-number what remains with ROW_NUMBER() OVER (PARTITION BY consultant_id ORDER BY week_start). Deleting before numbering is precisely what makes a leave week transparent to a run instead of a break in it; filtering afterwards would leave a gap in the sequence and split the spell.
Flag each surviving week as bench (billable_hours = 0) or active, then compute two row numbers per consultant: one over all surviving weeks, one partitioned additionally by the bench flag. Their difference is constant inside any run of adjacent same-flag weeks, which is the island key.
Group the bench weeks by (consultant_id, island key) and take MIN(week_start), MAX(week_start) and COUNT(*). Do not island on week_start minus an interval times a row number: that variant assumes the spine has no holes, and the leave removal has just deliberately put holes in it.
Pick the longest spell per consultant with ROW_NUMBER() OVER (PARTITION BY consultant_id ORDER BY week_count DESC, spell_start_week) so ties resolve deterministically, rather than a MAX that cannot carry the accompanying dates.
Handle both edges as censoring. A spell touching the last week of the series is still open and its length is a lower bound, so flag it; a spell touching the first week may have started earlier, so either flag it too or exclude it from any length distribution and say which.

Worked solution (40 min)

Filter out full_week_leave rows and count the surviving weeks per consultant.
Add the bench flag and the two row numbers, then print one consultant's full sequence to see the island key stay constant across a run.
Group bench weeks by consultant and island key and aggregate to spell boundaries and lengths.
Add the open-at-the-edge flag by comparing spell_end_week to the consultant's last surviving week.
Rank spells per consultant and select the longest with the deterministic tiebreak.

Expected result: One row per maximal bench run with its first and last week, its length in weeks and an is_open flag, plus a one-row-per-consultant longest-spell table. A consultant with no bench weeks appears in neither, which is correct and worth stating.

Verify it yourself

For each consultant, the sum of week_count across all spells equals their count of bench weeks in the leave-filtered series.
No two spells for the same consultant are adjacent in the re-numbered sequence; adjacency would mean the island key was computed wrongly.
The longest-spell table has at most one row per consultant and its week_count matches the maximum in the spells table.
Hand-check one consultant by printing their 26 weeks as a strip of zeros, hours and leave markers, and reading the runs off it.

Follow-up

A spell begins before your 26-week window. How do you detect that, and what do you report as its length?
Rebuild this at daily grain. What changes about weekends, holidays and the island key?
How would you separate true bench from unbilled delivery work on a live engagement, given only charge codes?

A client wants to segment their customer base to improve marketing ROI. How do you approach this?

Medium

Approach

Say what you would check first and why it is the highest-information step.
Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Walk me through your typical data cleaning and exploratory data analysis (EDA) process.

Medium

Approach

Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Define competitive win rate for a quarterly pipeline review

Easy · win rate · censoring · denominators

Pipeline reporting currently shows win rate as won proposals divided by all proposals created. Using fct_proposal (proposal_id, expected_value_usd, stage, loss_reason, is_competitive, created_at, submitted_at, decided_at), write a definition you would defend to a finance lead: numerator, denominator, window and every exclusion. Decide whether to weight by count or by expected_value_usd, and justify the choice. State what happens to proposals still in 'qualifying', 'scoping' or 'submitted' at the snapshot date, and why dropping them is not a neutral act.

Approach

Cohort on decided_at, not created_at. A created_at cohort mixes decided and undecided proposals, so the rate for a recent quarter is computed over whichever deals resolved fastest and drifts upward on every refresh.
Set the denominator to stage IN ('won','lost'). Excluding 'withdrawn' and 'no_decision' is defensible only if you report their count and value alongside, because they are not missing at random: they skew toward deals that were heading for a loss.
Filter is_competitive = TRUE. Sole-sourced follow-on work closes at near certainty and mechanically inflates the rate, so it belongs on its own line rather than inside the headline number.
Report count-weighted and value-weighted side by side. They diverge when large proposals lose more often than small ones, which is the diagnostically useful case; the value-weighted version is the one that ties to revenue.
Treat open proposals as right-censored rather than absent. Either restrict to cohorts old enough that most proposals have decided, picking that horizon off a Kaplan-Meier cycle-time curve, or publish the rate with the undecided count and value attached to it.

Worked solution (25 min)

Compute the naive rate (won / all created) and the proposed rate (won / won+lost, is_competitive = TRUE, cohorted on decided_at) for the same four quarters.
For each quarter, count and sum the value of proposals in 'withdrawn' and 'no_decision', and of proposals still open at the snapshot.
Compute the value-weighted version and compare it with the count-weighted one.
Re-run the most recent quarter as it would have looked 30 days ago and record how much each definition moved.

Expected result: Two numbers per trailing quarter, count-weighted and value-weighted competitive win rate, each with the denominator predicate written out in full, cohorted on decided_at, plus a reported count and value for withdrawn, no_decision and still-open proposals. The horizon is chosen so the undecided share is small rather than set to a round number of days.

Verify it yourself

The value-weighted and count-weighted rates differ. If they are identical, expected_value_usd is either constant or lost in a join.
Re-running a given quarter two weeks later moves the decided_at-cohorted number by much less than it moves the created_at version.
Withdrawn and no_decision appear as reported counts, not as a silent exclusion.

Follow-up

Win rate rose four points while proposal volume fell by a third. Which do you report first, and what single query separates a genuine quality shift from a volume shift?
How would you detect partners reclassifying likely losses as 'withdrawn' to protect the rate?

Win rate jumped after a pipeline hygiene push

Medium · denominator effects · selection · pipeline

Competitive win rate by value jumped from 38% to 51% in one quarter. The metric sums expected_value_usd from fct_proposal where stage = 'won' over the same sum where stage IN ('won','lost'), filtered to is_competitive = TRUE and decided_at in the window. That quarter, partners were asked to clear stale opportunities out of the pipeline. You also have fct_proposal_stage_event. Quantify how much of the jump is selling and how much is the denominator changing shape, and state what you would monitor from now on.

Approach

Stop looking at the ratio and plot its parts: won value and lost value by decision week, plus counts and value landing in the excluded stages 'withdrawn' and 'no_decision'. A ratio move with a flat numerator is a denominator story.
Date the intervention and look for a spike in excluded-stage terminations around it. Those deals are not missing at random: opportunities that go stale and get cleaned up are disproportionately ones that were heading for a loss.
Reconstruct history from fct_proposal_stage_event rather than reading the mutable fct_proposal.stage column, since the cleanup overwrote the current stage in place. Identify proposals that moved from 'submitted' or 'scoping' to 'withdrawn' after the push, and how long they had been open.
Bound the effect: recompute win rate with withdrawn and no_decision counted as losses. The truth lies between that lower bound and the reported figure; report both numbers rather than picking one.
Check the second lever: is_competitive is also mutable, so reclassifying a deal as sole-sourced removes it from the rate entirely. Count flips of that flag in the window.
Propose the standing monitor: share of decided value terminating in excluded stages, published beside win rate, so the denominator can never move silently again.

Follow-up

Some withdrawals are genuine, for example a client cancelling a procurement. How would you separate those from cleanup?
What would you have asked for before the hygiene push to make this measurable afterwards?
Does the same exclusion logic distort proposal cycle time, and in which direction?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

If a stakeholder asks you to build a predictive model but the data quality is exceptionally poor, what do you do?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Close with what you would do differently, concretely.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

What is your ideal project structure and team environment?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

How do you handle imbalanced datasets in a classification problem?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?