Interview guides · Amadeus

Amadeus Data Scientist Interview Questions & Guide 2026

Amadeus data scientist prep: probability, sampling distributions, and stating the assumption behind every test you run.

The 60-second brief

A Data Scientist at Amadeus sits at the intersection of massive-scale travel data and cutting-edge algorithmic innovation. As a leader in travel technology, Amadeus processes billions of transactions, providing the underlying infrastructure for airlines, airports, and hotels globally. Your role is to translate this complex, high-velocity data into actionable business intelligence and predictive models that optimize travel experiences and operational efficiency.

Product-sense cases reward reasoning from a mechanism to a testable prediction. Reciting every metric you can name reads as pattern matching; naming the single quantity that would move if your explanation were true reads as thinking.

Focus areas

Compute occupancy from sellable nights, never property averages
Cohort bookings on stay date, not booking date
Model demand against availability, not observed bookings

Overview

Whether you are working within the Advertising division or core infrastructure teams, you will be expected to bridge the gap between theoretical machine learning and real-world product impact. The work is intellectually rigorous, requiring a blend of statistical depth and product sense. You aren’t just building models; you are solving critical industry challenges—such as demand forecasting, personalized recommendations, or ad-tech optimization—in an environment where accuracy and scalability are paramount.

Successful candidates in this role demonstrate a deep curiosity for how travel ecosystems function and an ability to communicate complex findings to non-technical stakeholders. You will thrive here if you enjoy working with large, messy datasets and possess the drive to see your models move from a Jupyter notebook into a production environment that serves millions of travelers daily.

The Amadeus interview process

Amadeus candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screening

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Assessments

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Behavioral Interviews

Rounds of this kind usually include one question about work that did not go well, and it is the part that carries the most information. Anyone can narrate a shipped win. What the interviewer learns from a project that stalled is how you behave without a result to hide behind: whether you noticed the problem yourself, how long it took, and who you told. Answers that route the failure onto a data pipeline or a reorganisation close the topic without answering it, and the follow-up comes back to your own part.

What to demonstrate

Whether you found the error yourself or someone else found it, and how long it sat before anyone knew
What you changed afterwards, stated as a check you now run rather than a lesson you now believe
Whether the mistake you choose has real cost attached, such as a quarter of misdirected roadmap or a metric that was reported upward, instead of one that flatters you

How to prepare

Choose a failure you caught yourself and be ready to say what tipped you off. A story where someone else caught it is still usable, but you will be asked why you missed it.
Write down the check you added afterwards and where it lives now, so the correction is a concrete artefact rather than a resolution.
Rehearse saying the cost out loud. Candidates shrink the number by instinct once the interviewer is in the room.

Where candidates lose points

Fitting demand or price elasticity on observed bookings, when availability and restrictions censor the data

Bookings equal the minimum of demand and what was actually sellable, so a sold-out date records the capacity, not the demand behind it, and the censoring is worst precisely on the highest-demand dates. Meanwhile price is set from a forecast of that same demand, so high-demand dates carry high prices and the raw correlation between price and bookings is biased toward zero and frequently comes out positive, which reads as 'raising price increases demand'. Restrictions compound it: a minimum-length-of-stay rule or a closed-to-arrival flag suppresses bookings with no price movement at all, so the effect lands on the price coefficient if the restriction is not in the model. Join fct_rate_availability_snapshot at snapshot_date equal to the search or booking date, restrict the estimation sample to unit-dates that were genuinely open, carry the restriction flags as controls, and lean on an instrument or a deliberate price experiment before quoting an elasticity.

Using the search as the demand unit when the trip is the demand unit

One trip intent generates dozens of searches over days, across devices, mostly while logged out, and the number of searches per intent is a property of the interface rather than of demand. Any product change that encourages comparison or re-sorting inflates the denominator, so look-to-book falls while the product improves, and a change that reduces re-searching raises the metric while nothing about demand moved. The logged-out majority also means traveller_id is NULL for most early-funnel rows, so joining searches to bookings on traveller_id silently drops the part of the funnel you were trying to measure. Collapse on trip_intent_key first, then count, and report searches per intent separately as a diagnostic rather than letting it sit inside a conversion rate.

Dropping rows with missing values without naming the mechanism

Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.

Ending an analysis without a recommendation or next step

Close with what you would do and what would change your mind, stated as a condition you can check later. If the evidence is genuinely inconclusive, recommend the specific next measurement and say what it costs in time or exposure.

Questions & practice

How do you balance long-term user retention versus short-term revenue in an advertising model?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

Write invariant checks for a rate availability snapshot

Easy · data quality · pandas · null semantics

The DataFrame snap holds one row per supply_unit_id per forward stay_date per snapshot_date, with units_available, units_sold_to_date, units_blocked, listed_price_usd (nullable), is_closed_to_arrival, min_length_of_stay and lead_time_days. Write check(snap) returning a tidy failure frame with columns check_name, supply_unit_id, stay_date, snapshot_date, detail, plus a per-check summary count. Cover at least: primary-key uniqueness, lead_time_days equal to (stay_date - snapshot_date) in days, listed_price_usd null exactly when units_available is 0, and non-negative counts. It must return the right empty frame on empty input.

Approach

Write each check as a boolean mask over the whole frame, not a row-wise function, and append a small frame carrying check_name plus the three key columns and a detail string built from the offending values. Concatenate at the end with a typed empty frame in the list so an all-clean or empty input still returns the declared columns and dtypes. Declare the check names once as a module-level tuple, CHECK_NAMES, and drive both the run loop and the summary off that tuple rather than off whatever happened to fail.
Test nullity with .isna(); listed_price_usd is a float column so a NULL is NaN, and NaN == NaN is False, which means any == None or == np.nan comparison returns an all-False mask and the check passes vacuously on every row.
Check the price rule in both directions: a non-null price where units_available is 0 means someone imputed the last known price onto a sold-out unit-date, which is the single most damaging corruption here because it turns censored observations into apparently-observed ones for any elasticity work.
Separate hard invariants from business expectations and label them with a severity column. Uniqueness of (supply_unit_id, stay_date, snapshot_date), the lead-time identity and non-negativity are invariants. Monotonicity of units_sold_to_date across snapshot_date is not: cancellations legitimately reduce cumulative pickup, so it belongs at warn severity with a threshold on the size of the drop.
Build the summary by grouping the failure frame on check_name and then reindexing against CHECK_NAMES with fill_value=0. A bare groupby emits no row for a check that found nothing, so a clean input returns a summary listing only the broken checks and an empty input returns no summary at all — which on a dashboard is indistinguishable from the suite never having run, the one state a summary exists to rule out. Return counts per stay_date alongside it so a caller can see whether failures are uniform or concentrated on a single partition, which is what distinguishes a code bug from one bad ingest day.

Follow-up

Which of your checks would you make a hard pipeline failure and which a warning, and what is the cost of getting that wrong in each direction?
units_sold_to_date drops by 4 between two consecutive snapshots for one unit-date. Give two innocent explanations and one that should page someone.
How would you run these on a table too large to hold in memory, keeping the same failure frame?

Expand bookings into stay nights without losing cents

Easy · pandas · reshaping · revenue allocation · vectorisation

You are given a pandas DataFrame bookings with columns booking_id, check_in_date, check_out_date, nights, unit_revenue_usd (the booking total, two decimal places), about 200,000 rows. Write a function returning stay_nights at one row per booking per stay date, with columns booking_id, stay_date, night_index (1-based) and unit_revenue_usd allocated across the nights so the per-booking sum equals the original to the cent. No per-row Python loop and no apply over rows. State your rule for the departure date.

Approach

Confirm the grain rule first: a stay night is the arrival date through the night before check_out_date, so nights = (check_out_date - check_in_date).days and the departure date is never a row. Assert this against the stored nights column and report mismatches rather than trusting either side.
Build the expansion arithmetically instead of materialising date lists: counts = nights.to_numpy(); starts = np.repeat(np.cumsum(counts) - counts, counts); offsets = np.arange(counts.sum()) - starts. That gives a 0-based night offset per output row in one pass, and night_index = offsets + 1.
Compute stay_date as np.repeat(check_in_date.to_numpy(), counts) + offsets.astype('timedelta64[D]'). This is O(total nights) with no grouping and no Python-level iteration.
Allocate revenue in integer cents, not floats: cents = np.rint(unit_revenue_usd * 100); base = cents // counts; remainder = cents - base * counts; give the first remainder nights of each booking one extra cent. Dividing floats by nights leaves sub-cent drift that re-sums to 99.99 or 100.01 on a large fraction of bookings.
Verify by grouping the output back to booking_id and comparing integer cents to the input, then state which downstream numbers depend on this reconciliation (net revenue per sellable night, take rate, contribution margin per stayed night all read the night grain).

Worked solution (25 min)

Assert (check_out_date - check_in_date).dt.days equals nights; raise on any mismatch and print the offending booking_ids rather than silently trusting the column.
Build counts, starts and offsets with np.repeat and np.arange as above; total output rows equals counts.sum().
Derive stay_date by adding the day offsets to the repeated check_in_date, and night_index as offsets + 1.
Allocate integer cents with floor division plus largest-remainder top-up on the first remainder nights of each booking.
Reconcile: group the output by booking_id, sum the cents, compare exactly to the input cents, assert zero differing rows.

Expected result: Output row count equals bookings.nights.sum(). A three-night booking of 100.00 USD splits 33.34 / 33.33 / 33.33 and sums to exactly 100.00. Every booking's maximum stay_date is check_out_date minus one day, and night_index runs 1..nights with no gaps.

Verify it yourself

len(stay_nights) == bookings['nights'].sum()
Per-booking sum of allocated cents equals the input cents exactly, for all rows, with no tolerance needed
stay_nights.groupby('booking_id')['stay_date'].max() == check_out_date - 1 day, and .min() == check_in_date
night_index per booking has min 1, max nights, and nunique equal to nights

Follow-up

The booking is later amended from three nights to five. What has to happen to the already-written night rows, and which of them may not be rewritten?
Nights of one booking fall in two calendar months and two tax jurisdictions. Does your equal-split allocation still defend the monthly revenue number, and when would you weight nights by the nightly listed price instead?

Describe a situation where you had to join multiple complex tables; what were the challenges?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Explain how you would handle missing data or outliers in a large-scale travel dataset.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Say which table is the grain you start from, and join outward from it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

How do you optimize a query that is performing poorly on a massive dataset?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Compare booking pace against last year aligned on days out

Hard · running-aggregates · pace · self-join · seasonality

fct_rate_availability_snapshot carries units_sold_to_date, the cumulative pickup for a unit-date as at snapshot_date, and lead_time_days, which is stay_date minus snapshot_date. For one destination_market_id and one target stay week, build the booking curve: for each lead_time_days from 120 down to 0, the total units sold across the market's units and the seven-day pickup. Join the equivalent stay week last year aligned on lead_time_days rather than on calendar date, and return both curves side by side.

Approach

Restrict to the market's supply units and the seven stay_dates of the target week, then aggregate to lead_time_days by summing units_sold_to_date across units and across those seven stay_dates. This is a cross-section at a fixed days-out, so the snapshot_date differs across the week's stay dates, which is exactly what aligning on days out means.
Repeat for last year's aligned stay week, chosen on the market calendar. A 364-day offset preserves day of week but drifts a moving holiday; a same-date offset preserves the date and breaks day of week. State which you chose, and use market_holiday_flag on fct_stay_night to check whether a holiday sits inside one week and not the other.
Compute seven-day pickup as value minus LAG(value, 7) OVER (ORDER BY lead_time_days DESC). Ordering descending makes time run forward, because days out counts down towards arrival. Every value is a rolling window and adjacent rows share six of their seven days, so this column is not additive and must never be summed to recover total pickup.
Join the two curves on lead_time_days with a FULL OUTER JOIN so a days-out point missing on one side stays visible as a NULL rather than dropping the row and silently shortening the comparison.
Index each curve to its own final on-hand value when the two weeks differ in sellable size, so the comparison reads as pace rather than level.

Worked solution (45 min)

Pull the raw snapshot rows for one supply unit and one stay_date ordered by lead_time_days, and confirm units_sold_to_date is already cumulative before writing any SUM over time.
Aggregate to the market-week cross-section and check monotonicity as days out falls.
Add the LAG-based seven-day pickup and validate a single row straight against the cumulative column: pickup at lead_time_days 30 must equal cumulative at 30 minus cumulative at 37. Do not validate it by summing the column, which the overlap makes meaningless.
Build last year's curve, align on lead_time_days, and note where the holiday falls in each week before drawing any conclusion.
Join, index each series to its final value, and report the absolute and the indexed gap together.

Expected result: One row per lead_time_days from 120 down to 0 carrying this year's and last year's cumulative on-hand units and seven-day pickup; the cumulative series is non-decreasing as lead_time_days falls in both years. The first seven rows, lead_time_days 120 down to 114, carry a NULL pickup because LAG(value, 7) has no seventh preceding row, leaving 114 populated pickup values.

Verify it yourself

The cumulative curve never falls as days out decreases, except where cancellations genuinely reduced on-hand units; a drop is a finding to explain, not automatically a query defect.
The pickup column does not telescope, because the prompt asks for a window at every lead_time_days and those windows overlap sevenfold. Summing all 114 populated values lands near seven times the cumulative endpoint difference, not equal to it. Only a non-overlapping stride telescopes: the seventeen points at lead_time_days 113, 106, 99 and so on down to 1 sum exactly to cumulative at 1 minus cumulative at 120.
The cumulative value at lead_time_days = 0 is at or above the count of fct_stay_night rows for that week with night_status = 'stayed', the gap being same-day cancellations and no-shows.

Follow-up

At 30 days out you are 8% behind last year. What else do you need before telling revenue management to cut price?
Two new units joined this market in March. What does that do to this comparison, and how would you control for it?
The curve at days out zero sits above the stayed-night count for that week. Is that a bug, and how would you show which it is?

How would you design a success metric for a new flight recommendation feature?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

If you notice a sudden 10% drop in our booking conversion rate, how would you go about diagnosing the root cause?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

What are the most common pitfalls you encounter when designing a product experiment?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

What is the difference between statistical significance and practical significance?

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
Name the randomisation unit first; it decides the variance and what the test can detect.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

Build a readable activation metric for six-month conversion windows

Medium · cohort maturity · activation · leading indicators

Leadership wants a weekly number for new-traveller activation. The true metric, new-traveller first-stay completion rate, is readable only once a cohort's 95th-percentile lead time plus stay length has elapsed, which in this data is six to nine months. dim_traveller carries account_created_at_utc, first_booking_id, first_booking_created_at_utc and first_stay_completed_at_utc; fct_booking carries lead_time_days and booking_status. Design a weekly metric that is honest about what it can and cannot see, and state the condition under which acting on it is safe.

Approach

Split the two questions the weekly number is being asked to answer: is this week's cohort behaving differently, and what will this cohort finally do. Only the first is answerable weekly, and conflating them is the whole failure.
Publish a fixed-age metric rather than a raw rate: share of the cohort whose first booking reached booking_status = 'stayed' within 90 days of first_booking_created_at_utc, computed identically for every cohort, so every published value is comparable by construction rather than by luck.
Add the leading indicator that needs no trip to have happened: share of the cohort with any first booking, plus the p50 and p95 of lead_time_days on those bookings, because a cohort booking longer-lead trips shows a lower fixed-age rate with no behaviour change at all.
Pre-register the readout date for the true rate per cohort and mark every interim value as partial inside the artefact, in the row, not in a footnote nobody reads.
State the safety condition plainly: acting on the fixed-age number is defensible only when the cohorts being compared have similar lead-time mixes, because the 90-day cut truncates long-lead trips out of the numerator and does so unevenly.

Worked solution (25 min)

Cohort on first_booking_created_at_utc, not account_created_at_utc, and note the denominator difference the two produce.
Compute the 90-day fixed-age completion rate for every mature cohort and plot it against each cohort's final rate to see how much of the gap lead time explains.
Compute the p50 and p95 of lead_time_days per cohort and confirm they are stable enough for the fixed-age comparison to be safe.
Build the weekly artefact with three numbers per cohort week plus a per-cohort readout date, and mark incomplete cohorts in the row itself.
Write the one-sentence safety condition that accompanies every publication of the fixed-age rate.

Expected result: A weekly artefact carrying, per cohort week: first-booking rate (complete within days), 90-day first-stay completion rate (fixed age, comparable across cohorts), and lead_time_days at p50 and p95. Each cohort also carries a named calendar date on which its true completion rate becomes readable, and every incomplete cohort is marked partial in its own row.

Verify it yourself

For a fully mature cohort, the fixed-age rate must be less than or equal to its final rate, and the gap is exactly the long-lead trips the 90-day cut removes.
Two cohorts with identical fixed-age rates but different p95 lead times will not finish at the same final rate; calibrate that difference once on mature cohorts so the caveat carries a number.
Cohorting on account_created_at_utc instead of first_booking_created_at_utc changes the denominator by including accounts that never booked; compute both once and report the size of the difference.

Follow-up

The 90-day rate falls two weeks after a campaign launches in a long-haul market. Is that a product problem?
What is the shortest fixed age you would publish, and what determines the floor?

App conversion jumped while bookings stayed flat after a release

Easy · instrumentation · conversion-rate · platform-cut · release

Qualified intent-to-booking rate on app_ios rose from 4.1% to 5.6% the day after a client release. Desktop and mobile_web are unchanged, and app_ios bookings are flat in absolute terms. You have fct_search (search_id, trip_intent_key, visitor_id, searched_at_utc, device_type, results_returned_count, is_bot_flagged) and fct_booking (trip_intent_key, booked_at_utc). Decide whether this is a product win or a logging change, and say what you would tell the team that shipped it. Deliverable: a one-paragraph verdict plus the two numbers that settle it.

Approach

Stop looking at the ratio. Plot the numerator (distinct trip_intent_key with a booking within the 7-day window) and the denominator (distinct eligible trip_intent_key) as separate absolute series by device_type. A ratio that moves because one side collapsed is not a behaviour change, and the levels say which side it was.
If the denominator fell, find which searches stopped arriving. Cut fct_search row counts by device_type and by results_returned_count bucket at hourly resolution across the rollout, so the changepoint lines up with the deploy rather than with a calendar boundary.
Compute searches per trip_intent_key by device and week. Searches per intent is an interface property, not a demand property, so a release that removed a re-sort control or stopped logging refinements moves it first and moves it only on the affected platform.
Check whether the loss is selective. The denominator excludes results_returned_count = 0 by definition, so zero-result searches disappearing changes the denominator's composition as well as its size, and the two have different implications.
Exploit the rollout shape. A staged release leaves un-upgraded clients on the same days, which is the cleanest available comparison and costs nothing to construct.

Follow-up

Searches per intent fell from 9.2 to 5.8 on iOS only. What would make you call that a real interface improvement rather than lost events?
How would you make this metric robust to a logging change, so the next one is caught by the metric rather than by somebody noticing?
The release also rotated visitor_id per session. What does that do to trip_intent_key, and in which direction does it push this rate?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

How do you handle cases where an A/B test shows a positive result in one segment but a negative result in another?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Refuse the denominator that flatters a launch

Medium · integrity · occupancy denominators · supply withdrawal

A launch review is tomorrow. Occupancy computed on sellable nights, SUM(units_sold_to_date + units_available) from fct_rate_availability_snapshot, shows the launch market up 0.4pp. Computed on physical capacity, capacity_units from dim_supply_unit, it shows up 2.1pp, because individual_host suppliers blocked dates during the period and units_blocked is excluded from the first denominator but inside the second. The launch owner asks you to use the second, noting it is a documented convention. Prepare your response and what you put in the review document.

Approach

Confirm the mechanism before objecting, because the owner is right that both conventions exist. The difference here is not convention, it is that units_blocked grew during the measurement window, so the capacity-based number rises partly because supply was withdrawn rather than because more nights were sold.
Separate the two movements numerically: hold units_blocked at its pre-period level and recompute the capacity-based figure. Whatever remains of the 2.1pp after that is the real effect, and the difference is the withdrawal.
Reframe the blocking as a finding rather than an inconvenience. Hosts blocking dates during a launch is a supplier-side signal the launch owner needs, and it may matter more than the occupancy delta.
Put both numbers in the document with their denominators named in the row labels, plus the blocked-nights level as a separate line, so the reader can see the mechanism without being told which number to prefer.
Make the ask specific and small: pick one convention for this review, state it in the chart title, and never mix the two inside a single comparison. This is easier to agree to than a debate about which figure is right.

Follow-up

The owner says the blocking is seasonal and unrelated to the launch. How do you test that?
Your manager wants you to let it go because the difference is small. What do you do?
How would you stop this ambiguity reaching a review document next time?

Retract an occupancy comparison after the decision shipped

Hard · error disclosure · occupancy denominators · accountability

Three weeks ago you published a market occupancy comparison that led to marketing spend being moved out of one market. Reviewing the query, you find the two markets used different denominators: sellable nights from fct_rate_availability_snapshot (units_sold_to_date + units_available) for one, and capacity_units from dim_supply_unit for the other. The second market is mostly individual_host supply with large units_blocked, so its occupancy was understated. The spend move is already live. Prepare the correction: who you tell, in what order, and what the note says.

Approach

Quantify the error before telling anyone, because the first question will be 'how wrong'. Recompute both markets on sellable nights, which excludes units_blocked, and report the corrected gap and its sign, not just that the original was wrong.
Separate the numerical error from the decision error. The comparison was invalid, but the spend move may still have been right; establishing whether the decision would have flipped is a different analysis and it is the one the business needs.
Tell the decision-maker directly and first, before it appears in a dashboard or a peer surfaces it. Order matters because being told by a third party converts a mistake into a credibility problem.
Write the note with the correction, the size, the decision implication, and the reversal cost in that order. Three weeks of moved spend has a real cost to undo, and a correction that does not price the reversal forces the reader to do the work you skipped.
Name the specific control that would have caught it and put it in place in the same note: an assertion in the query that both arms use the same denominator expression, and the denominator named in the chart title. A retraction without a mechanism reads as an apology rather than a fix.

Follow-up

The corrected numbers still support the original decision. Do you still send the note, and does it read differently?
Your manager suggests quietly fixing the dashboard and not raising it. How do you respond?
What would you have had to do differently three weeks ago, in the query itself, rather than in your review habits?