REVOLVE Data Scientist Interview Questions & Guide 2026

The 60-second brief

At REVOLVE, a Data Scientist is not just a model builder; you are a strategic engine driving the intersection of fashion, technology, and e-commerce. REVOLVE relies heavily on data-driven decision-making to curate its vast inventory, forecast highly volatile fashion trends, and deliver hyper-personalized shopping experiences to millions of global customers. As a member of the data science team, your algorithms and insights directly impact the brand's bottom line, influencing everything from dynamic pricing and inventory management to automated marketing campaigns.

When randomisation is off the table, the skill being checked is naming an identification strategy together with the assumption it rests on: parallel trends for difference-in-differences, relevance and exclusion for an instrument, overlap and conditional ignorability for matching. Say the assumption out loud and say how you would try to break it.

Focus areas

• Join order, return and inventory grains without fan-out
• Separate true demand from stockout-censored sales
• Size promotions on incremental margin, not revenue

Overview

The scale and complexity of the data at REVOLVE make this role exceptionally challenging and rewarding. You will work with rich datasets spanning customer clickstream behavior, social media engagement signals, transaction histories, and digital apparel imagery. Your primary responsibility is to translate these complex data streams into actionable predictive models and business strategies that keep REVOLVE ahead of the fast-fashion curve.

This position requires a unique blend of deep technical expertise, business acumen, and creative problem-solving. Whether you are optimizing search relevancy on the platform, building recommendation engines, or predicting customer lifetime value, your work will directly shape how consumers interact with fashion online. Prepare for a fast-paced environment where innovation is highly valued and your models are put to the test in real-time market scenarios.

The REVOLVE interview process

REVOLVE candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

• Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
• Whether you name what you have not done instead of stretching to cover every line of the posting
• Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

• Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
• Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
• Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Dual-Assessment Screening

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

• Whether an ambiguous term becomes a specific column and filter before any computation happens
• Whether you read the schema for keys and cardinality rather than only for column names
• Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

• Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
• On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
• Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Take-Home Case Study

A take-home is graded as an argument, not as a notebook. Somebody reads the submission without you in the room, so every choice has to survive on the page: why the question was framed this way, and what was deliberately left out. The gap between a strong and a weak submission is almost never model quality. It is whether the writeup names the specific question it answers and commits to a recommendation, including what evidence would overturn it. A high-accuracy model attached to no conclusion reads as effort that stopped before the decision.

What to demonstrate

• Whether the question you answered is stated outright, and whether it is the question the prompt posed rather than an easier neighbour of it
• Whether the recommendation is specific enough to act on, with the uncertainty attached to it instead of parked in a caveats section at the end
• Whether analytical choices such as the metric definition, the population filter and the time window are justified in the prose, not merely visible in code

How to prepare

• Take a dataset you have already worked with, write the one-paragraph conclusion first, then check whether the analysis you were planning actually supports it and cut whatever does not
• Practise stating a metric in one sentence that fixes the population, the time window and the denominator, then confirm your query computes exactly that sentence and nothing adjacent to it
• Hand a draft to someone outside the problem and ask them to tell you back what you recommended and why; anything they cannot recover is not on the page yet

Live Coding Sessions

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

• Whether the query you write matches the plan you just described
• What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
• Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

• Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
• Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
• Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Where candidates lose points

Judging merchandising and recommendation changes on the surface they touch

Click-through or attributed revenue on a recommendation slot rises whenever the slot shows items the customer was going to buy anyway, so the surface metric measures capture rather than creation, and the units almost always come from a different slot, a search result or a later visit. The correct read is site-wide net revenue per session over a holdout, adjusted for returns, because surfacing more apparel or more discounted stock reliably moves both the return rate and the discount depth in the wrong direction while the click metric improves.

Crediting promotions and channels by last click, especially coupon codes

A discount code handed to a partner or an influencer is claimed by customers who were already at checkout, so last-click attribution books organic demand as partner-driven, and the reported return on that spend is an artefact of the code's visibility at the basket rather than of any persuasion. The same mechanism makes email and paid search look strong because they sit close to a purchase that other activity created. Incremental effects need a holdout, a geo split or a switched-off period; where none is possible, at minimum report the share of code redemptions from customers whose session began before any partner touchpoint, because that number alone usually collapses the claim.

Sizing estimates built on unnamed, unrevisable assumptions

Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.

Treating a non-significant result as proof of no effect

Say whether the confidence interval excludes the effect sizes you would have cared about. If it does not, the honest reading is that the test was underpowered, so report the minimum detectable effect the design could have found and what sample size would resolve it.

Questions & practice

How would you design a recommendation system for a fashion e-commerce platform with a high rate of inventory turnover?

Medium · statistics and probability

Approach

1. Sanity-check the answer against a simple bound or a simulated case.
2. Translate the result into the decision it informs, in one plain sentence.
3. Say what the estimate is of, and over what population it generalises.

Follow-up

• What sample size would you need to detect an effect half this size?
• Which assumption here is most likely to be violated in practice?

Explain the trade-offs between using a Random Forest model versus a Gradient Boosting machine for predicting customer churn.

Medium · machine learning and modelling

Approach

1. Frame the prediction: the label, the moment of prediction, and the action it triggers.
2. Say how the offline result would be validated online before it is trusted.
3. Check what information would not exist at prediction time, and exclude it.

Follow-up

• Where could label leakage enter this setup?
• What would you monitor after launch to know the model is still valid?

Correct a demand estimate for stockout-censored sales history

Hard · censoring · poisson exposure · lost sales · estimation

daily has 180 rows for one SKU at one node: snapshot_date, gross_units_sold, minutes_unavailable (0 to 1440), was_listed (bool), unit_paid_price_cents and is_promo. Recorded sales are the minimum of demand and what was sellable. Estimate the uncensored daily demand rate treating the available fraction of the day as exposure, then estimate the units lost to unavailability over the 180 days. Write the estimator yourself rather than calling a GLM library, and state the assumption the correction rests on.

Approach

1. Define exposure_d = was_listed * (1440 - minutes_unavailable) / 1440, in day-equivalents. A day with zero exposure had nothing sellable, so its gross_units_sold is 0, and it contributes 0 to both the numerator and the denominator of the closed-form rate below: it cannot move the estimate in either direction. Drop those days from the fit anyway, because the equivalent log-exposure offset form is undefined at log(0) and any GLM you later swap in will fail on them. Keep every one of them in the lost-units sum, where they carry the largest term.
2. Model sales as Poisson with mean lambda * exposure_d. For an intercept-only model with a log-exposure offset the maximum likelihood estimate has a closed form, lambda_hat = sum(sales) / sum(exposure), so no optimiser is needed; write that and derive it from the score equation rather than asserting it.
3. Estimate lost units as sum over all 180 days of lambda_hat * (1 - exposure_d), which is the demand the model says arrived while nothing was sellable. Convert to lost revenue only at the unit price actually in force on those days, and mark it as an upper bound.
4. State the assumption plainly: demand arrives at a constant rate within the day. It is wrong in the direction that matters, because a SKU that sells out does so after the day's demand has already concentrated in the earlier hours, so scaling by the available fraction understates true demand. End-of-day snapshots cannot distinguish a sellout at 09:00 from one at 23:00.
5. Extend to lambda_d = exp(b0 + b1is_promo + b2log(price)) * exposure_d if promo days are a material share, fitting by Newton or by iteratively reweighted least squares, and check the sign of the price coefficient for plausibility before trusting the lost-units figure.

Worked solution (40 min)

1. Build exposure, then report how many day-equivalents of exposure the 180 calendar days actually contain; if it is 150, a sixth of the history is unobservable.
2. Compute naive_rate = gross_units_sold.mean() over all 180 days and lambda_hat = gross_units_sold[exposure > 0].sum() / exposure[exposure > 0].sum(), then assert that dropping the exposure > 0 filter reproduces lambda_hat bit for bit, which is the statement that zero-exposure days carry no information about the rate.
3. Compute lost_units = (lambda_hat * (1 - exposure)).sum() over all 180 days, including the zero-exposure ones.
4. Bootstrap over days (resample the 180 day-rows with replacement, recompute lambda_hat) for a 95 percent interval on both the rate and the lost-unit figure.
5. Sanity-check by recomputing with minutes_unavailable forced to zero, which must return exactly the naive mean.

Expected result: lambda_hat = total units sold on positive-exposure days divided by total exposure in day-equivalents, which is greater than or equal to the naive daily mean over all 180 days, with equality only when there was no unavailability; lost_units equals lambda_hat times the total unavailable day-equivalents.

Verify it yourself

• lambda_hat / naive_rate equals 180 / exposure.sum() exactly, so lambda_hat >= the naive daily mean with equality only when the 180 calendar days hold a full 180 day-equivalents of exposure
• Setting minutes_unavailable to 0 and was_listed to True everywhere reproduces the naive mean exactly
• lambda_hat * total_exposure equals total observed units sold on the fitted days, which is the Poisson score equation holding
• The bootstrap interval on lambda_hat separates from the naive mean only when the exposure shortfall 180 / exposure.sum() - 1 exceeds roughly 2 / sqrt(total units sold), the relative Poisson standard error of the rate; report that comparison rather than assuming the two are distinguishable on a low-volume SKU

Follow-up

• Some of the lost demand moved to a substitute SKU or a different node. How would you bound the share that was genuinely lost rather than displaced?
• You only have end-of-day snapshots. What would you instrument to make this estimate honest, and at what data volume?
• The forecast that drove the buy was fitted on this same censored history. Walk through what happens over three consecutive seasons if nobody corrects it.

How would you handle missing values in a dataset containing customer demographic and purchasing behavior?

Medium · sql and data manipulation

Approach

1. Compute rates by summing numerator and denominator separately, never by averaging rates.
2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
3. State the window function and its partition and ordering out loud before writing it.

Follow-up

• How does the query change if the join becomes one-to-many?
• How would you verify this result without re-running the same query?

Write a Python function to find the first non-repeating character in a stream of customer search queries.

Medium · sql and data manipulation

Approach

1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
2. Compute rates by summing numerator and denominator separately, never by averaging rates.
3. Say which table is the grain you start from, and join outward from it.

Follow-up

• How does the query change if the join becomes one-to-many?
• How would you verify this result without re-running the same query?

Find consecutive stockout runs per SKU and node with gaps-and-islands

Medium · gaps and islands · window functions · inventory

From fct_inventory_snapshot (snapshot_date, sku_id, node_id, available_to_promise_units, was_listed, minutes_unavailable, forecast_units), find maximal runs of consecutive calendar days over the last 90 days where was_listed = TRUE and available_to_promise_units = 0, for each SKU and node. Return sku_id, node_id, run_start, run_end, run_length_days and forecast_units_lost, the sum of forecast_units over the run, keeping only runs of three days or more. The table is not guaranteed to contain a row for every SKU-node-day, nor at most one; state how you treat a missing day and how you treat a duplicated one.

Approach

1. Measure density and uniqueness before writing any window function. Per (sku_id, node_id), compare COUNT(), COUNT(DISTINCT snapshot_date) and the 90 days in the window. Rows fewer than distinct days means missing snapshots; COUNT() above COUNT(DISTINCT snapshot_date) means a SKU-node-day was written twice. They are different defects and they break different things.
2. Filter to the stockout condition, then flag a new island with LAG: new_run = (snapshot_date - LAG(snapshot_date) OVER (PARTITION BY sku_id, node_id ORDER BY snapshot_date)) > 1 OR LAG IS NULL.
3. Turn the flag into a group key with SUM(new_run::int) OVER (PARTITION BY sku_id, node_id ORDER BY snapshot_date ROWS UNBOUNDED PRECEDING).
4. Know what the arithmetic key does before choosing between them. Over the same filtered rows, snapshot_date - ROW_NUMBER() * INTERVAL '1 day' yields identical islands to the LAG running sum whenever snapshot_date is distinct within the partition: any break in the date sequence, whether the day is missing or merely in stock, shifts the arithmetic key and trips the LAG flag alike. The two diverge only on duplicated dates, where ROW_NUMBER advances while the date stands still: the key steps back a day, splitting one run into two that overlap on the duplicated date, whereas the LAG form sees a zero-day gap, keeps one island, and inflates COUNT(*) and forecast_units_lost instead. DENSE_RANK() OVER (PARTITION BY sku_id, node_id ORDER BY snapshot_date) makes the arithmetic key duplicate-safe; collapsing to one row per SKU-node-day first is cleaner still.
5. Decide and state the missing-day rule, because no island key decides it for you. Both forms end a run at a day with no row, which is the same as asserting the SKU was available that day. If a missing row instead means the snapshot job did not run, that day is unknown, and ending the run there shortens every incident that spans the outage.
6. Make censoring visible rather than assuming it away. Left join a generated date spine and record, per run, whether the day before run_start and the day after run_end carry a row at all. A run bounded by an observed available day genuinely ended; a run bounded by an absent day may be a fragment of a longer one, and the two must not be reported as the same thing.

Worked solution (30 min)

1. Run the density and uniqueness checks; record how many (sku_id, node_id) pairs have fewer rows than days in the window, and how many carry any duplicated snapshot_date.
2. Build the filtered CTE, add the LAG-based new_run flag and the running-sum island key.
3. Aggregate to one row per island: MIN and MAX of snapshot_date, COUNT(DISTINCT snapshot_date), COUNT(*) and SUM(forecast_units).
4. Add run_length_days = run_end - run_start + 1, confirm it equals the distinct-day count on every row, and treat any row where COUNT(*) exceeds it as a duplicated SKU-node-day rather than a longer run.
5. Left join the date spine at run_start - 1 and run_end + 1 and carry a censored flag onto the output.
6. Filter to run_length_days >= 3 and order by forecast_units_lost descending.

Expected result: One row per maximal run of three or more consecutively observed stockout days, ordered so the runs with the most forecast demand behind them surface first. run_length_days = run_end - run_start + 1 equals COUNT(DISTINCT snapshot_date) on every row by construction, since the island rule admits no internal gap, so that equality confirms the island logic and says nothing about data density. What it does catch is duplication: COUNT(*) above the span means a SKU-node-day was recorded more than once and its forecast_units were counted twice. Density shows up in the censored flag instead, as runs bounded by absent days rather than by observed available ones.

Verify it yourself

• run_end - run_start + 1 equals COUNT(DISTINCT snapshot_date) on every row. Where COUNT(*) exceeds it, the partition has duplicate snapshots and forecast_units_lost is overstated for that run.
• Runs for the same (sku_id, node_id) never overlap. Overlapping runs are the signature of the snapshot_date minus ROW_NUMBER key applied over duplicated SKU-node-days, which DENSE_RANK or a prior dedupe removes.
• Before the three-day filter, the summed distinct-day counts across islands equal COUNT(DISTINCT (sku_id, node_id, snapshot_date)) over rows meeting the stockout condition.
• Deleting one mid-run snapshot row on a test copy splits that run in two and can push both fragments under the three-day floor. The size of that shift is the exposure to snapshot outages, and it is why the censored flag ships beside the run length.
• Runs touching the first or last day of the 90-day window are censored by the window itself and are flagged the same way as runs bounded by absent days.

Follow-up

• A SKU was delisted mid-stockout, so was_listed flips to FALSE. Is the run over, and what does each choice do to the lost-demand number?
• How would you extend this to partial-day unavailability using minutes_unavailable rather than the binary zero-stock test?
• The same SKU is out at one node and in stock at another. Does that count as a stockout for the customer?

What statistical metrics would you use to measure the success of a new recommendation algorithm in an A/B test?

Medium · product sense and metrics

Approach

1. Fix the population and the time window before naming any metric.
2. Name one primary metric, then the guardrail that stops it being gamed.
3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

• How would you detect that the metric is being gamed rather than genuinely improving?
• What would you do if the primary metric and the guardrail moved in opposite directions?

How would you predict which fashion trends will emerge next season using social media data and historical sales?

Medium · product sense and metrics

Approach

1. Decompose the metric into the rates that drive it, and say which one you would check first.
2. Restate the decision this analysis has to support, and who acts on the answer.
3. Fix the population and the time window before naming any metric.

Follow-up

• How would you detect that the metric is being gamed rather than genuinely improving?
• What would you do if the primary metric and the guardrail moved in opposite directions?

How would you measure the incremental impact of influencer marketing campaigns on direct-to-consumer sales?

Medium · product sense and metrics

Approach

1. Name one primary metric, then the guardrail that stops it being gamed.
2. State what result would change your recommendation, so the answer is falsifiable.
3. Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

• Which segment would you cut first, and what would that rule out?
• How would you detect that the metric is being gamed rather than genuinely improving?

If customer lifetime value (CLV) is declining for a specific demographic, how would you investigate the root cause using data?

Medium · product sense and metrics

Approach

1. Name one primary metric, then the guardrail that stops it being gamed.
2. State what result would change your recommendation, so the answer is falsifiable.
3. Fix the population and the time window before naming any metric.

Follow-up

• What would you do if the primary metric and the guardrail moved in opposite directions?
• How would you detect that the metric is being gamed rather than genuinely improving?

Describe how you would design an experiment to test a new checkout flow, keeping in mind seasonal shopping spikes.

Medium · a/b testing and experimentation

Approach

1. Name the randomisation unit first; it decides the variance and what the test can detect.
2. State the primary metric and the minimum effect worth shipping, then size the test.
3. Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

• How would you handle interference between treated and control units?
• What would you conclude if the result is positive but the test is underpowered?

A ranking test that starves its own control arm

Medium · interference · switchback · cluster randomisation · stockout censoring

A ranking change promotes high-margin SKUs that have low days_of_cover in fct_inventory_snapshot. In a 50/50 visitor-level test, treatment converts 6 percent better. Inventory is pooled across arms: available_to_promise_units is shared at the node, so demand created by treatment raises minutes_unavailable and units_cancelled_oos for control visitors too. Explain why the 6 percent is biased and in which direction, name the evidence in fct_inventory_snapshot and fct_order_line that would confirm it, and design a version of the test whose estimate is not contaminated. State the analysis unit and what it costs.

Approach

1. Name the SUTVA violation precisely: a control visitor's outcome depends on treatment assignment of other visitors through the shared inventory pool, so the estimate is not the effect of the ranking change but the effect of the change plus the damage it does to the comparison group.
2. Fix the sign. Treatment depletes stock of the SKUs it promotes, control then sees those SKUs unavailable and converts worse than it would under full control, so the measured contrast overstates the true effect. A rollout to 100 percent removes the starved comparison and the lift shrinks.
3. Gather the evidence rather than asserting it: join fct_inventory_snapshot on the promoted SKU set and compare minutes_unavailable and units_cancelled_oos against the pre-period, and check whether control-arm conversion fell relative to its own pre-period rather than only against treatment. Check whether the lift concentrates in SKUs with low days_of_cover.
4. Redesign. The cheapest option restricts the promoted set to SKUs with days_of_cover above a threshold, which removes the interference but also removes the case the change was built for. Cluster randomisation on fulfilment node or market works only where inventory pools are genuinely node-partitioned and orders are node-local, which the fct_order_line to fct_inventory_snapshot join on (node_id, sku_id, snapshot_date) lets you verify.
5. Where the pool cannot be partitioned, use a switchback: apply one ranking to all traffic in randomised time blocks. Both arms then face the same inventory state within a block. Price it honestly: the analysis unit becomes the block, not the session, and the sample size collapses to the number of blocks.

Worked solution (40 min)

1. Quantify the interference: for the promoted SKU set, compute the change in SUM(minutes_unavailable) and units_cancelled_oos during the test relative to a matched pre-period, per node.
2. Split the treatment effect by days_of_cover decile; a lift concentrated in the lowest deciles is the signature of a starved control rather than a better ranking.
3. Design a switchback over 14 days with four six-hour blocks a day, randomising the ranking per block, giving 56 blocks and 28 per arm.
4. Compute the block-level MDE: with a between-block coefficient of variation of 12 percent on conversion, MDE_rel = 2.8016 * 0.12 * sqrt(2/28) = 0.090.
5. Discard the first 30 minutes of each block as burn-in for in-flight sessions and cluster the standard errors on block, or use randomisation inference over the block assignment.

Expected result: The 6 percent is an upper bound: shared inventory means treatment degraded control. A 56-block switchback over 14 days removes the contamination but detects nothing smaller than about a 9 percent relative change, so the measurable effect must be large or the design must instead partition inventory by node.

Verify it yourself

• Control-arm conversion compared with its own pre-period should be flat if there is no interference and down if there is; this test does not require the treatment arm at all.
• In a valid switchback, the pre-period block-level metric should show no difference between the block sets that will later be assigned to each arm.
• The between-block coefficient of variation used for power should be estimated from pre-period blocks of the same length, not assumed.

Follow-up

• Inventory depletes from one block into the next. How long would you make a block, what burn-in would you discard, and what does carryover do to the estimate?
• If you cluster on fulfilment node, how would you check that orders are actually node-local before trusting the design?

Revenue up six percent, margin per order down eleven

Hard · margin decomposition · mix shift · returns

Net merchandise revenue rose 6% month over month while contribution margin per delivered order fell 11%. You have fct_order_line (quantity, unit_list_price_cents, unit_paid_price_cents, line_discount_cents, discount_funding, promotion_id, unit_cost_cents, shipping_charged_cents, shipping_cost_cents), fct_return_line (refund_amount_cents, return_shipping_cost_cents, restocking_fee_cents, recovered_value_cents, disposition) and dim_sku (category_l1, is_own_brand). Deliverable: attribute the margin decline to named drivers measured in cents per delivered order that reconcile to the full 11%, and label each driver as a policy choice or a mix effect.

Approach

1. Write contribution per delivered order as an additive identity before touching data: net merchandise revenue, minus landed cost of units shipped, plus recovered value on returns, minus outbound and return shipping cost, minus payment fees, plus shipping charged and restocking fees, all over delivered orders. Compute every term for both months so the decline reconciles term by term with no residual bucket larger than the smallest named driver.
2. Split each term into rate and mix the same way as any blended metric. Discount per order is depth per unit times units per order; a category mix shift changes blended depth with no price change anywhere, and that distinction decides whether merchandising or pricing owns the answer.
3. Separate discount_funding before drawing any conclusion about discounting. Supplier-funded depth does not consume retailer margin, so a rise in promotional depth funded by suppliers is not a driver and must be excluded from the discount term.
4. Work the returns terms properly: attribute refunds back to the parent order's placed month rather than the refund month, and check recovered_value_cents per returned unit by disposition. A shift from restock_a_grade toward liquidate or destroy cuts margin with no change in return rate at all.
5. Work the freight terms: shipping_cost_cents per delivered order against shipping_charged_cents per delivered order. A lowered free-shipping threshold raises orders and revenue while turning charged freight into subsidised freight, which is exactly the pattern of revenue up and margin down.
6. Rank the drivers by cents per order, label each policy or mix, and state which single decision would recover the most margin and what it would cost in revenue.

Follow-up

• Your decomposition leaves a residual of a few cents per order. How do you decide whether to chase it or name it?
• The free-shipping threshold change was a deliberate test. What would you need to call it a win or a loss, and over what horizon?
• Finance's gross margin does not match your contribution figure. Which differences are expected by construction and which would worry you?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

• Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
• Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
• State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

• Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
• Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
• Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

• Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
• Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
• Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

• Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
• Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
• Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

• Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
• Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
• List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

• Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
• Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
• Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

• Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
• Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
• Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Getting an analysis acted on

Work that nobody used is a common and unflattering pattern in data careers, and interviewers probe for it. Have a story about an analysis that changed a decision, and be specific about how you got it in front of the person who could act. Also have one about work that went nowhere, with your reading of why.

Defend a promotion read that contradicts the revenue number

Medium · incrementality · pull-forward · stakeholder pushback

You analysed a site-wide twenty-percent weekend event. Revenue over the event week rose eleven percent year over year, contribution margin per delivered order fell, and the two weeks after the event ran below the pre-period baseline. The merchandising lead who owns the event calendar has already circulated the revenue figure. You have ten minutes at a planning meeting attended by that lead and their director. Deliver the finding and the recommendation you would make for the next event, and describe how you handle the pushback in the room.

Approach

1. Before the meeting, write the finding as one sentence about the event, not about the person who owns it: the event moved price and timing, not demand. Keeping the event owner's judgement out of the sentence is what keeps the holdout you want on the table.
2. Open with the number they already have, then decompose revenue into traffic, conversion, units per order, net price per unit and keep rate, and show which term moved. If price per unit fell and units per order held, the revenue gain was volume bought at a lower price.
3. Quantify the post-period dip as the pull-forward estimate, and state the measurement window explicitly. The window has to extend past the event because the demand was borrowed from the weeks after it.
4. Convert to incremental contribution margin. At gross margin m on list price and discount depth d, holding contribution flat requires a unit lift of d / (m - d), valid when m > d, when the discount applies to every unit sold, when unit cost is constant, and when nothing was cannibalised from full-price SKUs. Compare the observed lift to that threshold.
5. Say what would change your mind and propose the design that would settle it: a geo or customer holdout on the next event, with the number of units and weeks it needs.
6. End on the next event, not a verdict on the last one, so the meeting produces a decision rather than a defence.

Follow-up

• The lead says the event acquired new customers whose lifetime value justifies the margin loss. How would you test that claim with dim_customer and fct_order_line?
• Your director wants one number to take to finance. Which one, and what caveat travels with it?

Scope a one-line request for our best customers

Easy · scoping · stakeholder questions · definitions

A merchandising director messages you: can you pull our best customers. There is no other context. You have dim_customer, fct_order_line and fct_return_line. Best could mean highest net spend, highest contribution margin, most frequent, most recent, lowest return rate or highest expected future value, and the resulting lists differ enormously. You get one clarifying exchange before the director is in meetings for the rest of the day. Write what you send back, and describe how you proceed if no reply comes.

Approach

1. Ask about the decision, not the definition. What are you going to do with the list narrows six candidate metrics faster than asking which of six metrics they meant.
2. Put a stated default in the same message so a non-reply is still progress: trailing 365 days, customer grain, net of returns, and say so in one clause.
3. Make the ambiguity concrete with a measured number rather than a menu. One line reporting the overlap between the top decile by net revenue and the top decile by contribution margin forces the choice without a meeting.
4. Name the filters that depend on the use rather than on the metric. A mailing list needs email_consent = TRUE and excludes account_status in ('closed','fraud_blocked'); a buy-planning read wants neither filter.
5. If nothing comes back, ship the default with the definition, window and return treatment written at the top of the output, and attach the alternative cut so the conversation continues on an artefact.

Follow-up

• The director replies that it is for a loyalty upgrade offer. What changes in the query?
• How do you stop this becoming six different best-customer lists across the company?

Choose between three teams asking for the same week

Easy · prioritisation · stakeholder communication · reversibility

Three requests land on the same Monday. Merchandising wants a size-curve read before a buy deadline on Thursday. Growth wants a channel attribution rebuild that has been requested twice and dropped twice. Supply chain wants a stockout root-cause on a category that cancelled four thousand units last month. You have one week and no help, and each requester believes theirs is first. State what you do, in what order, and what you tell the two people who do not get the week.

Approach

1. Sort by the decision behind each request rather than by who sent it. A buy deadline is an irreversible commitment with a fixed date; an attribution rebuild changes no decision this week.
2. Ask each requester two questions: which decision changes, and what happens if the answer lands a week later. Those two separate a real deadline from felt urgency without arguing about either.
3. Look for the cheap partial before assuming any request consumes the week. If the size curve already exists at style grain, the merchandising read may be two hours rather than four days.
4. Decline explicitly with a start date attached instead of leaving a request in a silent queue. Growth has been dropped twice, so a third silent drop is a relationship cost you are choosing to pay; name it rather than incur it by default.
5. Escalate the collision upward once, with the three decisions and their dates side by side, so the tradeoff is resolved where it is owned rather than by whoever follows up hardest.

Follow-up

• Growth escalates to your manager saying analytics never supports them. What did you do before that happened, and what do you do now?
• The buy deadline moves to Tuesday. What do you cut from the size-curve read, and what do you refuse to cut?