Interview guides · Amazon Robotics

Amazon Robotics Data Scientist Interview Questions & Guide 2026

Practice explaining analysis before a Amazon Robotics data scientist interview: the recommendation, the uncertainty, the follow-ups.

The 60-second brief

A Data Scientist at Amazon Robotics occupies a unique and highly impactful position at the intersection of hardware, software, and advanced analytics. Unlike traditional data science roles that focus purely on digital products, your work here directly influences the physical orchestration of thousands of autonomous mobile robots, robotic arms, and automated sorting systems across Amazon's global fulfillment network. You will be responsible for translating massive streams of real-time sensor, telemetry, and operational data into actionable insights that drive efficiency, reduce latency, and ensure safety in the physical world.

Seniority shifts the scope more than the words in the title do. Earlier-career loops mostly check that you execute a well-posed analysis correctly; senior loops check that you can decide which question is worth answering and defend what you chose not to do.

Focus areas

Separate true demand from censored, stocked-out sales
Price a service-level change in working capital
Trace an on-time miss to one node

Overview

The impact of this role is immense. Every algorithmic optimization you develop or predictive model you deploy can save seconds per package, which translates to millions of dollars in cost savings and faster delivery times for customers worldwide. You will work on highly complex, large-scale problems such as predicting hardware failures before they occur, optimizing robot fleet paths to prevent warehouse congestion, and simulating the impact of new robotic technologies on fulfillment center throughput.

At Amazon Robotics, you will collaborate closely with hardware engineers, software developers, and product managers to build the future of supply chain automation. It is a challenging, fast-paced environment that demands both deep technical rigor and the ability to think big. If you are excited by the prospect of seeing your code and models directly control physical machines and optimize global logistics, this role offers an unparalleled playground of data and technology.

The Amazon Robotics interview process

Amazon Robotics candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Screen

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Virtual Onsite Loop

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Sizing safety stock as z times sigma_D times the square root of lead time

That form assumes lead time is deterministic. When lead time itself varies, the standard deviation of demand over lead time is sqrt(L_bar * sigma_D^2 + D_bar^2 * sigma_L^2), and the second term dominates whenever supply is unreliable, so the familiar formula can understate the requirement severalfold for a long, variable inbound lane. Two further preconditions are routinely forgotten: demand is assumed independent across periods, which promotions and order batching break, and z maps to cycle service level (the probability of no stockout in a replenishment cycle), not to fill rate, which additionally depends on order quantity through the unit normal loss function. Quoting a z-derived number as a fill rate overstates achieved service, and the gap widens as order quantity shrinks.

Treating shipped units as demand

Shipments are censored at available inventory: on a stockout day the recorded quantity is a supply ceiling, not customer intent, and orders that were never placed because the item showed as unavailable leave no row at all. A forecast fitted on that history learns the constraint, under-forecasts the fast movers that stock out most often, and produces the replenishment that causes the next stockout, so the error compounds in one direction rather than averaging out. The fix is to model demand_qty rather than shipped_qty, flag stockout days with stockout_flag and treat them as censored (fit with a censored likelihood, or estimate unconstrained demand from uncensored periods and comparable locations), and to report how much of the history was censored alongside any accuracy number.

Solving silently instead of narrating the reasoning

Say which branch you are taking and why you chose it over the alternative, for example checking the denominator first because it changes what the comparison means. A correct answer that arrives with no visible path scores below a rigorous one that needed a hint.

Analysing at a different unit than the one randomised

Say out loud what was randomised (user, device, account, cluster) and make the analysis unit match, or account for the clustering with cluster-robust standard errors, the delta method, or aggregation up to the randomised unit. Randomising users and then running a test over sessions understates variance and inflates the false-positive rate.

Questions & practice

What is the difference between L1 and L2 regularization, and how would you decide which one to use when modeling warehouse throughput?

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Quantify uncertainty explicitly rather than reporting a point estimate alone.
Say what the estimate is of, and over what population it generalises.

Follow-up

How would you explain this result to someone who does not know statistics?
Which assumption here is most likely to be violated in practice?

Given an array of integers representing robot coordinates, write an efficient algorithm to find the two robots that are closest to each other.

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

Explain the trade-offs between precision and recall when building a predictive model to identify robotic parts that are about to fail.

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

As-of join snapshots to order lines without merge_asof

Medium · as-of join · fan-out · searchsorted

You have fct_order_line (order_line_id, sku_id, ship_from_location_id, actual_ship_at_utc, shipped_qty, shipment_id) and fct_inventory_daily (inventory_date, sku_id, location_id, available_qty). For every shipped line, attach available_qty from the latest snapshot strictly before the ship date for that exact sku and node, plus the shipment's delivery timestamp from fct_shipment_leg (leg_id, shipment_id, delivered_at_utc, leg_status). The output must have exactly one row per input order line. pandas.merge_asof is not available; implement the as-of match yourself.

Approach

Collapse fct_shipment_leg to one row per shipment_id before joining anything. A shipment is one or more legs, so a direct line-to-leg join multiplies every line by its leg count and quietly inflates every sum computed afterwards.
Define the shipment-level delivery as the maximum delivered_at_utc across its legs, and set it null if any leg is not delivered, since the customer has the order only when the last leg lands.
Sort the snapshot frame by (sku_id, location_id, inventory_date) and the line frame by the same key plus ship date, then use np.searchsorted with side='left' on the sorted dates within each key group to find the insertion point; the matching snapshot is the row at position minus one.
Guard the boundary explicitly: an insertion point of 0 means no snapshot exists before that ship date, and the result must be null rather than the first available future snapshot. Count those lines and report them.
Assert the output row count equals the input line count immediately after each merge, before any aggregation. Row-count drift is the only symptom a silent fan-out produces, and it is invisible once the frame has been grouped.

Worked solution (30 min)

Aggregate legs by shipment_id to produce delivered_at_utc as the max, leg_count, and a delivered_all flag from leg_status.
Left join that one-row-per-shipment frame onto fct_order_line by shipment_id and assert the row count is unchanged.
Sort snapshots and lines by the composite key, then compute per-group insertion points with np.searchsorted and subtract one.
Mask positions below zero to null, take available_qty at the remaining positions, and attach it as available_qty_prior.
Validate on a single sku-location pair by printing its snapshot dates and one line's ship date and confirming the chosen date by eye.

Expected result: A frame with exactly as many rows as the input order lines, carrying available_qty_prior (null for lines with no earlier snapshot), delivered_at_utc and leg_count, plus a printed count of unmatched lines.

Verify it yourself

Assert len(output) equals len(fct_order_line) after every join step, not only at the end.
Assert every non-null matched snapshot date is strictly less than the line's ship date, and that no snapshot date for the same key falls between the match and the ship date.
Compare against a slow brute-force loop over 200 sampled lines; the two must agree on every row.
Confirm the unmatched count equals the number of lines whose ship date precedes the first snapshot for their sku-location pair.

Follow-up

The snapshot is taken at the node's local end of day and actual_ship_at_utc is UTC. Where does that break the strictly-before rule and what do you need from dim_location to fix it?
How would you extend this to allocate each leg's linehaul cost back to the order lines it carried?
What changes if a line ships in two partial shipments?

Write a SQL query using window functions to calculate the moving average of robot idle times over a rolling 24-hour period.

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

How would you handle duplicate entries and missing sensor readings in a dataset tracking robot battery levels?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Write a SQL query to join a table of robot maintenance logs with a table of daily operational hours, calculating the total active hours between maintenance events for each robot.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Perfect order rate against the original promise in local time

Hard · timezone boundaries · order-grain rollup · semi-join

Compute weekly perfect order rate at order grain. An order qualifies only if every fct_order_line has line_status = 'shipped_full', every line's actual_delivery_at_utc falls on or before original_promised_delivery_date at the ship-from node's local end of day (dim_location.timezone holds IANA names), and no fct_shipment_leg reached through shipment_id carries exception_code in ('damage','address_issue') or leg_status in ('refused','lost'). The denominator is orders whose original_promised_delivery_date lands in the week. Undelivered lines fail. Publish with a fourteen-day lag and declare invoice accuracy as out of scope.

Approach

Convert the event into local time rather than comparing against a UTC-anchored deadline: (actual_delivery_at_utc AT TIME ZONE 'UTC' AT TIME ZONE loc.timezone)::date <= original_promised_delivery_date. Casting the promise date to a timestamp puts the cut-off at UTC midnight starting the promise day, which is up to a full day early and wrong in both directions depending on the node's offset.
Evaluate the three components as booleans at line grain, then roll to the order with BOOL_AND, coercing a NULL actual_delivery_at_utc to FALSE explicitly so an undelivered line fails rather than propagating NULL and dropping the order out of the comparison.
Test the shipment exceptions with EXISTS against fct_shipment_leg, never with a join. One shipment has many legs and one order has many lines, so a join multiplies the order into the denominator and changes the metric.
Aggregate to order grain first, then count qualifying orders over orders promised in the week. The rate is orders over orders; a line-level average is a different and easier metric.
Apply the publication lag by restricting to weeks whose promise dates are at least fourteen days old, and emit the three component pass rates beside the headline so any movement is attributable to completeness, timeliness or exceptions.

Worked solution (40 min)

CTE line_flags: join fct_order_line to dim_location on ship_from_location_id, producing is_full, is_on_time using the AT TIME ZONE conversion with COALESCE to FALSE, and has_exception from an EXISTS over fct_shipment_leg.
CTE order_flags: GROUP BY order_id with BOOL_AND(is_full), BOOL_AND(is_on_time), BOOL_AND(NOT has_exception) and MIN(original_promised_delivery_date) for bucketing.
Aggregate per DATE_TRUNC('week', promise date): COUNT() as denominator, COUNT() FILTER (WHERE all three) as perfect_orders, and the three component counts.
Restrict output to weeks ending at least fourteen days ago.
Re-run the on-time expression once with a naive UTC comparison and record the delta as evidence the conversion is live.

Expected result: One row per week with orders_in_denominator, perfect_orders, rate, and three component pass rates, where the rate is less than or equal to every component rate and weeks inside the fourteen-day lag are absent from the output.

Verify it yourself

perfect_orders is less than or equal to the count passing any single component, for every week.
Switching the deadline to a naive UTC comparison shifts the on-time component by roughly the share of orders at nodes offset from UTC. No change at all means the conversion is not being applied.
The denominator equals COUNT(DISTINCT order_id) over orders promised in the week and does not change when the legs EXISTS clause is removed.
No order_id appears in more than one week.

Follow-up

All three component rates are flat but perfect order fell two points. How is that arithmetically possible, and what does it tell you about where the failures now land?
A node sits at UTC+13 and many orders are promised on the last day of the month. What does that do to your weekly and monthly buckets?
How do you handle an order whose lines ship from two nodes in different timezones, and which node's clock owns the promise?

Explain how a hash map works under the hood, and describe how you would use it to optimize a search operation on a large dataset of package IDs.

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

Write a function to detect if a directed graph representing warehouse conveyor belts contains a cycle (which would cause a physical deadlock).

Medium

Approach

Work from the decision backwards to the evidence you would need.
Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Recover the effect of expediting from an automatic threshold rule

Hard · regression discontinuity · fuzzy rd · late · manipulation test

The transport system upgrades service_level to 'expedite' whenever projected slack at tender falls below 12 hours, and charges expedite_premium_cents for it. Projected slack is promised_delivery_at_utc minus the planned arrival implied by planned_transit_hours, and it is recoverable from two years of fct_shipment_leg. Finance wants to know what the premium buys. No randomised holdout is possible, and the rule has run unchanged throughout. Estimate the causal effect on on-time delivery, state which shipments your estimate applies to, and give the checks that would invalidate it.

Approach

Recognise the design: treatment is a deterministic function of an observed continuous running variable with a known cutoff, so a regression discontinuity identifies the effect at that cutoff with no randomisation, provided nothing other than the expedite rule changes at 12 hours.
Estimate with local linear regression on each side of the cutoff, a triangular kernel and an MSE-optimal data-driven bandwidth, reporting robust bias-corrected confidence intervals. Do not fit a global high-order polynomial: it imports weight from observations far from the cutoff and manufactures jumps that are artefacts of the functional form.
Test manipulation, because planners can edit the promise date and that moves slack directly. Run a density test of the running variable at the cutoff and inspect the histogram for heaping just below 12 hours. If slack is recorded to the whole hour, treat the running variable as discrete, cluster by its value, and consider a donut estimate that drops the bins adjacent to the cutoff.
Test continuity of everything that is not the treatment: weight_kg, cube_m3, mode, stop_count, lane and carrier should not jump at 12 hours. Add placebo cutoffs at 8 and 16 hours, where the effect should be indistinguishable from zero.
Handle overrides. If dispatchers sometimes expedite above the cutoff or decline below it, the design is fuzzy: estimate the jump in the probability of expediting and divide the outcome jump by it, equivalently two-stage least squares with the below-cutoff indicator as the instrument, and report the first-stage jump, because a weak first stage makes the ratio unstable.
Finish with the arithmetic Finance asked for: the jump in on-time probability at the cutoff against mean expedite_premium_cents per unit inside the bandwidth, expressed as cost per incremental on-time delivery and labelled as applying only near 12 hours of slack.

Worked solution (40 min)

Reconstruct the running variable at tender time from tendered_at_utc, planned_transit_hours and promised_delivery_at_utc, then verify it reproduces the observed expedite assignment on both sides of 12 hours.
Plot binned means of on-time delivery against slack with a data-driven bin width, and look at the picture before fitting anything.
Fit local linear RD with a triangular kernel and MSE-optimal bandwidth; report the estimate with robust bias-corrected intervals plus sensitivity at half and double the bandwidth.
Run the density test at the cutoff, the covariate-continuity checks, and placebo cutoffs at 8 and 16 hours.
Convert to cost per incremental on-time delivery using mean expedite_premium_cents per unit within the bandwidth.

Expected result: A local average treatment effect on on-time delivery for shipments with projected slack near 12 hours, reported with robust bias-corrected intervals, accompanied by the first-stage jump if the rule turns out to be fuzzy, and converted to a cost per incremental on-time delivery from the premium actually paid inside the bandwidth.

Verify it yourself

The assignment check should be near-deterministic; if a material share of legs above 12 hours are expedited anyway, the design is fuzzy and the sharp estimate is wrong by the size of the first stage.
The estimate should be stable when the bandwidth is halved and doubled; a sign flip means the functional form is driving the result.
Placebo cutoffs at 8 and 16 hours should return effects indistinguishable from zero.
Any jump at the cutoff in weight, cube, mode mix or carrier share means something besides the expedite rule changes at 12 hours, and the design fails.

Follow-up

The jump is 4 points at the cutoff. What would you need before recommending the threshold move to 18 hours?
Slack is recorded to the whole hour and 9% of legs sit exactly at 12. What does that heaping do to the estimate, and how do you handle it?

On-time rate improved the week the promise field changed

Medium · definition change · metric hygiene · release artefact

Customer on-time delivery jumped from 88 percent to 95 percent in one week and has stayed there. That week a release began writing re-confirmed dates into fct_order_line.promised_delivery_date, while original_promised_delivery_date still carries the first promise and both columns are populated for all history. Using fct_order_line (actual_delivery_at_utc, promised_delivery_date, original_promised_delivery_date, line_status) and dim_location.timezone, determine how much of the seven points is real. Deliverable: two numbers that sum to seven and one sentence of recommendation.

Approach

Recompute the whole series twice on identical rows, once against promised_delivery_date and once against original_promised_delivery_date. Both columns exist for all history, so the counterfactual is free and the split is arithmetic rather than an estimate.
Read the gap between the two series in the weeks before the release. If it was near zero and opens abruptly in the release week, the jump is the measurement changing. If the original-promise series also rises, that component is real and should be credited.
Measure the mechanism directly rather than inferring it: by week, the share of lines where promised_delivery_date > original_promised_delivery_date and the mean slip in days. Any metric that can be satisfied by moving its own target is not measuring service.
Handle the date-versus-timestamp comparison explicitly. actual_delivery_at_utc is a UTC instant while the promise is a local date, so convert using dim_location.timezone and compare against local end of day. Doing the comparison in UTC moves an entire timezone's deliveries across the boundary and invents a second, smaller step.
Recommend the original-promise definition for the published metric and keep the re-promise series as a separate operational measure, stating that the two answer different questions rather than that one is simply wrong.

Follow-up

Re-promises are legitimate when a customer agrees to the new date. How would you let those count without reopening the loophole?
What automated check would catch the next definition change before it reaches a chart?
Supplier inbound on-time has the same structure. Where is its equivalent loophole, and what does the metric tree already say about it?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Where your data came from

Data people depend on systems owned by other teams, and much of the job is negotiating for instrumentation, access, or a fix to a broken pipeline. Prepare an example of getting something changed upstream that you did not control. Describe what you asked for, what you traded, and how you worked while you waited.

Describe a situation where you disagreed with a colleague or manager on a technical approach. How did you handle the disagreement, and what was the outcome?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Explain forecast uncertainty to a non-technical general manager

Easy · communicating uncertainty · service level · executive communication

A general manager who does not use statistics must approve a 2.1 million dollar finished-goods build. Your recommendation rests on a demand forecast with a wide error band on the Z-class items and on a 97 percent cycle service target. You get two minutes, no formulas, and the question you will get back is whether the forecast is right. Deliver the explanation as you would say it aloud, including how you describe the error band in physical terms and what the 97 percent number does and does not promise.

Approach

Convert the error band into the units the approver already thinks in: weeks of cover, pallets, or dollars at risk, not a percentage or a confidence interval.
Answer the right-or-wrong question directly rather than deflecting: the forecast will be wrong, the size of the wrongness is what you measured, and the build is sized against that size.
State precisely what the 97 percent buys: it is a cycle service level, the probability of not running out during one replenishment cycle, so roughly three cycles in a hundred see a stockout. It is not the share of units shipped from stock. Unit fill rate also depends on the replenishment quantity, and when that quantity is large relative to the standard deviation of lead-time demand, which is the normal case outside lot-for-lot ordering, the fill rate sits above the cycle service number, often above 99 percent at a 97 percent cycle target.
Give the two-sided consequence in money: what the build costs to carry at the applicable cost of capital plus obsolescence risk on shelf-life items, against the margin at risk from the stockouts it prevents.
Close with the decision you want and the trigger that would reverse it, for example a lag-7 bias check after four weeks that reopens the number.

Follow-up

The GM says just give me one number. What do you give, and what do you refuse to give?
How would your answer change if the items were frozen with a 90-day shelf life?

Disagree with a planning manager about a safety stock cut

Medium · disagreeing with data · censoring · forecast evaluation

A planning product manager proposes cutting safety stock 30 percent on every C and Z class sku-location cell, citing a MAPE improvement from 41 to 28 percent over two quarters. Checking the query, you find MAPE is computed over fct_inventory_daily rows where the actual is greater than zero, and the actual used is shipped_qty rather than demand_qty; stockout_flag is true on 14 percent of the rows that were kept. You have fifteen minutes in his planning review. Make the disagreement, and propose what you would do instead of the flat cut.

Approach

Lead with the decision at risk, not the metric error: a 30 percent cut on intermittent items is the cheapest way to convert a measurement artefact into stockouts eight weeks later, by which time nobody will connect the two.
Name the two defects precisely. Dropping zero-actual rows is not a rounding choice, it removes most of the history for a C-class item at a forward-stocking location and it removes the rows where over-forecasting is penalised, so the surviving MAPE is biased toward whichever series stocked out. Using shipped_qty as the actual scores the forecast against a supply ceiling, so a series looks more accurate the more often it ran out.
Offer the replacement metric in the same breath: WMAPE, sum of absolute errors over sum of actuals against demand_qty, computed at the sku-location-week grain the ordering decision uses and at the lag it uses, reported next to signed bias so a persistently short series cannot hide inside an accuracy number.
Propose the smaller action that is defensible now: recompute on the corrected basis, rank cells by bias rather than accuracy, and cut cover only where the corrected series is unbiased or over-forecasting, in a staged rollout with a service guardrail.
Give him the win he actually wants: if the corrected numbers still support cuts on a subset, say so in advance, so the disagreement is about evidence rather than about territory.

Follow-up

He says demand_qty is itself incomplete because customers stop ordering what shows out of stock. Is he right, and what do you do about it?
Which items would you leave alone regardless of what the corrected metric says?