Interview guides · AGCO

AGCO Data Scientist Interview Questions & Guide 2026

SQL practice for AGCO data scientist candidates: window functions, funnel and retention queries, joins that quietly drop rows.

The 60-second brief

This guide covers what a Data Scientist at AGCO is expected to do and how to prepare for the interview.

The boundary with engineering varies enough to be worth asking about directly. Some seats end at the analysis and the writeup; others carry feature pipelines, scheduling and alerting, which decides whether production data plumbing belongs in your prep at all.

Focus areas

Reconcile amounts in minor units and currency
Separate authorization, settlement and dispute outcomes cleanly
Set fraud thresholds by expected cost

The AGCO interview process

AGCO candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Phone Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Deep-Dive

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Team Fit Evaluation

This conversation decides whether you can be handed a problem nobody has finished defining and left alone with it for a few weeks. The manager is listening for how you behave when the brief is thin: what you clarify before starting, and what you settle on your own rather than escalating. Most candidates over-index on technical depth here and under-describe the decisions they actually owned. Say who wanted the work, what you chose not to do, and where you would have stopped and asked. A clean account of your own judgement carries this round further than a longer project list.

What to demonstrate

Whether you can name a decision that was yours alone, as opposed to one the team arrived at
How you respond to a request that arrives with no success metric attached to it
Whether the effort you estimate for a piece of work matches the work you just described doing
What you escalate, and how long you sit on a problem before you do

How to prepare

For each project you plan to raise, write one sentence saying what would not have happened if you had not been on it, and check that the sentence is about an outcome rather than an artefact
List the decisions in your last project that were genuinely yours, and for each one write down the option you rejected and why
Prepare the project that went badly: the point at which you knew, who you told, and what it cost before it was caught

Where candidates lose points

Recalibrating an underwriting cutoff on approved and funded applicants only

Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.

Counting authorizations instead of weighting them, and summing amounts across currencies

Declines skew toward high-value, cross-border and card-not-present transactions, so an unweighted approval rate can sit flat while approved value falls. Merchant retry logic also turns one declined purchase into several rows, inflating the denominator by an amount that varies by merchant and by decline reason. Amounts are held in the minor unit of the transaction currency and that unit is not always two decimals, since some currencies have none and some have three, so summing amount_minor across currencies produces a figure with no interpretation at all.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Over-explaining the method and under-explaining the implication

Lead with the answer and what you would do about it, then give the approach when asked. Roughly one sentence of method per three of implication is the right ratio for a stakeholder-facing answer; the interviewer already knows what a regression is.

Questions & practice

Bootstrap a fraud loss rate that clusters within merchant

Medium · bootstrap · clustered resampling · heavy tails

You have a per-transaction frame with auth_id, merchant_id, settled_amount_reporting and net_loss_reporting, both already in one reporting currency. Most rows carry zero loss, a few carry large ones, and losses cluster within merchant. Using only numpy's random generator and no resampling helper from any library, write a bootstrap that returns a 95 percent interval for net fraud loss in basis points of settled volume, resampling merchants with replacement and taking all rows belonging to each drawn merchant. Also produce the naive row-level interval and state which you would report.

Approach

State the estimator before writing it: total net loss divided by total settled volume, times 10,000. It is a ratio of sums, so each replicate recomputes both sums. Averaging per-transaction loss rates instead would weight a five-unit transaction like a five-thousand-unit one.
Pre-aggregate loss and volume to merchant level once. For a ratio of sums, drawing merchants and taking all their rows is arithmetically identical to drawing merchant-level (loss_sum, volume_sum) pairs, so a replicate becomes one integer draw plus two vectorised sums rather than a groupby inside the loop.
Draw B replicates of M merchant indices with replacement, where M is the observed merchant count, compute the ratio per replicate, and take the 2.5th and 97.5th percentiles. Say explicitly that this is a percentile interval and that BCa would correct the skew-induced bias if the decision is close.
Repeat with independent row draws for the naive interval and compare widths on the same replicate count.
Report the clustered interval. Rows within a merchant share an acceptance profile, a category code and a fraud exposure, so they are not independent, and the row-level interval understates variance by roughly the design effect.

Follow-up

Your clustered interval is three times wider. How do you explain that to someone who wanted a tighter number?
One merchant accounts for 40 percent of losses. What does that do to the interval, and what would you do about it?
How does this change if the question is whether two months differ rather than what this month's rate is?

Simulate false alarms in a merchant chargeback monitoring rule

Medium · simulation · rare events · monitoring thresholds

Baseline matured first-chargeback rate is 12 per 10,000 settled transactions. A monitoring rule alerts when a merchant's observed monthly rate exceeds twice baseline. For monthly settled transaction counts of 500, 2,000, 10,000 and 50,000, simulate the false-alarm probability per merchant-month under the baseline, and the power to detect a merchant whose true rate is 30 per 10,000. Then, for a portfolio of 4,000 merchants split 60, 25, 10 and 5 percent across those four counts, give the expected number of false alarms per month.

Approach

Recognise the rule is a threshold on an integer count, not on a continuous rate. At n = 500, twice baseline is 24 per 10,000, so the first observable value above it is 2 chargebacks, or 40 per 10,000. Derive the trigger count for every n before simulating anything.
Draw binomial counts with numpy at p = 0.0012 and take the share at or above the trigger for the false-alarm rate, then repeat at p = 0.0030 for power. Use at least 200,000 draws per cell so a probability near 0.001 has a usable standard error.
Cross-check every simulated cell against the Poisson approximation with lambda = n*p, which is tight here because p is tiny. A mismatch almost always means the trigger count is off by one.
Weight the per-merchant false-alarm probabilities by the portfolio mix, and report the share of expected alerts contributed by each size band rather than only the total.
Close on the operating consequence: a fixed multiplicative threshold is not a constant false-alarm rate across merchant sizes, so either the threshold scales with n or small merchants need a minimum volume before the rule applies.

Worked solution (30 min)

For each n, compute trigger = floor(2 * 0.0012 * n) + 1 and print the four values before simulating.
Simulate 200,000 binomial draws per n at p = 0.0012 and take the share at or above the trigger.
Repeat at p = 0.0030 and record power for the same triggers.
Compute the Poisson tail 1 - CDF(trigger - 1, lambda = n*p) for both p values and confirm agreement within Monte Carlo error.
Multiply the false-alarm probabilities by 2400, 1000, 400 and 200 merchants and sum.

Expected result: Trigger counts of 2, 5, 25 and 121. False-alarm probability roughly 12 percent at n = 500, roughly 10 percent at n = 2,000, under 0.1 percent at n = 10,000 and effectively zero at n = 50,000. Power against 30 per 10,000 roughly 44, 72, 85 and above 99 percent. Expected false alarms about 390 per month, with over 99 percent of them coming from the two smallest bands.

Verify it yourself

The four trigger counts are derived, not guessed: each is the smallest integer strictly above 0.0024n, giving 2, 5, 25 and 121. Derive them in exact or integer arithmetic, because 20.0012*10000 evaluates to 23.999999999999996 in binary floating point and floor-plus-one silently returns 24.
Simulated probabilities match the Poisson tail at lambda = n*p within about two Monte Carlo standard errors.
Power exceeds the false-alarm rate at every n; if it does not, the null and alternative rates are swapped.
Doubling the draw count moves each probability by less than its own simulated standard error.

Follow-up

How would you set a threshold that holds the false-alarm rate roughly constant across merchant size?
The rule reads the transaction month, but disputes arrive for up to 120 days afterwards. What does that do to the alert and how would you fix it?
What does a month of these false alarms cost, and how would you decide whether it is worth paying?

Collapse retry chains and compute a dollar-weighted approval rate

Medium · sessionisation · window functions · dollar-weighted rates

fct_payment_authorization gives auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, requested_at, auth_result, is_reversal, channel and issuer_country. Two reference frames give the minor-unit exponent per currency and a daily rate to one reporting currency. Collapse retry chains first: attempts sharing card_token_id, merchant_id and amount_minor whose consecutive gaps are under 15 minutes form a single attempt, whose outcome is its last row. Exclude reversals and zero-amount verifications. Return a 7-day rolling dollar-weighted approval rate by channel and issuer_country.

Approach

Filter before grouping: drop is_reversal rows and zero-amount verifications, since neither is a purchase attempt and both would otherwise sit in the denominator.
Sort by card_token_id, merchant_id, amount_minor and requested_at, take the gap to the previous row within that key, mark a chain start where the gap exceeds 15 minutes or the key changes, and label chains with a cumulative sum of that flag. This is a gap rule between consecutive attempts, not a fixed clock bucket, so a chain may span more than 15 minutes in total.
Keep each chain's terminal row by requested_at. If a retry was approved, the purchase was approved; keeping the first row reports the decline that caused the retry as the outcome.
Convert amounts exactly once: amount_minor divided by 10 to the power of the currency exponent, multiplied by the reference rate for the authorization date. Do not reach for settlement_fx_rate, which is null on precisely the declined rows the denominator needs.
Build the rolling window as a ratio of two rolling sums, approved value over total value, per channel and issuer_country. A rolling mean of daily ratios weights a quiet Sunday the same as a busy Friday.

Follow-up

The count-weighted rate is flat while the dollar-weighted rate falls 80 basis points. What do you look at first?
How would you choose the 15-minute window rather than inheriting it?
A merchant moves from two retries to five. Which of your two rates moves, and is that a real change in approval quality?

Write a query using SQL window functions to calculate a rolling average of machine downtime over the last 30 days.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Given two tables, one containing sensor logs and the other user metadata, how would you join them to analyze performance by region?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Reconcile captured authorizations against the daily settlement total

Easy · reconciliation · currency · rounding

fct_payment_authorization holds captured_amount_minor in transaction_currency, and settlement_amount_minor in settlement_currency with settlement_fx_rate applied at settlement rather than at authorization. The rate is quoted in major units of settlement_currency per major unit of transaction_currency, and dim_currency.minor_unit_exponent carries the ISO 4217 exponent for each code (0, 2 or 3 depending on the currency). Produce a daily reconciliation: for each settled_at date and settlement_currency, return settled_count, total settlement_amount_minor, and the sum of captured_amount_minor converted into settlement minor units. Flag any date and currency pair whose two totals differ by more than one minor unit per settled authorization. Do not sum amounts across currencies anywhere in the output.

Approach

Restrict to rows that actually settled: settled_at is not null and settlement_amount_minor is not null, which is a smaller population than captured rows because a capture can still be in flight.
Truncate settled_at to a date with an explicit time zone so the cut matches the ledger's cut, since settled_at is timestamptz and date_trunc on timestamptz silently uses the session time zone.
Join dim_currency twice, once on transaction_currency and once on settlement_currency, so both exponents are on the row. Minor units are not a common scale: a bare captured_amount_minor * settlement_fx_rate is correct only when the two exponents are equal, and a zero-decimal currency settling into a two-decimal one is wrong by a factor of 100.
Convert per row as ROUND(captured_amount_minor::numeric / POWER(10::numeric, exp_txn) * settlement_fx_rate * POWER(10::numeric, exp_settle)) — minor units to major in the transaction currency, apply the major-per-major rate, then back to minor units in the settlement currency. The collapsed form ROUND(captured_amount_minor::numeric * settlement_fx_rate * POWER(10::numeric, exp_settle - exp_txn)) is the same expression. Round per row and then sum, not SUM(...) * an average rate, because the rate varies row by row and rounding per row is what the settlement file did.
Group by the settlement date and settlement_currency together, never by date alone, and carry the currency into every output column name or row.
Compare the two totals with a tolerance scaled by settled_count, since per-row rounding accumulates linearly in the number of rows rather than being a fixed constant.

Worked solution (25 min)

Filter to settled rows and derive settlement_date from settled_at with an explicit time zone.
Join dim_currency on transaction_currency and again on settlement_currency to pick up exp_txn and exp_settle; fail the run if either is null rather than defaulting to 2.
Aggregate by settlement_date and settlement_currency: COUNT(*), SUM(settlement_amount_minor), and SUM(ROUND(captured_amount_minor::numeric / POWER(10::numeric, exp_txn) * settlement_fx_rate * POWER(10::numeric, exp_settle))).
Add a derived difference column and a boolean flag where ABS(difference) > settled_count.
Order by the flag first and then by settlement_date so the exceptions surface at the top.

Expected result: One row per settlement_date and settlement_currency, carrying settled_count, settlement_total_minor, converted_capture_total_minor, their signed difference, and a boolean exception flag. On clean days the difference is small relative to settled_count and has no consistent sign. Pairs with unequal exponents reconcile exactly as well as equal-exponent pairs; if they do not, the exponent terms are missing rather than the FX rate being wrong.

Verify it yourself

The difference divided by settled_count stays well under one minor unit on unflagged days, which is the expected size of per-row rounding.
Restricted to rows where exp_txn = exp_settle, the exponent-aware conversion and a bare captured_amount_minor * settlement_fx_rate agree exactly; restricted to rows where the exponents differ by 2, the bare form is smaller by exactly a factor of 100. That is the direct test that the dim_currency join is load-bearing.
Re-running with SUM over all currencies at once produces a number that changes when a zero-decimal currency's volume moves, which demonstrates why the cross-currency total is meaningless.
settled_count summed over all dates equals COUNT(*) on the same settled-row filter with no grouping.

Follow-up

A partial capture means captured_amount_minor is less than amount_minor. Where does that show up in this reconciliation, and where does it not?
On one currency pair the converted total is consistently about one hundredth of the settlement total, on every date, while the other pairs reconcile. Which two columns do you inspect first, and what single change fixes it?
The rate is documented as major-per-major. If a feed started publishing it minor-per-minor instead, which pairs would still reconcile and which would break?
How would you present a total across currencies to a finance partner who has asked for one number?

How would you measure the impact of an offline-to-online transition for a specific user journey?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you define the success metrics for a new digital feature on our machinery?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize which product features to analyze when faced with multiple requests?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

If we notice a sudden drop in a core product metric, walk me through your diagnostic process.

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

What are the most common experimentation pitfalls you have encountered in previous projects?

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

How do you determine the sample size required for an A/B test?

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

Estimate the marginal effect of approval at a score cutoff

Hard · regression discontinuity · instrumental variables · selection

Policy auto-approves applications with bureau_score at or above 660 and routes 640 to 659 to manual review, which approves about 35 percent of them. You cannot randomise approvals. Using fct_loan_application and a bureau-sourced panel that reports 12-month serious delinquency on any trade line for approved and declined applicants alike, estimate the effect of approval at the margin. State the identifying assumptions you would test rather than assert, the estimator and its tuning choices, and exactly what the estimate does and does not license.

Approach

Set the design up as a fuzzy regression discontinuity, because crossing 660 changes approval probability sharply but not from zero to one. Let Z = 1{bureau_score >= 660} be the instrument, approval be the treatment, and 12-month serious delinquency be the outcome. The estimate is the ratio of the jump in the outcome to the jump in approval probability, which is two-stage least squares with Z as the instrument and identifies a local average treatment effect for compliers at the cutoff.
Insist on the bureau panel as the outcome source. Funded loans exist on both sides of 660 only because manual review approves some applicants below it, and those are selected on whatever the reviewer saw; an outcome defined only on funded loans would reintroduce exactly the selection the design is meant to remove.
Test the assumptions instead of listing them. Check density continuity of bureau_score at 660 with a McCrary-style or local-polynomial density test, since brokers and applicants can sometimes trigger a bureau refresh and a heaped density at the threshold kills the design. Check that pre-determined covariates, including declared_annual_income_minor, channel, product_code and model_version, are continuous at the cutoff. Check policy_rule_hits and the pricing table for any other rule that fires at exactly 660, because a second discontinuity at the same point is not separable from the first.
Choose the estimator deliberately. Local linear regression with a triangular kernel and an MSE-optimal bandwidth, with bias-corrected robust confidence intervals. Do not fit a high-order global polynomial, which puts weight on observations far from the cutoff and produces artefacts at the boundary. Because bureau_score is an integer, the running variable has mass points, so cluster standard errors by score value or use an approach designed for discrete running variables.
Respect outcome maturity. Include only application cohorts with a full 12 months of bureau observation; a recent cohort with partial observation will look cleaner and will drag the estimate.
State the limits in the deliverable. The estimate is the effect at 660 on applicants whose approval status is determined by the threshold. It does not license moving the cutoff to 620, because both the first stage and the outcome relationship differ away from 660 and because a policy change at scale shifts the applicant mix. Report bandwidth sensitivity alongside the point estimate.

Worked solution (40 min)

Assemble the application-level panel: bureau_score, decision, funded status, and the bureau-sourced 12-month serious delinquency flag, restricted to cohorts with 12 full months of outcome observation.
Plot the first stage, being approval probability against bureau_score in one-point bins, and confirm a visible jump of roughly 0.65 at 660 (about 1.00 above against about 0.35 below).
Plot the reduced form, being delinquency against bureau_score in the same bins, and read the jump at 660 off a local linear fit on each side.
Estimate the Wald ratio as reduced form over first stage, equivalently two-stage least squares with Z = 1{score >= 660}, using a triangular kernel, an MSE-optimal bandwidth, bias-corrected robust intervals, and standard errors clustered by integer score.
Run the assumption battery: density continuity at 660, covariate continuity, placebo cutoffs at 650 and 670, and bandwidth sensitivity at 0.5x and 2x the selected bandwidth.
Write the result as a complier average effect at 660 with its interval, its bandwidth sensitivity table, and an explicit sentence on what it does not license.

Expected result: A fuzzy RDD estimate delivered as the Wald ratio, jump in 12-month serious delinquency divided by a first-stage jump in approval probability of about 0.65, fitted by local linear regression with a triangular kernel at an MSE-optimal bandwidth, with robust bias-corrected intervals and standard errors clustered on the integer score. It is reported as a local average treatment effect for compliers at 660 only, accompanied by the density test, the covariate-continuity table and the bandwidth sensitivity, with an explicit statement that it does not support moving the cutoff.

Verify it yourself

Reduced form divided by first stage must equal the two-stage least squares coefficient to numerical precision; if it does not, the sample or the bandwidth differs between the two fits.
Placebo cutoffs at 650 and 670 should show no jump in either the first stage or the outcome.
Pre-determined covariates should be continuous at 660; a jump in income or channel mix at the threshold means applicants or brokers are sorting and the design is compromised.
The point estimate should be stable when the bandwidth is halved and doubled; large movement means the local linear fit is being driven by observations far from the cutoff.

Follow-up

Manual reviewers below the cutoff see documents the score does not. What does that do to the monotonicity assumption, and how would you look for defiers?
You are offered a randomised approval band across 650 to 659 for one quarter. What does it buy you that the discontinuity does not, and how would you price the expected loss of running it?
The density test shows a modest pile-up just above 660. What are the possible mechanisms, and which of them leave the design usable?

Fraud losses appear to halve in recent transaction months

Medium · right censoring · development factors · maturity

A weekly chart attributes fct_card_dispute cases to the requested_at month of the linked fct_payment_authorization row. The two most recent months show the first-chargeback rate falling by half, and a risk rule shipped six weeks ago. Columns: dispute_id, auth_id, dispute_category, dispute_stage, opened_at, disputed_amount_minor, liability_shift_flag, outcome, net_loss_minor, resolved_at. Decide whether the rule worked, and produce the version of the chart you would sign your name to.

Approach

Separate the two dates explicitly. opened_at is when a case was filed, requested_at is when the transaction happened. Attributing by transaction month is the right causal choice and is exactly what makes the newest months structurally incomplete.
Measure the filing lag rather than assuming it: the distribution of opened_at minus requested_at over fully developed months, split by dispute_category, and the age at which around 95 percent of cases have arrived.
Build a development triangle of transaction month by months of development on cumulative case counts, and estimate age-to-age factors from the columns that are complete.
Develop the immature months with those factors and plot the result as an estimate with a visible band, kept visually distinct from the matured series rather than blended into it.
State the assumption the method needs: a stable development pattern across cohorts. A change in filing behaviour, merchant mix or the dispute team's own backlog breaks it, so inspect factor stability down each column before relying on the estimate.
Only then evaluate the rule, comparing pre-change and post-change cohorts at equal development age.

Follow-up

What leading indicator would you accept while the cohort matures, and what is its known bias?
How does liability_shift_flag change which disputes you should expect to see in the first place?
If the rule also blocked good transactions, where does that cost appear, and is any of it in this chart?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Rigour against a deadline

Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.

Explain the concept of statistical significance to a non-technical stakeholder.

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Explain an incomplete dispute chart to a non-technical executive

Easy · dispute maturity · stakeholder communication · right-censoring

A finance lead is looking at first-chargeback rate by transaction month, built from fct_card_dispute joined to fct_payment_authorization on auth_id and attributed to requested_at. The last three months slope sharply down and the lead wants to announce a fraud improvement at tomorrow's review. Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, so those months are not complete. In five minutes, with no statistics vocabulary, explain why the decline is not yet evidence and say exactly what you would put on the slide instead.

Approach

Lead with the mechanism in the listener's own terms, not with the statistical name for it: a dispute is attributed to the month the transaction happened, but it can be filed up to roughly 120 days later, so recent months contain only the disputes filed so far.
Show completeness rather than arguing about the rate: for each transaction month, plot the share of its eventual disputes already filed, estimated from months that are fully matured. The last three months will sit visibly below 100 percent.
Replace the chart with two artefacts: a matured series that stops 120 days back and is labelled final, and a development-factor estimate for the immature months drawn as a dashed range and labelled an estimate.
Hand over one sentence the executive can repeat without you in the room: the recent months look better because the disputes have not arrived yet, not because fewer will arrive.
Offer a weekly signal they can watch instead, such as the risk-score mix of approved volume or the decline-rule hit rate, and state up front what it does and does not predict.

Follow-up

The deck ships tomorrow regardless. What exactly goes on the slide, and what wording do you insist on?
How would you estimate the development factors, and how would you notice if they had shifted?

Retract a published number after finding a currency bug

Medium · error disclosure · minor units · process repair

Two weeks ago you published an interchange and fraud analysis that summed amount_minor across fct_payment_authorization without converting currencies. Minor units are not two decimals everywhere: some currencies carry none and some carry three, so the sum has no interpretation. A pricing decision is already in flight on the back of it. You now have corrected figures. Produce the retraction: what you send, to whom, in what order, and what you change in the process so this class of error is caught next time rather than trusted next time.

Approach

Size the error before announcing it, because saying the number is wrong without a magnitude and a direction forces every reader to assume the worst case.
Check whether the conclusion actually flips: if the ranking that drove the pricing decision is unchanged, that belongs in the first sentence beside the correction rather than buried at the end.
Tell the person acting on it first and directly, then the wider distribution, using the same text, so nobody learns about it secondhand.
Write the correction as four parts: the old number, the cause in one clause, the effect on the pending decision, and the new number. Leave out self-flagellation, which makes the reader do emotional work instead of acting.
Fix the class rather than the instance: a rule that a sum over amount_minor either groups by transaction_currency or passes through both conversion steps, exponent scaling and then a dated rate into one named reporting currency, plus a standing reconciliation of the settled subset to the settlement ledger inside each settlement_currency.

Follow-up

The corrected figures do not change the decision. Do you still send the correction, and what does that choice signal?
What automated check would have caught this, where would it live, and what would it cost in false alarms?