Interview guides · Crédit Agricole

Crédit Agricole Data Scientist Interview Questions & Guide 2026

Case practice for Crédit Agricole data scientist candidates: scope a vague prompt into a measurable question before you analyze it.

The 60-second brief

As a Data Scientist at Crédit Agricole, you occupy a strategic position at the intersection of traditional banking excellence and modern digital transformation. Your work directly influences how the group leverages its vast datasets to refine risk assessment, personalize customer financial journeys, and optimize internal operational efficiency. You are not just building models; you are crafting the analytical backbone that supports a major international financial institution’s decision-making processes.

The boundary with engineering varies enough to be worth asking about directly. Some seats end at the analysis and the writeup; others carry feature pipelines, scheduling and alerting, which decides whether production data plumbing belongs in your prep at all.

Focus areas

Report only matured cohorts for loss metrics
Separate authorization, settlement and dispute outcomes cleanly
Reconcile amounts in minor units and currency

Overview

The role demands a balance of high-level technical proficiency and a deep understanding of the banking domain. You will work within multidisciplinary teams, collaborating with business managers and IT engineers to translate complex business challenges into scalable data solutions. Whether you are developing predictive models for credit scoring or uncovering insights from transaction patterns, your contributions have a measurable impact on the group's stability and service quality in an increasingly competitive landscape.

The Crédit Agricole interview process

Crédit Agricole candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Personality Assessment

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

HR Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Evaluations

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Where candidates lose points

Using written premium as the denominator of a loss ratio

Premium is written at inception and earned pro rata across the exposure period, so in a growing book written premium runs ahead of earned premium and the loss ratio comes out too low, with the error reversing when the book shrinks. The numerator has the mirror-image problem if it omits incurred-but-not-reported reserves, since recent accident periods then look profitable twice over. Both sides must refer to the same exposure period, which is what an accident-period view at a fixed development age enforces.

Reading the most recent months of fraud and dispute rates as final

Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.

Reading experiment results before checking the arm split

Compare observed arm counts against the intended allocation ratio, not an assumed even split, and set the alarm far below the conventional 0.05: at 0.05 roughly one healthy experiment in twenty trips it, which is why sample-ratio checks usually run at p < 0.001 or stricter. The test's power scales with sample size, so it misses a real diversion on a small experiment and fires on an imbalance too small to move the estimate on a very large one. A flag means go find the assignment or logging fault before reading any outcome, not report a mismatch.

Treating a non-significant result as proof of no effect

Say whether the confidence interval excludes the effect sizes you would have cared about. If it does not, the honest reading is that the test was underpowered, so report the minimum detectable effect the design could have found and what sample size would resolve it.

Questions & practice

Can you explain the logic behind the specific algorithm you chose for your past project?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Set a baseline first, so any model has something honest to beat.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Estimate a delinquency roll-rate matrix and project twelve months

Hard · roll rates · markov chain · survivorship

fct_loan_performance_monthly gives loan_id, as_of_month_end, months_on_book, delinquency_bucket, charge_off_flag, prepaid_in_full_flag and restructured_flag. Build a month-to-month transition matrix over the five delinquency buckets plus absorbing charged_off and prepaid states. Loans that stop appearing must be routed to an absorbing state rather than dropped. Project the current book forward 12 months by repeated matrix multiplication and report the projected share reaching charge-off. Handle restructured_flag explicitly, and name one place the Markov assumption fails on this data.

Approach

Build consecutive month pairs per loan by shifting as_of_month_end within loan_id, then verify the shifted value is exactly one month later. A gap is not a transition, it is an exit you have not resolved yet.
Resolve exits before counting anything. A loan whose last row carries charge_off_flag moves to charged_off, one carrying prepaid_in_full_flag moves to prepaid, and one that disappears with neither is a data question to raise rather than silently discard, because discarding it is survivorship that inflates every cure rate.
Count pairs into a 7 by 7 matrix and row-normalise. Assert every row sums to one and the two absorbing rows are the identity; a row that does not sum to one means exits were dropped.
Decide and state the restructure rule. Restructuring resets days_past_due, so a dpd_60_89 to current move on a restructured loan is not a cure. Either give restructured loans their own state or carry the pre-restructure bucket, but do not let that move land in the cure cell.
Project by taking the current bucket distribution as a row vector and multiplying by the matrix twelve times. Report the charged_off entry, and report it again from an all-current starting vector so the reader can see how much of the projection comes from loans that are already delinquent today.
State the homogeneity failure plainly: transition rates depend strongly on months_on_book, so one pooled matrix applied to a book with a young mix understates early-life delinquency. If the mix is moving, estimate separate matrices by seasoning band.

Follow-up

How would you validate the projection against what actually happened, and over what window?
The cure rate out of dpd_30_59 rose five points last quarter. What are the candidate explanations and how would you separate them?
When would you prefer a vintage curve to a roll-rate projection, and why?

Bootstrap a fraud loss rate that clusters within merchant

Medium · bootstrap · clustered resampling · heavy tails

You have a per-transaction frame with auth_id, merchant_id, settled_amount_reporting and net_loss_reporting, both already in one reporting currency. Most rows carry zero loss, a few carry large ones, and losses cluster within merchant. Using only numpy's random generator and no resampling helper from any library, write a bootstrap that returns a 95 percent interval for net fraud loss in basis points of settled volume, resampling merchants with replacement and taking all rows belonging to each drawn merchant. Also produce the naive row-level interval and state which you would report.

Approach

State the estimator before writing it: total net loss divided by total settled volume, times 10,000. It is a ratio of sums, so each replicate recomputes both sums. Averaging per-transaction loss rates instead would weight a five-unit transaction like a five-thousand-unit one.
Pre-aggregate loss and volume to merchant level once. For a ratio of sums, drawing merchants and taking all their rows is arithmetically identical to drawing merchant-level (loss_sum, volume_sum) pairs, so a replicate becomes one integer draw plus two vectorised sums rather than a groupby inside the loop.
Draw B replicates of M merchant indices with replacement, where M is the observed merchant count, compute the ratio per replicate, and take the 2.5th and 97.5th percentiles. Say explicitly that this is a percentile interval and that BCa would correct the skew-induced bias if the decision is close.
Repeat with independent row draws for the naive interval and compare widths on the same replicate count.
Report the clustered interval. Rows within a merchant share an acceptance profile, a category code and a fraud exposure, so they are not independent, and the row-level interval understates variance by roughly the design effect.

Worked solution (30 min)

Compute the point estimate directly on the full data and keep it for comparison.
Aggregate to merchant-level loss and volume arrays, record M, and set B to 2,000 with a seeded numpy Generator.
In a vectorised loop, draw integer indices of shape (B, M), index both arrays, sum along axis 1, and take the ratio times 10,000.
Repeat for the row-level version using the per-transaction arrays and N draws.
Take the 2.5 and 97.5 percentiles of each replicate array and report both intervals alongside the point estimate.

Expected result: Two intervals in basis points sharing the same point estimate. The clustered interval is materially wider, commonly two to four times, and is right-skewed because a heavy merchant can be drawn more than once in a replicate.

Verify it yourself

The point estimate equals the direct ratio on the full data and lies inside both intervals.
A synthetic input where every merchant holds exactly one transaction makes the clustered and naive intervals coincide; this is the test that the resampling unit is wired correctly.
Raising B from 2,000 to 10,000 shifts the endpoints by less than the reporting precision; if it does not, a handful of merchants is driving the tail and the estimate needs that caveat.
Replicate ratios never fall outside the range implied by drawing only the lowest-loss or only the highest-loss merchants.

Follow-up

Your clustered interval is three times wider. How do you explain that to someone who wanted a tighter number?
One merchant accounts for 40 percent of losses. What does that do to the interval, and what would you do about it?
How does this change if the question is whether two months differ rather than what this month's rate is?

How do you ensure your models remain performant when deployed in a production environment?

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Say which table is the grain you start from, and join outward from it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Please walk me through your process for optimizing a Python script that is running slowly.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Count-weighted and dollar-weighted approval rates on one currency

Easy · aggregation · denominators · minor units

Using fct_payment_authorization, report the trailing 7-day authorization approval rate two ways for transaction_currency = 'EUR': count-weighted, and dollar-weighted on amount_minor. Exclude is_reversal = true, exclude incremental authorizations (parent_auth_id not null), and exclude zero-amount account verifications. auth_result = 'approved' is the numerator; the four declined_* values make up the rest of the denominator. Return channel, attempts, approved_attempts, approval_rate_count and approval_rate_value. State every exclusion and its reason before you write the SELECT.

Approach

Say the denominator out loud first: attempts on a single transaction currency, excluding reversals, incremental authorizations and zero-amount verifications, because none of those is a purchase attempt a merchant is trying to get approved.
Filter requested_at against a half-open interval (>= start AND < end) so the boundary day is neither dropped nor double counted.
Compute both rates in one pass with FILTER clauses: COUNT() FILTER (WHERE auth_result = 'approved') over COUNT(), and SUM(amount_minor) FILTER (WHERE auth_result = 'approved') over SUM(amount_minor).
Cast one side of each ratio to numeric before dividing, since amount_minor and the counts are integers and integer division silently truncates to zero.
Group by channel and sort by the value-weighted rate, then read the gap between the two rates as a statement about where the declines sit rather than as noise.

Worked solution (20 min)

Write the exclusion list as comments above the query: is_reversal = false, parent_auth_id is null, amount_minor > 0, transaction_currency = 'EUR'.
Build a single aggregate query over fct_payment_authorization with a half-open requested_at predicate and those four filters.
Emit attempts, approved_attempts, approval_rate_count and approval_rate_value with FILTER clauses and a numeric cast on the numerator.
Group by channel, order by approval_rate_value ascending so the worst channel is on top.

Expected result: One row per channel present in the window, with attempts and approved_attempts as integers and both rates as numerics strictly between 0 and 1. approval_rate_value is typically lower than approval_rate_count on ecommerce and moto, because declines concentrate in higher-value card-not-present traffic.

Verify it yourself

SUM(attempts) across channels equals a separate COUNT(*) run with the same four filters and no GROUP BY.
Dropping the amount_minor > 0 filter raises attempts and moves approval_rate_count while leaving approval_rate_value unchanged, which confirms the zero-amount rows were only ever affecting the count-weighted figure.
Both rates are strictly less than 1 unless a channel had no declines at all in the window.

Follow-up

The two rates diverge by four points on the ecommerce channel but agree on card_present. What does that tell you, and what would you cut next?
How would you extend this to all currencies without summing amount_minor across them?
Which of the four decline reasons belong in the denominator of a rate you would put in front of a risk team, and which are really the network's problem?

Success metrics for loosening a fraud decline threshold

Medium · cost asymmetry · guardrails · cohort maturity

A risk team proposes lowering the risk_score cutoff that produces auth_result = 'declined_risk_rule'. Settled volume per active customer is the north star; net fraud loss in basis points of settled volume is the guardrail. The two move in opposite directions by construction. Specify the readout: primary metric, guardrail, the maturity window each is read at, and the decision rule agreed before launch. Show the expected-cost arithmetic that sets the cutoff using an average ticket of 200 units, a 1.5 percent contribution margin, 35 percent recovery on fraud losses, and 12 units of downstream value lost per false decline.

Approach

Refuse the two-metric framing and convert both sides into one currency. Approving a fraudulent transaction costs the amount net of recovery; declining a good one costs the forgone margin plus the downstream value of the customer's reaction. Decline when p times C_FN exceeds (1 minus p) times C_FP, so the break-even probability is p* = C_FP / (C_FP + C_FN).
Put the numbers in. C_FN = 200 times (1 minus 0.35) = 130, C_FP = 200 times 0.015 plus 12 = 15, so p* = 15 / 145 = 10.3 percent. Then show the threshold is amount-dependent: at a 2,000 ticket C_FN = 1,300 and C_FP = 42, giving p* = 3.1 percent, so a single global cutoff is already the wrong shape before any tuning starts.
State the precondition that makes this arithmetic legal: risk_score has to be calibrated, so that a score of 0.10 corresponds to an observed 10 percent fraud rate. A score that only ranks makes p* meaningless. Check the reliability curve before quoting any cutoff to anyone.
Set the maturity windows separately. Volume is readable within days, fraud loss is not, so the guardrail is read only on transaction months with at least 120 days of dispute maturity and the decision stays open until then, or a leading indicator is agreed in advance with its bias written down.
Agree the stopping rule before launch in the right units: revert if matured net fraud loss per unit of incremental settled volume exceeds the figure implied by p*. Fraud loss is supposed to rise when the cutoff loosens, so a rule that triggers on any rise is a rule that was never going to allow the change.
Report the swap set rather than portfolio totals: the transactions the new cutoff approves that the old one declined, and their realised loss rate. Portfolio aggregates dilute the change into invisibility.

Worked solution (30 min)

Compute p* at ticket sizes of 50, 200 and 2,000 with the given margin, recovery and false-decline cost, and tabulate them.
Bucket historical declined_risk_rule authorizations by risk_score decile and, for each bucket, write down what outcome data exists and what does not.
Write the readout spec: primary metric, guardrail, the 120-day maturity rule, the swap-set table and the numeric stopping rule.
Write the calibration precondition in two sentences and say how you would test it.

Expected result: A threshold table giving p* = 28.2 percent at a 50 ticket, 10.3 percent at 200 and 3.1 percent at 2,000, plus a readout spec whose guardrail is explicitly unreadable for 120 days and whose stopping rule is expressed as loss per incremental unit of settled volume rather than as a loss level.

Verify it yourself

p* falls as ticket size rises, because fraud loss scales with the amount while the false-decline cost is mostly a fixed relationship term.
Setting C_FP equal to C_FN returns p* = 0.5. If your formula does not, the numerator and denominator are the wrong way round.
The declined population carries no fraud labels, so any realised fraud rate quoted for the region below the old cutoff has to come from a randomised approval sample rather than from the historical table.

Follow-up

Fraud loss in basis points falls after launch. Name two ways that happens without any improvement in decisioning.
How do you keep observing outcomes in the region the rule still declines?
What changes if the 12 units of downstream value is a guess with no evidence behind it?

Judge a rate increase without letting price flatter the ratio

Hard · mix shift · adverse selection · loss development

A rating change raises premium on one product_line's renewal book. Using fct_policy_period_monthly (earned_premium_minor, written_premium_minor, exposure_units, paid_loss_minor, case_reserve_minor, ibnr_reserve_minor, loss_adjustment_expense_minor, rating_tier, policy_status, renewal_flag, policy_term_end), the accident-period loss ratio improves two points over the following year. Explain why that number alone cannot tell you the change worked, decompose the movement into the effects you would separate, and specify the primary metric and guardrails you would commit to before the next rating change.

Approach

Start with the arithmetic. A rate increase raises earned premium per exposure unit, so the loss ratio falls even if every policyholder behaves identically and every claim is unchanged. Part of the two points is mechanical and carries no information about risk at all.
Switch the risk measure to one price cannot move: pure premium, being incurred losses (paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, with loss_adjustment_expense_minor included or excluded consistently and the choice stated) divided by exposure_units, evaluated by accident period at a fixed development age. Flat pure premium alongside an improved loss ratio means the improvement was entirely price.
Decompose the loss ratio movement into three named components: the price effect at constant exposure and mix, the mix effect from which rating_tiers renewed and which lapsed, and the residual change in pure premium within tier. Only the third is evidence about risk selection, and it is usually the smallest.
Name the adverse-selection risk directly. Price sensitivity and loss propensity are not independent, and the policyholders most able to leave after a rate rise are often the ones worth keeping. Retention by rating_tier is therefore a guardrail with teeth, and it has to be read at tier level because a flat blended retention hides tiers moving in opposite directions.
Add prior-period reserve development as the second guardrail. The same two points are producible by setting case reserves or IBNR light, which surfaces only later as adverse development, so the reserve guardrail is what stops the primary metric being satisfiable by an accounting choice.
Commit the primary before the next change: underwriting margin per exposure unit, being earned premium minus incurred losses minus loss adjustment expense minus allocated expense, over exposure_units, at a fixed development age, reported by accident period and by rating_tier with exposure volume printed beside it so that improving margin by shrinking the book is visible in the same table.

Follow-up

Retention is flat overall but fell nine points in the lowest-loss tier. What do you expect next year's pure premium to do?
Why not use written premium as the denominator, and where would you still legitimately see it used?
How would you separate a genuine underwriting improvement from a year of mild weather?

A single day of settled volume comes in thirty percent low

Medium · pipeline failure · null rates · data freshness

Yesterday's settled volume is 30 percent below the same weekday across the previous eight weeks, while authorization counts and approved counts for the day look normal. You have fct_payment_authorization with requested_at, captured_at, captured_amount_minor, settled_at, settlement_amount_minor, settlement_currency, created_at and updated_at, plus warehouse load metadata. Decide within the hour whether this is a real drop or a load problem, and say what you would post to the on-call channel in either case.

Approach

Check the data before the business. Compare max(updated_at) and row counts per load partition for the affected day against the prior eight same weekdays; a truncated load usually shows as a count deficit confined to a specific hour range.
Measure null rates per column by requested_at date. Authorization fields present while settled_at and settlement_amount_minor are null points at the settlement feed; a uniform deficit across all columns points at ingestion.
Separate late arrival from loss. Settlement lands after authorization by design, so re-measure the same day 24 and 48 hours later. A deficit that closes on the next load is lag, not a drop.
Reconcile against the independent source: tie the day's settled total to the settlement ledger or acquirer file. If the ledger has the money and the warehouse does not, the business number was never wrong.
Only after those four steps consider a business explanation, and then only if a segment cut produces a coherent story rather than a flat haircut spread evenly across every merchant and channel.
Close by splitting the alert: a freshness and completeness check on the settlement feed, separate from the volume alert, so the next occurrence is classified automatically.

Follow-up

How would you make the daily volume alert immune to settlement lag without also hiding a genuine drop?
What is the right way to restate a published number after a backfill lands?
Which check catches a partial load that is uniform across the day rather than concentrated in one window?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Explaining a method to someone who will not follow the maths

Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.

How do you handle missing data or outliers in a large-scale financial dataset?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Can you describe a time you had to explain a complex technical result to a non-technical stakeholder?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.
Close with what you would do differently, concretely.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

Turn a one-line fraud-number request into a scoped brief

Easy · scoping · metric definition · denominators

A stakeholder messages: what is our fraud rate, and is it going up? You have fct_payment_authorization, fct_card_dispute and dim_customer. At least four defensible answers exist: count-weighted or value-weighted, attributed to the transaction month or to the dispute filing month, and gross or net of recoveries and successful representments. You get one reply before someone else produces an uncaveated number. Write that reply: the clarifying questions you ask, the single default you will produce if nobody answers, and what the default excludes.

Approach

Establish the decision behind the question first, because a risk-rule change, a board number and a merchant contract negotiation need different denominators, and asking which one is not stalling.
Offer a short menu rather than an open question: a stakeholder can choose between two named options but cannot specify a denominator from scratch.
Commit to a default so the reply is useful even if nobody answers, for example net fraud loss in basis points of settled volume, attributed to the requested_at month, matured months only.
State the exclusions in the same breath as the default: non-fraud dispute categories, transaction months with less than 120 days of maturity, and first-party abuse that arrives coded as consumer_dispute.
Give a delivery time for the default and a longer one for the fuller cut, so the choice between them carries a visible cost.

Follow-up

They come back wanting it by merchant for a contract negotiation. What changes in the definition and in the maturity rule?
How would you separate first-party abuse from third-party fraud in this data, and what would you refuse to conclude from the split?