Interview guides · AgroStar

AgroStar Data Scientist Interview Questions & Guide 2026

AgroStar data scientist prep on data pitfalls: selection bias, target leakage, and checking an outlier's provenance before you drop it.

The 60-second brief

A Data Scientist at AgroStar sits at the intersection of complex agricultural challenges and data-driven innovation. As a key player in the organization, your work directly influences how AgroStar optimizes its supply chain, enhances farmer engagement, and refines its digital product offerings. You are not just building models; you are solving real-world problems that have tangible impacts on the agricultural ecosystem.

Seniority shifts the scope more than the words in the title do. Earlier-career loops mostly check that you execute a well-posed analysis correctly; senior loops check that you can decide which question is worth answering and defend what you chose not to do.

Focus areas

Decompose expected loss into PD, LGD, EAD
Read vintage curves, not blended portfolio averages
Report only matured cohorts for loss metrics

Overview

The role requires a blend of rigorous technical application and deep product intuition. You will be expected to translate ambiguous business requirements into actionable data strategies, ranging from designing experiments to diagnosing metric fluctuations. By leveraging large-scale datasets, you will provide the insights necessary to drive strategic decision-making across the company, ensuring that AgroStar remains at the forefront of digital agriculture.

The AgroStar interview process

AgroStar candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Screening

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Technical Assessments

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Take-Home Business Case

Before any modelling, the dataset is itself the first test. Take-home data usually carries something broken: rows duplicated at an unexpected grain, a join that silently drops part of the population, timestamps stored in more than one timezone, or missingness correlated with the outcome. An hour spent profiling row counts, key uniqueness and date ranges is not overhead, because it decides whether every number after it is real. What separates submissions is whether you report the defects you found and adapt the analysis to them, rather than modelling over them quietly and hoping the aggregate absorbs it.

What to demonstrate

Whether you established the grain of each table and checked row counts after every join, and said so in the writeup
Whether data defects you found are surfaced with their effect on the conclusion, instead of being dropped without comment
Whether filters and exclusions are reproducible from the submitted code, with the size of the excluded population quantified

How to prepare

Write a short profiling script you can point at any unfamiliar table: row count, distinct key count, null rate per column, and the min and max of every date field, then run it before anything else
Write the funnel or the join chain as one query and check the row count at each grain, so a silent fan-out shows up as a number rather than as a wrong answer later
On a past dataset, list every exclusion you applied and how many rows each one removed, then draft the single sentence about it you would put in a report

Where candidates lose points

Recalibrating an underwriting cutoff on approved and funded applicants only

Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.

Counting authorizations instead of weighting them, and summing amounts across currencies

Declines skew toward high-value, cross-border and card-not-present transactions, so an unweighted approval rate can sit flat while approved value falls. Merchant retry logic also turns one declined purchase into several rows, inflating the denominator by an amount that varies by merchant and by decline reason. Amounts are held in the minor unit of the transaction currency and that unit is not always two decimals, since some currencies have none and some have three, so summing amount_minor across currencies produces a figure with no interpretation at all.

Treating a non-significant result as proof of no effect

Say whether the confidence interval excludes the effect sizes you would have cared about. If it does not, the honest reading is that the test was underpowered, so report the minimum detectable effect the design could have found and what sample size would resolve it.

Reporting a p-value with no effect size or interval

Give the estimated difference with a confidence interval in the units the business cares about, then say whether that whole interval is worth acting on. A p-value only addresses whether you can rule out exactly zero; it says nothing about magnitude.

Questions & practice

Build a vintage delinquency table without pivot or unstack

Easy · vintage analysis · cohorts · groupby · cumulative max

fct_loan_performance_monthly gives loan_id, origination_month, months_on_book, days_past_due, charge_off_flag and restructured_flag. Produce a DataFrame with one row per origination_month and columns for months_on_book 0 through 12, each cell holding the share of that vintage's funded loans that had ever reached 90 or more days past due, or charge-off, by that age. You may not use pivot, pivot_table, crosstab or unstack. Cells for ages a cohort has not yet reached must be NaN rather than zero.

Approach

Define the per-row indicator as days_past_due >= 90 or charge_off_flag, then take a cumulative maximum of it per loan ordered by months_on_book, because the metric is reached-by-age-m, not in-that-state-at-age-m.
Deal with restructuring before the cumulative max. Restructuring resets days_past_due, so a restructured loan re-enters at current and, without the cumulative maximum carrying its pre-restructure worst state, reads as a cure.
Fix the denominator once as the count of distinct loan_id per origination_month across the whole cohort. Prepaid and charged-off loans stop producing rows, so a denominator recomputed at each age silently shrinks exactly where losses land.
Aggregate with groupby(['origination_month','months_on_book'])['ever_90'].sum(), then pre-build the output frame indexed by sorted origination months with integer columns 0 to 12 and assign from the grouped Series by .loc on its index.
Mask cells beyond each cohort's maximum observed months_on_book so an immature cell reads NaN instead of an artificially low rate.

Follow-up

Two adjacent vintages diverge at months_on_book 6. How would you separate seasoning, mix shift and a genuine credit-quality change?
The three most recent vintages look best on this table. What do you check before saying so?
How does the table change if charge-off policy moved from 180 to 120 days past due partway through the series?

Write integrity checks for the authorization and settlement lifecycle

Easy · data quality · minor units · fx reconciliation

You are given fct_payment_authorization as a pandas DataFrame with auth_id, requested_at, amount_minor, transaction_currency, auth_result, decline_reason_code, is_reversal, parent_auth_id, captured_at, captured_amount_minor, settled_at, settlement_amount_minor, settlement_currency and settlement_fx_rate. Write a function returning one row per integrity check with the check name, failing row count, failing share and up to five example auth_id values. Cover at least six checks, one of which reconciles captured_amount_minor against settlement_amount_minor through settlement_fx_rate. Partial capture, zero-amount verification and a decline with no capture are all legitimate and must not be flagged.

Approach

Separate contract violations from observations before writing any code: an approved row carrying a decline_reason_code is structurally impossible, while a capture two days after requested_at is merely slow and belongs in a different severity tier.
Express each check as a boolean mask over the whole frame and collect the masks in a dict, so the summary table is one comprehension over mask.sum() rather than a row loop.
For the reconciliation, leave minor units before comparing: expected = captured_amount_minor / 10exponent[transaction_currency] * settlement_fx_rate * 10exponent[settlement_currency]. Build the exponent table covering zero-decimal and three-decimal currencies instead of assuming two everywhere.
Guard the legitimate cases explicitly so each mask fires only on the genuine contradiction: captured_amount_minor below amount_minor is partial capture, amount_minor of zero on an approved row is account verification, a null captured_at on a declined row is correct.
Sort the output by failing share times a stated severity weight, because a check firing on 0.01 percent of rows can still be the one that breaks a ledger reconciliation.

Worked solution (25 min)

Assert auth_id is unique, then build a currency exponent lookup that includes the zero-decimal and three-decimal currencies present in the data.
Define masks for: approved with non-null decline_reason_code; declined with non-null captured_at; captured_amount_minor above amount_minor with parent_auth_id null; captured_at before requested_at; settled_at before captured_at; is_reversal true with parent_auth_id null; settlement_currency differing from transaction_currency while settlement_fx_rate is null.
Add the exponent-aware reconciliation mask with a tolerance of one minor unit plus a small relative term.
Assemble a frame of check_name, n_failing, pct_failing and up to five sample auth_id values, ordered by severity then share.
Read five flagged rows per check by hand and confirm each is genuinely contradictory before reporting any counts.

Expected result: A DataFrame of at least eight rows, one per check, each with n_failing, pct_failing and example auth_id values. The reconciliation check should fire on a whole currency at once rather than on scattered rows, because an exponent error is systematic while an FX error is not.

Verify it yourself

Every mask returns false on three hand-constructed legitimate rows: a partial capture, a zero-amount verification and a decline with no capture.
Each pct_failing uses the denominator that check actually applies to, with reversals and verifications excluded where they are out of scope.
The reconciliation residual is centred at zero with spread under one minor unit; residuals clustered at exactly 100x or 0.01x mean an exponent is wrong, not a rate.

Follow-up

Which of these would you run as a blocking pipeline assertion and which as a monitored metric, and why?
The FX check fails on 3 percent of rows, all in one settlement currency. How do you decide between a data bug and a rounding convention?
How would you detect that a currency's minor-unit exponent is wrong in your reference table, using only the transaction data?

Measure calibration of a twelve-month default probability from scratch

Hard · calibration · brier score · binning

fct_loan_application gives application_id, model_pd_12m, model_version, decision, funded_at and loan_id. fct_loan_performance_monthly gives loan_id, months_on_book, days_past_due and charge_off_flag. Define the outcome as ever 90 or more days past due, or charged off, by months_on_book = 12. Without sklearn or scipy, build an equal-count binned reliability table, the expected calibration error, the Brier score and its reliability, resolution and uncertainty components, and report the residual the binned identity leaves behind. Restrict to cohorts that have actually reached 12 months on book.

Approach

Build the label first and name the population it covers out loud: only funded loans have outcomes, so this measures calibration on the approved population. The declined region is unmeasured, and no binning scheme repairs that.
Restrict to applications whose loans have reached months_on_book = 12. A cohort observed at 8 months has a mechanically lower default rate and will read as systematic over-prediction that is really just immaturity.
Bin by equal count, deciles of model_pd_12m through a rank-based cut, not equal width. The PD distribution is heavily right-skewed, so equal-width bins put most of the mass in the first bin and leave the risky bins with single-digit counts whose observed rates mean nothing.
Per bin compute n, mean predicted, observed rate, and the binomial standard error sqrt(o(1-o)/n) so a gap can be read against noise. ECE is the count-weighted mean absolute gap between mean predicted and observed.
Compute Brier directly as the mean squared error, then reliability = sum of n_k (pbar_k - obar_k)^2 over N, resolution = sum of n_k (obar_k - obar)^2 over N, uncertainty = obar(1 - obar). Report residual = Brier - (reliability - resolution + uncertainty). That identity is exact only for discrete forecasts, so with binned continuous scores the residual is the within-bin spread of the score; a large one means the bins are too wide to support the decomposition.
Split by model_version. A mixed-version population can look well calibrated in aggregate while each version is biased in opposite directions.

Follow-up

AUC is unchanged after a population shift but the reliability curve has moved. What happened, and what do you do about it?
How would you recalibrate without retraining, and what would you check afterwards?
The top decile shows observed default well above predicted. Is that a calibration problem or a policy problem?

How would you handle missing data when joining two large tables in a distributed environment?

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Say which table is the grain you start from, and join outward from it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Write a query to identify the top 10% of users based on their transaction frequency using window functions.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Collapse merchant retry chains before measuring authorization approval

Medium · window functions · gaps and islands · sessionisation

Merchant retry logic turns one declined purchase into several rows in fct_payment_authorization. Collapse attempts into chains: inside a partition of (card_token_id, merchant_id, amount_minor), consecutive rows whose requested_at gap is 15 minutes or less belong to the same chain. Return one row per chain with chain_started_at, attempt_count, amount_minor, channel, and chain_outcome set to 'approved' when any attempt in the chain was approved. Then report a 7-day rolling dollar-weighted approval rate by channel over chains rather than attempts. Exclude is_reversal rows.

Approach

In a CTE, take LAG(requested_at) OVER (PARTITION BY card_token_id, merchant_id, amount_minor ORDER BY requested_at) and set is_new_chain when the previous value is null or the gap exceeds 15 minutes.
Turn the flag into a chain identifier with SUM(is_new_chain::int) OVER (same partition, same order, rows unbounded preceding to current row), which is the standard gaps-and-islands construction.
Aggregate to one row per chain: MIN(requested_at), COUNT(*), and BOOL_OR(auth_result = 'approved') as the chain outcome, because the purchase succeeded if any attempt in the chain did.
Roll the chains to a daily grain per channel, then apply the rolling window as PARTITION BY channel ORDER BY chain_date RANGE BETWEEN INTERVAL '6 days' PRECEDING AND CURRENT ROW. An interval-offset RANGE frame is value-based: it admits every row in the partition whose chain_date falls in the closed window [D - 6 days, D], so a channel with no chains on some dates still gets the right seven-day sums — sparsity is exactly what this construct tolerates. ROWS BETWEEN 6 PRECEDING AND CURRENT ROW is the one that needs a dense grain, because it counts rows and on a sparse grain reaches back further than seven days.
Left-join a dense date spine only if the output needs a row on days with no chains. That is a presentation requirement, not a correctness one, and it leaves every rolling sum unchanged. The RANGE frame's preconditions are that chain_date is a date or timestamp (an interval offset needs an ordering column it can be added to) and that the aggregation left one row per (channel, chain_date).
Divide approved chain value by total chain value inside the rolling frame, never the average of daily rates, since averaging rates weights a quiet day the same as a heavy one.

Worked solution (40 min)

CTE 1: filter out is_reversal rows, then compute prev_requested_at with LAG and the is_new_chain flag.
CTE 2: build chain_id with a running SUM of the flag over the same partition and order.
CTE 3: group by partition keys plus chain_id to emit chain_started_at, attempt_count, amount_minor, channel and chain_outcome.
CTE 4: aggregate chains to (chain_date, channel) with approved value and total value. Add the date spine here only if the report must show empty days.
Final SELECT: two rolling SUMs over PARTITION BY channel ORDER BY chain_date RANGE BETWEEN INTERVAL '6 days' PRECEDING AND CURRENT ROW, divided at the end.

Expected result: A rolling series with one row per date and channel, where the chain count is strictly below the attempt count on ecommerce and recurring, and the chain-level approval rate is at or above the attempt-level rate for every channel, since collapsing removes failed attempts that later succeeded.

Verify it yourself

SUM(attempt_count) over all chains equals the filtered attempt row count exactly; if it does not, the chain identifier is not partitioned the same way as the LAG.
Every chain has exactly one row whose is_new_chain flag is true, so COUNT(DISTINCT chain_id) equals the number of flagged rows.
A chain whose attempt_count is 1 must have chain_outcome equal to that single row's approved status.
The rolling denominator on day D equals the sum of the daily denominators over every date present between D-6 and D inclusive — seven terms on a dense grain, fewer on a sparse one, and the same total either way.
Adding or removing the date spine leaves every rolling sum identical, because spine rows contribute zero. Swapping RANGE for ROWS changes them on any channel whose grain has a gap, which is the direct test of which construct tolerates sparsity.

Follow-up

This definition chains off the previous attempt, so a chain can span an hour if attempts arrive every 14 minutes. When is that right, and when would you anchor the window to the first attempt instead?
Two genuine purchases of the same amount at the same merchant, four minutes apart, collapse into one chain. How large is that error and how would you bound it?
A channel goes dark for three days mid-window. Which of the two frame types would silently widen, and what would the resulting rate look like?
What changes if the merchant retries with a slightly different amount to dodge a velocity rule?

How do you prioritize your work when you have multiple competing requests from different departments?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you balance short-term engagement metrics with long-term retention?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you design a metric to measure the success of a new farmer-facing feature?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

If we notice a sudden drop in daily active users on our platform, how would you diagnose the root cause?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you handle a situation where an A/B test shows a positive impact on one metric but a negative impact on another?

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Say whether units interfere with each other, and switch design if they do.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

How do you determine the sample size required for an A/B test?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

Price a false decline when the label does not exist

Hard · proxy metrics · selection bias · randomised holdout

Your expected-cost threshold needs a figure for what a false decline costs beyond the margin on the blocked transaction. fct_payment_authorization records the decline, nothing records what the customer did next elsewhere, and declined transactions never produce a fraud outcome, so neither side of the error is directly observable. Propose the measurement: the proxy you would build from the tables you have, the design that yields an unbiased estimate for at least part of the score range, the bias in each, and the single sentence you would attach to the number when it reaches a pricing decision.

Approach

Write down what is unobservable and why. The counterfactual spend of a customer who was not declined, and the fraud label on any transaction the rule blocked, are both missing because of the decision itself. Missingness that depends on the decision is not fixed by matching on observed covariates.
Build the observational proxy anyway and be specific. For customers receiving a first risk-rule decline in a window, compare settled volume and active status over the following 30 and 90 days against customers matched on pre-period settled volume, tenure, segment and channel mix who attempted a comparable transaction and were approved. The bias runs one way: matching conditions on having attempted something that scored near the cutoff, and part of the declined group are genuine fraudsters whose disappearance is a saving rather than a loss, so the estimate overstates the damage.
Buy one unbiased local estimate. Hold a small random share of authorizations inside a defined risk_score band out of the decline rule and approve them, sizing the sample in advance from the expected fraud rate in that band so the cost of the experiment is known before it runs. The result is unbiased for that band only, and it is simultaneously the only source of fraud labels in the declined region.
Bound the extrapolation instead of hiding it. Run the holdout in two or three adjacent bands and report the spread. If the effect is flat across bands a constant is defensible; if it is steep, quote band-specific figures and decline to supply a single number.
Handle the window. Attrition after a decline can resolve over months, so a 90-day window truncates it and the randomised estimate is a lower bound on long-run damage at the same time as the observational version is an upper bound. Stating both directions is what makes the number safe to use.
Write the sentence that travels with the number: what it is (an estimate from a randomised holdout in one score band over a 90-day window), what it is not (a measurement anywhere else on the score range), and which way it is likely to be wrong.

Worked solution (40 min)

Define the decline cohort and the matched comparison cohort precisely, including matching variables and the pre-period window, and produce the 30-day and 90-day settled-volume difference.
Decompose the declined cohort into customers who never transact again and customers who transact less, since fraudsters concentrate in the first group and that split tells you how much of the estimate is contamination.
Write the holdout design: score band, sample share, expected fraud rate in band, expected cost of running it, and the run length needed to detect an effect large enough to change the cutoff.
Recompute p* = C_FP / (C_FP + C_FN) at the top and bottom of your estimated cost range and state whether the range changes the cutoff you would set.
Write the one-sentence caveat that will be quoted alongside the number in the pricing decision.

Expected result: Two numbers with opposite biases, an observational estimate labelled as an upper bound and a randomised in-band estimate labelled unbiased for that band only, plus a sensitivity table showing how p* moves across the range and an explicit statement of whether the residual uncertainty changes the decision.

Verify it yourself

If the cost range moves p* by less than the granularity at which the rule can actually be set, further measurement is not worth buying and that is the finding.
The observational estimate should exceed the randomised one in magnitude. If it does not, the matching is capturing something other than the effect of the decline and needs re-examining.
Removing customers with zero prior settled volume from both cohorts should shrink the observational estimate, since new-account fraud concentrates there.

Follow-up

Compliance and finance both object to deliberately approving transactions you believe are fraudulent. What is your answer, and how do you size the holdout?
Your interval spans the decision boundary. What do you recommend?
How would you detect that this number has gone stale?

Manual review rate doubles on the day two changes ship

Hard · definition change · model version · population stability

Manual review rate on fct_loan_application, defined as decided_by = 'manual' over decisions in ('approve','decline'), stepped from 9 to 18 percent on a single day. That day a new model_version went live, and the same release changed which referral paths are recorded as manual. Columns: submitted_at, channel, bureau_score, model_pd_12m, model_version, policy_rule_hits, decision, decided_by, decision_at. Attribute the step across definition, model and population, with a number against each.

Approach

Establish the shape before the cause. A definition or code change produces a step at the deploy timestamp; a population change produces a ramp. Plot at hourly granularity either side of the release and read which one this is.
Dual-run the definition. Recompute the metric under both the old and the new recording rules across several months of history. If the historical series under the new rule sits near 18 percent throughout, that part of the step is definitional and should be restated, not investigated.
Hold the model constant and decompose the residual by policy_rule_hits identifier. A referral rule keyed to a score threshold shows its own step, and naming the rule is what makes the finding actionable to the policy owner.
Test the score, not the applicants. Compare model_pd_12m distributions by model_version on the same applications where shadow scores exist; otherwise compare adjacent weeks with a population stability index, treating the customary 0.1 and 0.25 marks as heuristics rather than tests, and assess calibration separately from ranking.
Test the applicants, not the score. Compare channel mix, bureau_score distribution and null-bureau share across the boundary. If those are stable, population is not a driver and should be reported as approximately zero rather than left unquantified.
Deliver three components that sum to the observed step, each with the team that owns it.

Follow-up

If the new model is better calibrated, is a higher referral rate the correct outcome or a threshold nobody retuned?
How would you have caught the recording change before it reached the dashboard?
What is the swap set here, and which swap group would you inspect first?

Suggested practice checklist

For someone who has spent the last year in notebooks, dashboards or modelling work and has not written raw SQL under time pressure. The first four days rebuild query fluency against a fixture you control and can verify by hand; the last three attach that fluency to the rest of the loop.

Day 1 — Build a fixture you can check answers against

Create a local Postgres or SQLite database with four tables (users, sessions, events, orders) holding roughly 200 rows you generated yourself, so you know the contents well enough to predict every result.
Deliberately seed the cases that break queries: a user with no sessions, a session with no events, two orders sharing a timestamp, a NULL in one join key, and one duplicated user row.
Before writing any SQL, hand-compute five answers on paper (how many users placed at least one order, median orders per ordering user, and three others) and save them as the ground truth for the week.

Deliverable: A one-command seed script plus a text file of five hand-computed answers to grade every later query against.

Day 2 — Joins, filters and NULL semantics

Answer "which users have no orders" three ways (LEFT JOIN with IS NULL, NOT EXISTS, NOT IN) and confirm that the NOT IN version returns zero rows once the subquery contains a NULL, because the comparison is never TRUE.
Reproduce the LEFT JOIN that silently collapses to an inner join by putting a right-table predicate in WHERE, then fix it by moving the predicate into the ON clause, and record both row counts.
Create a fan-out bug on purpose by joining orders to order_items and summing the order total, then correct it with a pre-aggregated subquery and explain in one line which table changed the grain.

Deliverable: One annotated .sql file holding the three join traps, each with the wrong result and the corrected result side by side.

Day 3 — Window functions and frames

Write three window queries against the fixture: a running order total per user, the rank of each order within its user by value, and the day gap to that user's previous order, then check each against the day-one ground truth.
Run ROW_NUMBER, RANK and DENSE_RANK over a column containing ties, print all three side by side, and write one sentence on when each is the correct choice.
Switch one query from the default frame (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, which is what you get when ORDER BY is present and no frame is written) to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and explain why the output differs only when the ORDER BY column has duplicates.

Deliverable: Three verified window queries plus a short note explaining the RANGE versus ROWS difference in your own words.

Day 4 — The four analytical query patterns

Write a monthly retention grid: first order month per user, then months-since-first as the column, and verify that month zero equals the cohort size exactly.
Sessionize the events table under a 30-minute inactivity rule using LAG plus a cumulative sum over a new-session flag.
Build a four-step funnel that counts distinct users rather than events at each step, and state the rule you applied to a user who reaches step three without ever logging step two.

Deliverable: One file with the retention, sessionization and funnel patterns, each carrying a one-line note on the assumption it bakes in.

Day 5 — Write SQL the way you will have to write it live

Set a 12-minute timer and solve three medium prompts in a plain editor with no execution and no autocomplete, then run them and tally syntax errors separately from logic errors.
Narrate one solution aloud while writing it, stating the grain of each intermediate result (one row per user, one row per user-day) before you type its body.
Rewrite your slowest solution as a CTE chain where every CTE name states its grain, and time yourself re-solving it from blank.

Deliverable: A recording of one narrated solution plus an error tally that separates syntax from logic.

Day 6 — One day for everything that is not SQL

Write the preconditions of the two-sample t-test from memory, then check them: independent observations, and a difference in means whose sampling distribution is approximately normal, which at large sample sizes follows from the central limit theorem rather than from normality of the raw values.
Write the difference between an odds ratio from logistic regression and a relative risk, and state the condition under which the two are close (low outcome prevalence).
Prepare a 90-second answer to "how would you know this model is any good" that names the metric, the baseline you would beat, and the cost of the errors you care about.

Deliverable: One page of notes covering test preconditions, the odds-ratio caveat and the model-quality answer.

Day 7 — Full loop rehearsal

Run a 45-minute mock with someone willing to interrupt: 20 minutes of SQL, 15 minutes defining a metric, 10 minutes on a past project.
Re-solve from blank the two queries you were slowest on this week and compare the times against day five.
Write a five-line answer to "walk me through a project" that puts a number in the first sentence and names the decision the work changed.

Deliverable: Mock feedback notes plus a timed project narrative you can deliver without reading it.

Bring numbers, including the ones you are unsure of

An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.

Describe a situation where you had to influence a stakeholder to adopt a data-driven recommendation.

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What would you do differently if you ran that project again?
What did you decide not to do, and why?

Defend a vintage finding that contradicts the portfolio dashboard

Medium · vintage analysis · mix shift · stakeholder pushback

The lending dashboard shows blended 90-plus days-past-due falling for four consecutive quarters while originations grew 60 percent. Using fct_loan_performance_monthly, you build a vintage view keyed on origination_month by months_on_book and find the three most recent vintages are worse than their predecessors at the same age. The business lead presents that dashboard weekly and pushes back hard, suggesting you picked favourable cohorts. You get one meeting and the vintage table. Present the finding so it survives the cherry-picking objection and ends in a decision.

Approach

Reconcile before you contradict: show that aggregating your vintage table along the calendar diagonal reproduces the published blended series, so the disagreement is about age mix rather than about data quality.
Make the mechanism arithmetic rather than rhetorical: a loan cannot reach 90 days past due before it is 90 days old, so rapid origination growth shifts weight onto young months-on-book where the rate is structurally near zero.
Show every vintage rather than a selected pair, all indexed at months_on_book equal to 12, with cohort sizes printed beside each curve so nobody can claim the divergence rests on a thin cohort.
Handle restructuring explicitly, because restructured_flag resets days_past_due: count each loan on its worst pre-restructure state, or recent vintages will look better than they are.
Close on the decision rather than the chart: state what the divergence implies for the cutoff or the channel mix, and state in advance what evidence would make you withdraw the claim.

Follow-up

Two cohorts differ at month 12. How do you separate a seasoning effect from a genuine credit-quality effect?
Someone argues the recent vintages are simply a broker-channel mix shift. How do you test that, and what would confirm it?

Retract a published number after finding a currency bug

Medium · error disclosure · minor units · process repair

Two weeks ago you published an interchange and fraud analysis that summed amount_minor across fct_payment_authorization without converting currencies. Minor units are not two decimals everywhere: some currencies carry none and some carry three, so the sum has no interpretation. A pricing decision is already in flight on the back of it. You now have corrected figures. Produce the retraction: what you send, to whom, in what order, and what you change in the process so this class of error is caught next time rather than trusted next time.

Approach

Size the error before announcing it, because saying the number is wrong without a magnitude and a direction forces every reader to assume the worst case.
Check whether the conclusion actually flips: if the ranking that drove the pricing decision is unchanged, that belongs in the first sentence beside the correction rather than buried at the end.
Tell the person acting on it first and directly, then the wider distribution, using the same text, so nobody learns about it secondhand.
Write the correction as four parts: the old number, the cause in one clause, the effect on the pending decision, and the new number. Leave out self-flagellation, which makes the reader do emotional work instead of acting.
Fix the class rather than the instance: a rule that a sum over amount_minor either groups by transaction_currency or passes through both conversion steps, exponent scaling and then a dated rate into one named reporting currency, plus a standing reconciliation of the settled subset to the settlement ledger inside each settlement_currency.

Follow-up

The corrected figures do not change the decision. Do you still send the correction, and what does that choice signal?
What automated check would have caught this, where would it live, and what would it cost in false alarms?