Information Technology Senior Management Forum Data Scientist Interview Questions & Guide 2026

The 60-second brief

A Data Scientist at Information Technology Senior Management Forum occupies a pivotal position at the intersection of advanced analytics, product strategy, and technical innovation. You are responsible for transforming complex datasets into actionable insights that drive high-stakes business decisions. Whether you are working on Card Intelligence, Travel Intelligence, or People Strategy & Analytics, your work directly influences the strategic direction of the organization.

The boundary with engineering varies enough to be worth asking about directly. Some seats end at the analysis and the writeup; others carry feature pipelines, scheduling and alerting, which decides whether production data plumbing belongs in your prep at all.

Focus areas

• Decompose expected loss into PD, LGD, EAD
• Set fraud thresholds by expected cost
• Separate authorization, settlement and dispute outcomes cleanly

Overview

This role requires a blend of rigorous statistical thinking and a pragmatic, product-oriented mindset. You will not just be building models; you will be identifying the right problems to solve, designing experiments to validate your hypotheses, and communicating complex findings to senior leadership. At Information Technology Senior Management Forum, the Data Scientist is viewed as a strategic partner, expected to champion data-driven culture and influence product roadmaps through evidence-based recommendations.

Where to focus your preparation

Treat the sections below as preparation areas and confirm the format with your recruiter.

Preparation focus

No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.

What to demonstrate

• Breadth across SQL, experimentation and product reasoning
• Ability to state assumptions before choosing a method

How to prepare

• Drill the practice exercises below and time yourself
• Prepare three quantified stories about decisions you drove

Where candidates lose points

Reading the most recent months of fraud and dispute rates as final

Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.

Using written premium as the denominator of a loss ratio

Premium is written at inception and earned pro rata across the exposure period, so in a growing book written premium runs ahead of earned premium and the loss ratio comes out too low, with the error reversing when the book shrinks. The numerator has the mirror-image problem if it omits incurred-but-not-reported reserves, since recent accident periods then look profitable twice over. Both sides must refer to the same exposure period, which is what an accident-period view at a fixed development age enforces.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Sizing estimates built on unnamed, unrevisable assumptions

Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.

Questions & practice

How would you handle missing or null data points when preparing a dataset for a predictive model?

Medium · machine learning and modelling

Approach

1. Frame the prediction: the label, the moment of prediction, and the action it triggers.
2. Set a baseline first, so any model has something honest to beat.
3. Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

• How would you choose the decision threshold, and who owns that choice?
• Where could label leakage enter this setup?

Estimate a delinquency roll-rate matrix and project twelve months

Hard · roll rates · markov chain · survivorship

fct_loan_performance_monthly gives loan_id, as_of_month_end, months_on_book, delinquency_bucket, charge_off_flag, prepaid_in_full_flag and restructured_flag. Build a month-to-month transition matrix over the five delinquency buckets plus absorbing charged_off and prepaid states. Loans that stop appearing must be routed to an absorbing state rather than dropped. Project the current book forward 12 months by repeated matrix multiplication and report the projected share reaching charge-off. Handle restructured_flag explicitly, and name one place the Markov assumption fails on this data.

Approach

1. Build consecutive month pairs per loan by shifting as_of_month_end within loan_id, then verify the shifted value is exactly one month later. A gap is not a transition, it is an exit you have not resolved yet.
2. Resolve exits before counting anything. A loan whose last row carries charge_off_flag moves to charged_off, one carrying prepaid_in_full_flag moves to prepaid, and one that disappears with neither is a data question to raise rather than silently discard, because discarding it is survivorship that inflates every cure rate.
3. Count pairs into a 7 by 7 matrix and row-normalise. Assert every row sums to one and the two absorbing rows are the identity; a row that does not sum to one means exits were dropped.
4. Decide and state the restructure rule. Restructuring resets days_past_due, so a dpd_60_89 to current move on a restructured loan is not a cure. Either give restructured loans their own state or carry the pre-restructure bucket, but do not let that move land in the cure cell.
5. Project by taking the current bucket distribution as a row vector and multiplying by the matrix twelve times. Report the charged_off entry, and report it again from an all-current starting vector so the reader can see how much of the projection comes from loans that are already delinquent today.
6. State the homogeneity failure plainly: transition rates depend strongly on months_on_book, so one pooled matrix applied to a book with a young mix understates early-life delinquency. If the mix is moving, estimate separate matrices by seasoning band.

Follow-up

• How would you validate the projection against what actually happened, and over what window?
• The cure rate out of dpd_30_59 rose five points last quarter. What are the candidate explanations and how would you separate them?
• When would you prefer a vintage curve to a roll-rate projection, and why?

Bootstrap a fraud loss rate that clusters within merchant

Medium · bootstrap · clustered resampling · heavy tails

You have a per-transaction frame with auth_id, merchant_id, settled_amount_reporting and net_loss_reporting, both already in one reporting currency. Most rows carry zero loss, a few carry large ones, and losses cluster within merchant. Using only numpy's random generator and no resampling helper from any library, write a bootstrap that returns a 95 percent interval for net fraud loss in basis points of settled volume, resampling merchants with replacement and taking all rows belonging to each drawn merchant. Also produce the naive row-level interval and state which you would report.

Approach

1. State the estimator before writing it: total net loss divided by total settled volume, times 10,000. It is a ratio of sums, so each replicate recomputes both sums. Averaging per-transaction loss rates instead would weight a five-unit transaction like a five-thousand-unit one.
2. Pre-aggregate loss and volume to merchant level once. For a ratio of sums, drawing merchants and taking all their rows is arithmetically identical to drawing merchant-level (loss_sum, volume_sum) pairs, so a replicate becomes one integer draw plus two vectorised sums rather than a groupby inside the loop.
3. Draw B replicates of M merchant indices with replacement, where M is the observed merchant count, compute the ratio per replicate, and take the 2.5th and 97.5th percentiles. Say explicitly that this is a percentile interval and that BCa would correct the skew-induced bias if the decision is close.
4. Repeat with independent row draws for the naive interval and compare widths on the same replicate count.
5. Report the clustered interval. Rows within a merchant share an acceptance profile, a category code and a fraud exposure, so they are not independent, and the row-level interval understates variance by roughly the design effect.

Worked solution (30 min)

1. Compute the point estimate directly on the full data and keep it for comparison.
2. Aggregate to merchant-level loss and volume arrays, record M, and set B to 2,000 with a seeded numpy Generator.
3. In a vectorised loop, draw integer indices of shape (B, M), index both arrays, sum along axis 1, and take the ratio times 10,000.
4. Repeat for the row-level version using the per-transaction arrays and N draws.
5. Take the 2.5 and 97.5 percentiles of each replicate array and report both intervals alongside the point estimate.

Expected result: Two intervals in basis points sharing the same point estimate. The clustered interval is materially wider, commonly two to four times, and is right-skewed because a heavy merchant can be drawn more than once in a replicate.

Verify it yourself

• The point estimate equals the direct ratio on the full data and lies inside both intervals.
• A synthetic input where every merchant holds exactly one transaction makes the clustered and naive intervals coincide; this is the test that the resampling unit is wired correctly.
• Raising B from 2,000 to 10,000 shifts the endpoints by less than the reporting precision; if it does not, a handful of merchants is driving the tail and the estimate needs that caveat.
• Replicate ratios never fall outside the range implied by drawing only the lowest-loss or only the highest-loss merchants.

Follow-up

• Your clustered interval is three times wider. How do you explain that to someone who wanted a tighter number?
• One merchant accounts for 40 percent of losses. What does that do to the interval, and what would you do about it?
• How does this change if the question is whether two months differ rather than what this month's rate is?

Describe how you would optimize a slow-running query that processes millions of rows.

Medium · sql and data manipulation

Approach

1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
2. State the window function and its partition and ordering out loud before writing it.
3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

• What breaks if events arrive late or out of order?
• How does the query change if the join becomes one-to-many?

Write a query using SQL window functions to calculate a rolling 7-day average of user transactions.

Medium · sql and data manipulation

Approach

1. Check whether any join is one-to-many before aggregating, or the sums inflate.
2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

• How does the query change if the join becomes one-to-many?
• How would you verify this result without re-running the same query?

Accident-quarter loss ratio on earned rather than written premium

Hard · loss ratio · earned vs written · development age

From fct_policy_period_monthly, compute the accident-quarter loss ratio by product_line: incurred losses, being paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, over earned_premium_minor for the same accident quarter. State explicitly whether loss_adjustment_expense_minor is included and apply that choice consistently. Also output the same ratio computed on written_premium_minor so the two can be compared. The table holds current values with no valuation-date snapshot. Say in one line which comparison this schema cannot support and what you would need to support it.

Approach

1. Derive the accident quarter from as_of_month with date_trunc, and note that the table already attributes losses to the month of the loss event while earning premium pro rata into the same month, which is what makes the two sides comparable at all.
2. Aggregate earned_premium_minor, written_premium_minor and the three loss components to product_line and accident quarter in one pass, keeping loss adjustment expense as its own column so the inclusion choice is a final-select decision rather than something buried in a CTE.
3. Compute both ratios side by side and a third column for their difference, because the size and sign of that difference is a direct read on whether the book grew or shrank in the quarter.
4. State the limitation plainly: every row carries today's reserve estimate, so each accident quarter is observed at a different development age and a cross-quarter comparison mixes development with underwriting. A fixed development age needs a valuation-date dimension, that is one row per accident period per valuation, which this table does not have.
5. Guard against the mirror-image error on the numerator by confirming ibnr_reserve_minor is non-zero on recent quarters; if it is null or zero there, the recent periods are understated twice over and the series is not usable.

Worked solution (40 min)

1. CTE quarterly: group fct_policy_period_monthly by product_line and date_trunc('quarter', as_of_month), summing earned_premium_minor, written_premium_minor, paid_loss_minor, case_reserve_minor, ibnr_reserve_minor and loss_adjustment_expense_minor.
2. Final SELECT: build incurred_minor as the three loss components plus the LAE column, with the LAE inclusion written as a named expression so the choice is visible on the page.
3. Emit loss_ratio_earned and loss_ratio_written, both cast to numeric, plus their difference and the written-to-earned premium ratio.
4. Order by product_line and accident quarter, and append the one-line note about the missing valuation dimension to the query as a comment.

Expected result: One row per product_line and accident quarter with earned and written premium, incurred losses, both ratios and their difference. In quarters where written premium exceeds earned, loss_ratio_written is strictly below loss_ratio_earned, and the gap widens with the growth rate.

Verify it yourself

• For a mature product line with flat volume, summed earned premium over a full policy term is close to summed written premium, so the two ratios converge there.
• The sign of loss_ratio_earned minus loss_ratio_written matches the sign of written premium minus earned premium in every quarter, which is an arithmetic identity and not a data finding.
• Recent accident quarters have non-zero ibnr_reserve_minor; a zero there means the numerator is missing its tail and the ratio is understated.
• Within a quarter, loss_ratio_earned divided by loss_ratio_written equals written_premium_minor divided by earned_premium_minor exactly, because both ratios share one numerator. Any deviation means the LAE term was built into one ratio and not the other, which is the real test that the inclusion choice was applied consistently.
• Recomputing with LAE excluded scales each quarter's ratio by incurred_minor / (incurred_minor + lae_minor). That factor is not constant across quarters unless LAE happens to be booked as a fixed fraction of incurred losses, so a uniform shift is evidence about how LAE is loaded, not evidence that the query is self-consistent — use the previous check for that.

Follow-up

• Written premium exceeds earned premium by 18 percent this quarter and by 3 percent two years ago. What happened to the book, and what does it do to each ratio?
• How would you build a development triangle from a valuation-dated version of this table, and what would you use the chain-ladder factors for?
• Statutory presentation conventionally takes the expense ratio on written premium while the loss ratio uses earned. How do you avoid a combined ratio that quietly mixes the two bases?

If a primary product metric drops suddenly, how would you go about diagnosing the root cause?

Medium · product sense and metrics

Approach

1. State what result would change your recommendation, so the answer is falsifiable.
2. Fix the population and the time window before naming any metric.
3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

• What would you do if the primary metric and the guardrail moved in opposite directions?
• How would you detect that the metric is being gamed rather than genuinely improving?

How would you design a metric to measure the success of a new feature in our mobile banking app?

Medium · product sense and metrics

Approach

1. Restate the decision this analysis has to support, and who acts on the answer.
2. State what result would change your recommendation, so the answer is falsifiable.
3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

• Which segment would you cut first, and what would that rule out?
• How would you detect that the metric is being gamed rather than genuinely improving?

How do you balance short-term conversion gains with long-term user retention when designing product experiments?

Medium · a/b testing and experimentation

Approach

1. Name the guardrails that would stop a launch even on a positive primary result.
2. State the primary metric and the minimum effect worth shipping, then size the test.
3. Say whether units interfere with each other, and switch design if they do.

Follow-up

• How would you handle interference between treated and control units?
• What would you do if you could not randomise at all?

Walk me through the steps you take to design a robust A/B test.

Medium · a/b testing and experimentation

Approach

1. State the primary metric and the minimum effect worth shipping, then size the test.
2. Decide the analysis before seeing data, including how long it runs and when you look.
3. Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

• What would you do if you could not randomise at all?
• What would you conclude if the result is positive but the test is underpowered?

Build the metric tree for a new instalment loan

Medium · metric tree · vintage analysis · leading indicators

A new instalment loan product launches next quarter. Build the metric tree from fct_loan_application and fct_loan_performance_monthly: one primary metric, the intermediate metrics that explain it, and the guardrails. The product owner wants a go or no-go call six weeks after the first disbursements, while the primary outcome is defined at twelve months on book. Name the leading indicator you would accept for the six-week call, state its bias in one sentence, and say what you would refuse to conclude from it.

Approach

1. Put the primary where the trade is real: risk-adjusted margin per 1,000 units of principal originated, evaluated at months_on_book = 12, charging funding cost at the internal transfer rate. Approval rate and origination volume can both be moved in a week by lowering the cutoff, and neither charges anything for the consequence.
2. Lay the tree out by layer so a movement can be localised: approval rate on a decision-reaching denominator (decision in ('approve','decline'), excluding withdrawn and expired) feeds offer-to-funded conversion within 30 days by offered_apr_bps band, which feeds the vintage 90+ rate at months_on_book = 12, which feeds risk-adjusted yield on average balances. Each layer's denominator is the numerator of the layer above.
3. Choose guardrails that catch the opposite failure to the primary's. A margin target is trivially met by shrinking the book to its safest slice, so report approval rate within each model_pd_12m band, where that collapse shows up first, alongside adverse-impact monitoring of the decision rule under compliance review.
4. For the six-week call, accept first-payment behaviour: the share of funded loans missing the first scheduled payment, or reaching days_past_due at or above 30 by months_on_book = 3. Estimate its relationship to the 12-month 90+ rate from completed vintages of the nearest comparable product, and report the dispersion of that ratio across those vintages as the uncertainty rather than quoting a point estimate.
5. State the bias in one sentence: early default is loaded toward application fraud and income misstatement and is close to blind to affordability deterioration that develops over months, which is the failure mode a new product with a new population is most likely to have, and the historical ratio breaks exactly when the population or the macro environment changes.
6. Say what six weeks cannot answer: nothing about the 12-month level in absolute terms, and nothing whatsoever about the applicants the policy declined.

Worked solution (35 min)

1. Write each metric in the tree as numerator, denominator, exclusions and window, four lines each, before writing any SQL.
2. Build a vintage table from fct_loan_performance_monthly with origination_month as rows and months_on_book as columns for an existing comparable product, and read the diagonal as calendar time.
3. For the last eight to twelve complete vintages, compute both the 30+ rate at months_on_book = 3 and the 90+ rate at months_on_book = 12, take the ratio, and record its minimum, median and maximum.
4. Write the six-week readout using that ratio range as an interval, with the one-sentence bias statement attached to the number itself rather than to a footnote.

Expected result: A four-layer tree whose primary is risk-adjusted margin at months_on_book = 12, plus a six-week readout that converts an observed three-month 30+ rate into a range for the twelve-month 90+ rate using the historical ratio's dispersion, labelled as an estimate with its bias stated inline.

Verify it yourself

• Every denominator in the tree is the numerator of the layer above it. If one is not, a movement cannot be attributed to a layer.
• The ratio between the three-month and twelve-month rates varies across vintages. If your data shows it constant, confirm you are not comparing cohorts at unequal maturity.
• Restructured loans enter the 90+ numerator on their pre-restructure worst state, because restructuring resets days_past_due and would otherwise erase the event.

Follow-up

• The first vintage looks excellent at three months on book. What would make you distrust it?
• How do restructured loans enter your twelve-month numerator?
• Which metric in this tree would you refuse to put on a weekly dashboard, and why?

Portfolio delinquency improving while the loan book doubles

Easy · vintage analysis · age mix · right censoring

The blended 90-plus days-past-due rate across fct_loan_performance_monthly fell from 3.1 to 2.2 percent over two quarters while monthly funded volume roughly doubled. Credit leadership wants to know whether underwriting improved. Columns: loan_id, as_of_month_end, origination_month, months_on_book, original_principal_minor, principal_balance_minor, days_past_due, delinquency_bucket, restructured_flag, charge_off_flag, charge_off_date. Produce the view that answers the question honestly, and state in one sentence what the blended rate can and cannot tell you.

Approach

1. Name the mechanical floor first. A first instalment falls due roughly a month after funding, so a loan cannot reach dpd_90_plus until around its fourth month on book. Every recent origination therefore enters the denominator with a numerator that is structurally zero.
2. Build a vintage table: rows origination_month, columns months_on_book, cell equal to the share of that cohort whose worst days_past_due reached 90 or more, or whose charge_off_flag became true, at or before that age.
3. Use each loan's worst state to date rather than its current bucket, and take the pre-restructure worst state, because restructuring resets days_past_due and would otherwise read as a cure.
4. Compare cohorts only at equal months_on_book, and render cells beyond a cohort's current maturity as absent rather than zero, so the table cannot be misread left to right.
5. Decompose the blended move into an age-mix component and a within-age component, so the write-up states how much of the 0.9 point improvement is arithmetic rather than asserting it.

Follow-up

• What does the diagonal of a vintage table represent, and when is reading it the right thing to do?
• How would a change in charge-off timing policy show up in this table, and how would you separate it from credit quality?
• Which single chart goes in front of the credit committee, and what do you say when someone asks for the blended series anyway?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

• Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
• Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
• Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

• Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
• On the third attempt, write the grain of every CTE as a comment before writing its body.
• Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

• Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
• Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
• Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

• Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
• Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
• Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

• Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
• Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
• Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

• Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
• Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
• Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

• Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
• Re-read only your own notes from this week, and open no new material.
• Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Stories where someone disagreed with you

Have two ready. In one, the data was on your side and you had to move someone who outranked you. In the other, the pushback was correct and you changed position. The second is the harder story and it lands better, because it shows you separate being right from being attached to an answer. Name the person's actual objection.

Recommend a decision whose true outcome matures a year later

Hard · decision under censoring · leading indicators · staged rollout

An underwriting rule change must be decided in six weeks. Its real outcome, the vintage 90-plus rate at months_on_book 12 in fct_loan_performance_monthly, matures in a year. The executive wants a yes or no, not a range. Randomising the credit decision across the whole population is not available. Name the leading indicator you would accept, state its bias and the direction of that bias, define the decision rule and stopping condition before any rollout starts, and say what reading would make you recommend reversing the change.

Approach

1. Fix the readout before the rollout, because a readout chosen after the data arrives is a story rather than a decision rule: indicator, window, threshold and reversal condition all go in writing first.
2. Choose the leading indicator on its measured relationship to the matured outcome in historical vintages rather than on availability. Early delinquency, typically the share reaching dpd_1_29 or missing a first scheduled payment by months_on_book 3, is the usual candidate, and you quantify how well it predicted the 12-month rate across past cohorts.
3. State the bias and its direction plainly: early delinquency under-represents default that emerges later and is contaminated by servicing and payment-date effects, so treat it as a floor on risk rather than an estimate of it.
4. Buy identification where full randomisation is unavailable: a narrow randomised approval band around the cutoff, or a staged rollout by channel or region read as a difference-in-differences, with the parallel-trends assumption stated and checked in the pre-period rather than assumed.
5. Give the executive the binary they asked for with the trigger attached in the same sentence: yes, conditional on the month-3 indicator staying inside a stated band, with an automatic hold if it breaches.

Follow-up

• How would you validate that the month-3 indicator predicts the 12-month outcome, and what evidence would invalidate it mid-rollout?
• Compliance refuses a randomised band. What is your next-best identification strategy, and what precision do you lose by taking it?

State honestly what your cutoff change actually contributed

Hard · impact attribution · swap set · counterfactual

Six months ago your recommendation moved a credit cutoff, using fct_loan_application and fct_loan_performance_monthly. Since then approval rate rose four points and the 12-month vintage 90-plus rate on affected cohorts is flat. In the same window the bureau changed a score attribute, marketing shifted channel mix toward broker, and the internal funding rate moved. Your performance review asks for impact in currency terms. Give the number you would stand behind, the counterfactual it rests on, and the part of the observed movement you would not claim.

Approach

1. Define the counterfactual before computing anything: the claim is not what happened after the change, it is what would have happened had the old cutoff scored the same applications, which means replaying the old threshold on the post-change population.
2. Build the swap set: applications the new cutoff approves that the old one declined, applications the old one approved that the new one declines, and everyone else held out as unaffected. Only the swap groups carry your effect. Note that the swap-out group has no outcome under the new rule, because those loans were never funded, so its forgone margin must be estimated from matched pre-change approvals rather than observed.
3. Price each swap group at months_on_book equal to 12 on the measure the cutoff was meant to move: interest and fees collected, minus net charge-offs, minus funding cost at the internal transfer rate.
4. Strip the confounders explicitly. Cohorts affected by the bureau attribute change are either recomputed on the old attribute or excluded; channel mix is held fixed by reweighting to the pre-change mix; funding cost is charged at the rate in force each month rather than one blended average.
5. State the residual you will not claim, with its size, and give a range rather than a point wherever cohorts have not yet reached 12 months on book.

Follow-up

• The swap-in group is only 6 percent of applications. How does that change the way you present the number, and to whom?
• What would you have needed to set up at launch to make this attribution clean, and why was a randomised band around the cutoff not used?

Turn a one-line fraud-number request into a scoped brief

Easy · scoping · metric definition · denominators

A stakeholder messages: what is our fraud rate, and is it going up? You have fct_payment_authorization, fct_card_dispute and dim_customer. At least four defensible answers exist: count-weighted or value-weighted, attributed to the transaction month or to the dispute filing month, and gross or net of recoveries and successful representments. You get one reply before someone else produces an uncaveated number. Write that reply: the clarifying questions you ask, the single default you will produce if nobody answers, and what the default excludes.

Approach

1. Establish the decision behind the question first, because a risk-rule change, a board number and a merchant contract negotiation need different denominators, and asking which one is not stalling.
2. Offer a short menu rather than an open question: a stakeholder can choose between two named options but cannot specify a denominator from scratch.
3. Commit to a default so the reply is useful even if nobody answers, for example net fraud loss in basis points of settled volume, attributed to the requested_at month, matured months only.
4. State the exclusions in the same breath as the default: non-fraud dispute categories, transaction months with less than 120 days of maturity, and first-party abuse that arrives coded as consumer_dispute.
5. Give a delivery time for the default and a longer one for the fuller cut, so the choice between them carries a visible cost.

Follow-up

• They come back wanting it by merchant for a contract negotiation. What changes in the definition and in the maturity rule?
• How would you separate first-party abuse from third-party fraud in this data, and what would you refuse to conclude from the split?