Interview guides · First American Financial
First American Financial Data Scientist Interview Questions & Guide 2026
Data scientist prep for First American Financial candidates: drill A/B design, power and minimum detectable effect on original exercises.
The 60-second brief
As a Data Scientist at First American Financial, you operate at the intersection of complex financial data and industry-leading real estate solutions. Your work supports the organization’s ability to assess risk, streamline title and settlement services, and provide data-driven insights that help First American Financial's clients make informed decisions. You will be responsible for building, deploying, and maintaining predictive models that directly influence the company's core business operations.
Most of the loop measures decision-making under uncertainty rather than recall. You are scored on whether you state your assumptions, commit to an estimate you can defend, and say explicitly what evidence would change it.
Focus areas
- Report only matured cohorts for loss metrics
- Reconcile amounts in minor units and currency
- Set fraud thresholds by expected cost
Overview
This role is both challenging and high-impact. Because First American Financial deals with massive datasets related to property, insurance, and lending, you will often find yourself tackling problems involving high cardinality features and complex data structures. Whether you are automating manual processes or refining risk assessment algorithms, your contributions will be central to how the company maintains its competitive edge in a highly regulated and data-intensive sector.
The data science function at First American Financial is evolving; emphasize your ability to bring structure to ambiguous problems and communicate technical value to non-technical stakeholders.
The First American Financial interview process
First American Financial candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.
Initial Screening
Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.
What to demonstrate
- Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
- Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
- The substance of the questions you ask back, which an experienced screener reads as a level signal
How to prepare
- Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
- Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
- Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered
Virtual Interviews
An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.
What to demonstrate
- Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
- Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
- How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact
How to prepare
- Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
- For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
- Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.
Where candidates lose points
Counting authorizations instead of weighting them, and summing amounts across currencies
Declines skew toward high-value, cross-border and card-not-present transactions, so an unweighted approval rate can sit flat while approved value falls. Merchant retry logic also turns one declined purchase into several rows, inflating the denominator by an amount that varies by merchant and by decline reason. Amounts are held in the minor unit of the transaction currency and that unit is not always two decimals, since some currencies have none and some have three, so summing amount_minor across currencies produces a figure with no interpretation at all.
Averaging delinquency across a book that is growing
A loan three months old cannot be 90 days past due, so a portfolio with many recent originations reports a low blended 90+ rate purely from age mix. The blended rate falls fastest exactly when originations grow fastest, which is precisely when credit quality most needs watching, so the metric moves in the reassuring direction during the riskiest period. Only comparisons at equal months on book are valid, which is what a vintage or roll-rate view enforces.
Dropping rows with missing values without naming the mechanism
Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.
Defining the cohort on a post-treatment condition
Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.
Questions & practice
Describe the process of feature selection when you have hundreds of potential variables.
Medium · machine learning and modelling
Approach
- Say how the offline result would be validated online before it is trusted.
- Check what information would not exist at prediction time, and exclude it.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
Follow-up
- What would you monitor after launch to know the model is still valid?
- Where could label leakage enter this setup?
How would you handle high cardinality features in a predictive model?
Medium · machine learning and modelling
Approach
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Say how the offline result would be validated online before it is trusted.
- Pick an evaluation metric that matches the cost of each error type, not a default.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- Where could label leakage enter this setup?
How do you validate a model to ensure it generalizes well to new, unseen data?
Medium · machine learning and modelling
Approach
- Set a baseline first, so any model has something honest to beat.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Check what information would not exist at prediction time, and exclude it.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- What would you monitor after launch to know the model is still valid?
Bootstrap a fraud loss rate that clusters within merchant
Medium · bootstrap · clustered resampling · heavy tails
You have a per-transaction frame with auth_id, merchant_id, settled_amount_reporting and net_loss_reporting, both already in one reporting currency. Most rows carry zero loss, a few carry large ones, and losses cluster within merchant. Using only numpy's random generator and no resampling helper from any library, write a bootstrap that returns a 95 percent interval for net fraud loss in basis points of settled volume, resampling merchants with replacement and taking all rows belonging to each drawn merchant. Also produce the naive row-level interval and state which you would report.
Approach
- State the estimator before writing it: total net loss divided by total settled volume, times 10,000. It is a ratio of sums, so each replicate recomputes both sums. Averaging per-transaction loss rates instead would weight a five-unit transaction like a five-thousand-unit one.
- Pre-aggregate loss and volume to merchant level once. For a ratio of sums, drawing merchants and taking all their rows is arithmetically identical to drawing merchant-level (loss_sum, volume_sum) pairs, so a replicate becomes one integer draw plus two vectorised sums rather than a groupby inside the loop.
- Draw B replicates of M merchant indices with replacement, where M is the observed merchant count, compute the ratio per replicate, and take the 2.5th and 97.5th percentiles. Say explicitly that this is a percentile interval and that BCa would correct the skew-induced bias if the decision is close.
- Repeat with independent row draws for the naive interval and compare widths on the same replicate count.
- Report the clustered interval. Rows within a merchant share an acceptance profile, a category code and a fraud exposure, so they are not independent, and the row-level interval understates variance by roughly the design effect.
Worked solution (30 min)
- Compute the point estimate directly on the full data and keep it for comparison.
- Aggregate to merchant-level loss and volume arrays, record M, and set B to 2,000 with a seeded numpy Generator.
- In a vectorised loop, draw integer indices of shape (B, M), index both arrays, sum along axis 1, and take the ratio times 10,000.
- Repeat for the row-level version using the per-transaction arrays and N draws.
- Take the 2.5 and 97.5 percentiles of each replicate array and report both intervals alongside the point estimate.
Expected result: Two intervals in basis points sharing the same point estimate. The clustered interval is materially wider, commonly two to four times, and is right-skewed because a heavy merchant can be drawn more than once in a replicate.
Verify it yourself
- The point estimate equals the direct ratio on the full data and lies inside both intervals.
- A synthetic input where every merchant holds exactly one transaction makes the clustered and naive intervals coincide; this is the test that the resampling unit is wired correctly.
- Raising B from 2,000 to 10,000 shifts the endpoints by less than the reporting precision; if it does not, a handful of merchants is driving the tail and the estimate needs that caveat.
- Replicate ratios never fall outside the range implied by drawing only the lowest-loss or only the highest-loss merchants.
Follow-up
- Your clustered interval is three times wider. How do you explain that to someone who wanted a tighter number?
- One merchant accounts for 40 percent of losses. What does that do to the interval, and what would you do about it?
- How does this change if the question is whether two months differ rather than what this month's rate is?
Accident-quarter loss ratio on earned rather than written premium
Hard · loss ratio · earned vs written · development age
From fct_policy_period_monthly, compute the accident-quarter loss ratio by product_line: incurred losses, being paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, over earned_premium_minor for the same accident quarter. State explicitly whether loss_adjustment_expense_minor is included and apply that choice consistently. Also output the same ratio computed on written_premium_minor so the two can be compared. The table holds current values with no valuation-date snapshot. Say in one line which comparison this schema cannot support and what you would need to support it.
Approach
- Derive the accident quarter from as_of_month with date_trunc, and note that the table already attributes losses to the month of the loss event while earning premium pro rata into the same month, which is what makes the two sides comparable at all.
- Aggregate earned_premium_minor, written_premium_minor and the three loss components to product_line and accident quarter in one pass, keeping loss adjustment expense as its own column so the inclusion choice is a final-select decision rather than something buried in a CTE.
- Compute both ratios side by side and a third column for their difference, because the size and sign of that difference is a direct read on whether the book grew or shrank in the quarter.
- State the limitation plainly: every row carries today's reserve estimate, so each accident quarter is observed at a different development age and a cross-quarter comparison mixes development with underwriting. A fixed development age needs a valuation-date dimension, that is one row per accident period per valuation, which this table does not have.
- Guard against the mirror-image error on the numerator by confirming ibnr_reserve_minor is non-zero on recent quarters; if it is null or zero there, the recent periods are understated twice over and the series is not usable.
Follow-up
- Written premium exceeds earned premium by 18 percent this quarter and by 3 percent two years ago. What happened to the book, and what does it do to each ratio?
- How would you build a development triangle from a valuation-dated version of this table, and what would you use the chain-ladder factors for?
- Statutory presentation conventionally takes the expense ratio on written premium while the loss ratio uses earned. How do you avoid a combined ratio that quietly mixes the two bases?
Count-weighted and dollar-weighted approval rates on one currency
Easy · aggregation · denominators · minor units
Using fct_payment_authorization, report the trailing 7-day authorization approval rate two ways for transaction_currency = 'EUR': count-weighted, and dollar-weighted on amount_minor. Exclude is_reversal = true, exclude incremental authorizations (parent_auth_id not null), and exclude zero-amount account verifications. auth_result = 'approved' is the numerator; the four declined_* values make up the rest of the denominator. Return channel, attempts, approved_attempts, approval_rate_count and approval_rate_value. State every exclusion and its reason before you write the SELECT.
Approach
- Say the denominator out loud first: attempts on a single transaction currency, excluding reversals, incremental authorizations and zero-amount verifications, because none of those is a purchase attempt a merchant is trying to get approved.
- Filter requested_at against a half-open interval (>= start AND < end) so the boundary day is neither dropped nor double counted.
- Compute both rates in one pass with FILTER clauses: COUNT() FILTER (WHERE auth_result = 'approved') over COUNT(), and SUM(amount_minor) FILTER (WHERE auth_result = 'approved') over SUM(amount_minor).
- Cast one side of each ratio to numeric before dividing, since amount_minor and the counts are integers and integer division silently truncates to zero.
- Group by channel and sort by the value-weighted rate, then read the gap between the two rates as a statement about where the declines sit rather than as noise.
Worked solution (20 min)
- Write the exclusion list as comments above the query: is_reversal = false, parent_auth_id is null, amount_minor > 0, transaction_currency = 'EUR'.
- Build a single aggregate query over fct_payment_authorization with a half-open requested_at predicate and those four filters.
- Emit attempts, approved_attempts, approval_rate_count and approval_rate_value with FILTER clauses and a numeric cast on the numerator.
- Group by channel, order by approval_rate_value ascending so the worst channel is on top.
Expected result: One row per channel present in the window, with attempts and approved_attempts as integers and both rates as numerics strictly between 0 and 1. approval_rate_value is typically lower than approval_rate_count on ecommerce and moto, because declines concentrate in higher-value card-not-present traffic.
Verify it yourself
- SUM(attempts) across channels equals a separate COUNT(*) run with the same four filters and no GROUP BY.
- Dropping the amount_minor > 0 filter raises attempts and moves approval_rate_count while leaving approval_rate_value unchanged, which confirms the zero-amount rows were only ever affecting the count-weighted figure.
- Both rates are strictly less than 1 unless a channel had no declines at all in the window.
Follow-up
- The two rates diverge by four points on the ecommerce channel but agree on card_present. What does that tell you, and what would you cut next?
- How would you extend this to all currencies without summing amount_minor across them?
- Which of the four decline reasons belong in the denominator of a rate you would put in front of a risk team, and which are really the network's problem?
Walk me through how you would architect a solution for a real-time data streaming challenge.
Medium · product sense and metrics
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
If given a vague business objective, how do you scope the data science project?
Medium · product sense and metrics
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Restate the decision this analysis has to support, and who acts on the answer.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
How do you prioritize your work when you have multiple competing requests from different business units?
Medium · product sense and metrics
Approach
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
- Name one primary metric, then the guardrail that stops it being gamed.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Judge a rate increase without letting price flatter the ratio
Hard · mix shift · adverse selection · loss development
A rating change raises premium on one product_line's renewal book. Using fct_policy_period_monthly (earned_premium_minor, written_premium_minor, exposure_units, paid_loss_minor, case_reserve_minor, ibnr_reserve_minor, loss_adjustment_expense_minor, rating_tier, policy_status, renewal_flag, policy_term_end), the accident-period loss ratio improves two points over the following year. Explain why that number alone cannot tell you the change worked, decompose the movement into the effects you would separate, and specify the primary metric and guardrails you would commit to before the next rating change.
Approach
- Start with the arithmetic. A rate increase raises earned premium per exposure unit, so the loss ratio falls even if every policyholder behaves identically and every claim is unchanged. Part of the two points is mechanical and carries no information about risk at all.
- Switch the risk measure to one price cannot move: pure premium, being incurred losses (paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, with loss_adjustment_expense_minor included or excluded consistently and the choice stated) divided by exposure_units, evaluated by accident period at a fixed development age. Flat pure premium alongside an improved loss ratio means the improvement was entirely price.
- Decompose the loss ratio movement into three named components: the price effect at constant exposure and mix, the mix effect from which rating_tiers renewed and which lapsed, and the residual change in pure premium within tier. Only the third is evidence about risk selection, and it is usually the smallest.
- Name the adverse-selection risk directly. Price sensitivity and loss propensity are not independent, and the policyholders most able to leave after a rate rise are often the ones worth keeping. Retention by rating_tier is therefore a guardrail with teeth, and it has to be read at tier level because a flat blended retention hides tiers moving in opposite directions.
- Add prior-period reserve development as the second guardrail. The same two points are producible by setting case reserves or IBNR light, which surfaces only later as adverse development, so the reserve guardrail is what stops the primary metric being satisfiable by an accounting choice.
- Commit the primary before the next change: underwriting margin per exposure unit, being earned premium minus incurred losses minus loss adjustment expense minus allocated expense, over exposure_units, at a fixed development age, reported by accident period and by rating_tier with exposure volume printed beside it so that improving margin by shrinking the book is visible in the same table.
Worked solution (40 min)
- By accident quarter at a fixed twelve-month development age, compute both the loss ratio (incurred over earned_premium_minor) and the pure premium (incurred over exposure_units), before and after the change.
- Build the three-way decomposition: move price only at constant mix and exposure, then move the tier mix to the post-change distribution at constant price, then take the remainder as the within-tier pure premium change.
- Compute renewal retention by rating_tier from policies reaching policy_term_end, excluding cancelled_midterm and terms where no renewal offer was made, and cross each tier's retention with its prior pure premium.
- Pull prior-period reserve development for the periods used and state whether the improvement survives it.
Expected result: A decomposition splitting the two-point improvement into price, mix and within-tier pure premium components that reconcile to the total, alongside retention by rating_tier and an explicit statement of whether the within-tier component is distinguishable from zero.
Verify it yourself
- The three components reconcile to the observed change up to the interaction term. Decide in advance where the interaction is allocated and say so on the chart.
- Pure premium contains no price term by construction, so a step change in it exactly at the rate effective date points to an exposure or earning-pattern error rather than to a risk effect.
- Recent accident periods have no value at the fixed development age, which means unavailable rather than favourable, so any period younger than that age is excluded from the comparison.
- Retention measured at tier level reconciles to the blended figure once terms with no renewal offer are excluded from both.
Follow-up
- Retention is flat overall but fell nine points in the lowest-loss tier. What do you expect next year's pure premium to do?
- Why not use written premium as the denominator, and where would you still legitimately see it used?
- How would you separate a genuine underwriting improvement from a year of mild weather?
A single day of settled volume comes in thirty percent low
Medium · pipeline failure · null rates · data freshness
Yesterday's settled volume is 30 percent below the same weekday across the previous eight weeks, while authorization counts and approved counts for the day look normal. You have fct_payment_authorization with requested_at, captured_at, captured_amount_minor, settled_at, settlement_amount_minor, settlement_currency, created_at and updated_at, plus warehouse load metadata. Decide within the hour whether this is a real drop or a load problem, and say what you would post to the on-call channel in either case.
Approach
- Check the data before the business. Compare max(updated_at) and row counts per load partition for the affected day against the prior eight same weekdays; a truncated load usually shows as a count deficit confined to a specific hour range.
- Measure null rates per column by requested_at date. Authorization fields present while settled_at and settlement_amount_minor are null points at the settlement feed; a uniform deficit across all columns points at ingestion.
- Separate late arrival from loss. Settlement lands after authorization by design, so re-measure the same day 24 and 48 hours later. A deficit that closes on the next load is lag, not a drop.
- Reconcile against the independent source: tie the day's settled total to the settlement ledger or acquirer file. If the ledger has the money and the warehouse does not, the business number was never wrong.
- Only after those four steps consider a business explanation, and then only if a segment cut produces a coherent story rather than a flat haircut spread evenly across every merchant and channel.
- Close by splitting the alert: a freshness and completeness check on the settlement feed, separate from the volume alert, so the next occurrence is classified automatically.
Follow-up
- How would you make the daily volume alert immune to settlement lag without also hiding a genuine drop?
- What is the right way to restate a published number after a backfill lands?
- Which check catches a partial load that is uniform across the day rather than concentrated in one window?
Suggested practice checklist
Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.
Day 1 — Diagnostic, scored before you study anything
- Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
- Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
- Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.
Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.
Day 2 — Largest gap: find the boundary rather than the subject
- Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
- Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
- Re-solve one of them from memory four hours later, on paper, with nothing open.
Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.
Day 3 — Largest gap: drill the blocking sub-skill
- Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
- Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
- Have someone else read your one-sentence rule and find the precondition you left out.
Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.
Day 4 — Second gap, plus maintenance on your strongest area
- Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
- Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
- Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.
Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.
Day 5 — The gap that is not a skill
- Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
- Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
- Deliver one answer again with a hard 90-second limit to force structure before detail.
Deliverable: Two recordings with a counted improvement in time-to-first-question.
Day 6 — Retest under day-one conditions
- Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
- Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
- Write which single block you would still lose the offer on.
Deliverable: A second scored rubric placed next to the first, with one named remaining risk.
Day 7 — Full loop under interview conditions
- Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
- Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
- Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.
Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.
Bring numbers, including the ones you are unsure of
An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.
Tell me about a time you failed to meet a project goal. What did you learn?
Medium · behavioural and stakeholder questions
Approach
- Quantify the outcome, including what you would not claim credit for.
- Close with what you would do differently, concretely.
- Name the disagreement or constraint, and how you resolved it with evidence.
Follow-up
- How did you know the outcome was caused by your change?
- What would you do differently if you ran that project again?
Describe a project where you faced significant data quality issues. How did you resolve them?
Medium · behavioural and stakeholder questions
Approach
- Pick a story where you drove the decision, not one where you observed it.
- Quantify the outcome, including what you would not claim credit for.
- Close with what you would do differently, concretely.
Follow-up
- What did you decide not to do, and why?
- How did you know the outcome was caused by your change?
Explain an incomplete dispute chart to a non-technical executive
Easy · dispute maturity · stakeholder communication · right-censoring
A finance lead is looking at first-chargeback rate by transaction month, built from fct_card_dispute joined to fct_payment_authorization on auth_id and attributed to requested_at. The last three months slope sharply down and the lead wants to announce a fraud improvement at tomorrow's review. Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, so those months are not complete. In five minutes, with no statistics vocabulary, explain why the decline is not yet evidence and say exactly what you would put on the slide instead.
Approach
- Lead with the mechanism in the listener's own terms, not with the statistical name for it: a dispute is attributed to the month the transaction happened, but it can be filed up to roughly 120 days later, so recent months contain only the disputes filed so far.
- Show completeness rather than arguing about the rate: for each transaction month, plot the share of its eventual disputes already filed, estimated from months that are fully matured. The last three months will sit visibly below 100 percent.
- Replace the chart with two artefacts: a matured series that stops 120 days back and is labelled final, and a development-factor estimate for the immature months drawn as a dashed range and labelled an estimate.
- Hand over one sentence the executive can repeat without you in the room: the recent months look better because the disputes have not arrived yet, not because fewer will arrive.
- Offer a weekly signal they can watch instead, such as the risk-score mix of approved volume or the decline-rule hit rate, and state up front what it does and does not predict.
Follow-up
- The deck ships tomorrow regardless. What exactly goes on the slide, and what wording do you insist on?
- How would you estimate the development factors, and how would you notice if they had shifted?