Interview guides · Berkshire Hathaway Homestate Companies
Berkshire Hathaway Homestate Companies Data Scientist Interview Questions & Guide 2026
Metric-diagnosis practice for Berkshire Hathaway Homestate Companies data scientist candidates: a number moves week over week and you work out why, splitting by segment, ruling out a logging or release change, and separating a mix shift from a real change in user behaviour before naming a cause.
The 60-second brief
The role of a Data Scientist at Berkshire Hathaway Homestate Companies is vital for driving data-driven decision-making across the organization. As a Data Scientist, you will analyze complex datasets, develop predictive models, and generate insights that influence strategic initiatives. This position is critical in shaping the company's approach to risk assessment, customer engagement, and operational efficiency, allowing you to make a tangible impact on products and services.
Most of the loop measures decision-making under uncertainty rather than recall. You are scored on whether you state your assumptions, commit to an estimate you can defend, and say explicitly what evidence would change it.
Focus areas
- Report only matured cohorts for loss metrics
- Reconcile amounts in minor units and currency
- Set fraud thresholds by expected cost
Overview
In this role, you will collaborate closely with cross-functional teams, including product development, marketing, and operations, to address real-world business challenges. You will work on projects like optimizing insurance claim processes, enhancing customer segmentation strategies, and improving underwriting models. The complexity and scale of data at Berkshire Hathaway Homestate Companies provide an exciting opportunity to apply advanced analytics and machine learning techniques, making this role not only challenging but also deeply rewarding.
The Berkshire Hathaway Homestate Companies interview process
Berkshire Hathaway Homestate Companies candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Initial Screening
Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.
What to demonstrate
- Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
- Whether you name what you have not done instead of stretching to cover every line of the posting
- Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage
How to prepare
- Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
- Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
- Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live
Technical Interviews
Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.
What to demonstrate
- Whether the query you write matches the plan you just described
- What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
- Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly
How to prepare
- Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
- Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
- Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock
Behavioral Assessments
This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.
What to demonstrate
- Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
- The counterfactual: what would have been done without your analysis, and why that default was worse
- How far your involvement ran past the handoff, and whether you checked that the change did what you predicted
How to prepare
- Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
- Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
- Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.
Final Assessments
A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.
What to demonstrate
- Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
- Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
- Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method
How to prepare
- Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
- Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
- Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
- Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times
Where candidates lose points
Counting authorizations instead of weighting them, and summing amounts across currencies
Declines skew toward high-value, cross-border and card-not-present transactions, so an unweighted approval rate can sit flat while approved value falls. Merchant retry logic also turns one declined purchase into several rows, inflating the denominator by an amount that varies by merchant and by decline reason. Amounts are held in the minor unit of the transaction currency and that unit is not always two decimals, since some currencies have none and some have three, so summing amount_minor across currencies produces a figure with no interpretation at all.
Averaging delinquency across a book that is growing
A loan three months old cannot be 90 days past due, so a portfolio with many recent originations reports a low blended 90+ rate purely from age mix. The blended rate falls fastest exactly when originations grow fastest, which is precisely when credit quality most needs watching, so the metric moves in the reassuring direction during the riskiest period. Only comparisons at equal months on book are valid, which is what a vintage or roll-rate view enforces.
Optimising accuracy on a heavily imbalanced target
State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.
Reading a dozen metrics with no multiplicity control
Nominate one primary metric before launch and treat the rest as guardrails or exploratory, with Bonferroni or Benjamini-Hochberg applied when you intend to make claims from them. Twenty independent tests at 0.05 under the null produce at least one false positive about 64 percent of the time.
Questions & practice
Write a Python function to calculate the mean and standard deviation of a list of numbers.
Medium · statistics and probability
Approach
- Sanity-check the answer against a simple bound or a simulated case.
- Quantify uncertainty explicitly rather than reporting a point estimate alone.
- Say what the estimate is of, and over what population it generalises.
Follow-up
- Which assumption here is most likely to be violated in practice?
- What sample size would you need to detect an effect half this size?
What metrics would you use to evaluate the performance of a classification model?
Medium · machine learning and modelling
Approach
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- Where could label leakage enter this setup?
- What would you monitor after launch to know the model is still valid?
Can you describe a machine learning project you have worked on from start to finish?
Medium · machine learning and modelling
Approach
- Check what information would not exist at prediction time, and exclude it.
- Say how the offline result would be validated online before it is trusted.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
Follow-up
- What would you monitor after launch to know the model is still valid?
- Where could label leakage enter this setup?
Collapse retry chains and compute a dollar-weighted approval rate
Medium · sessionisation · window functions · dollar-weighted rates
fct_payment_authorization gives auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, requested_at, auth_result, is_reversal, channel and issuer_country. Two reference frames give the minor-unit exponent per currency and a daily rate to one reporting currency. Collapse retry chains first: attempts sharing card_token_id, merchant_id and amount_minor whose consecutive gaps are under 15 minutes form a single attempt, whose outcome is its last row. Exclude reversals and zero-amount verifications. Return a 7-day rolling dollar-weighted approval rate by channel and issuer_country.
Approach
- Filter before grouping: drop is_reversal rows and zero-amount verifications, since neither is a purchase attempt and both would otherwise sit in the denominator.
- Sort by card_token_id, merchant_id, amount_minor and requested_at, take the gap to the previous row within that key, mark a chain start where the gap exceeds 15 minutes or the key changes, and label chains with a cumulative sum of that flag. This is a gap rule between consecutive attempts, not a fixed clock bucket, so a chain may span more than 15 minutes in total.
- Keep each chain's terminal row by requested_at. If a retry was approved, the purchase was approved; keeping the first row reports the decline that caused the retry as the outcome.
- Convert amounts exactly once: amount_minor divided by 10 to the power of the currency exponent, multiplied by the reference rate for the authorization date. Do not reach for settlement_fx_rate, which is null on precisely the declined rows the denominator needs.
- Build the rolling window as a ratio of two rolling sums, approved value over total value, per channel and issuer_country. A rolling mean of daily ratios weights a quiet Sunday the same as a busy Friday.
Worked solution (35 min)
- Filter out reversals and zero-amount rows, then sort by the chain key and requested_at.
- Compute the within-key time difference, derive the chain start flag and the chain id, and take the last row per chain with groupby(chain_id).tail(1) after sorting.
- Join the exponent and daily rate tables, compute value_reporting, and assert no nulls remain after the join.
- Aggregate approved value and total value to a daily grain by channel and issuer_country, reindex to a complete date range per group so missing days are zero rather than absent.
- Take 7-day rolling sums of both columns and divide, then confirm one hand-picked group-day against a direct filter.
Expected result: A DataFrame keyed by date, channel and issuer_country with approved_value, total_value and approval_rate. The collapsed attempt count is materially below the raw row count, with the gap concentrated in declined ecommerce rows, and ecommerce sits below card_present.
Verify it yourself
- Within every chain, all consecutive requested_at gaps are under 15 minutes; across chains with the same key, at least one gap is not.
- Chain count is strictly below raw attempt count, and the difference is concentrated in declined rows rather than spread evenly.
- Total converted value on a single-currency single-day subset equals the raw amount_minor sum divided by that currency's exponent power, times that day's rate.
- The rolling rate on one chosen day matches a direct filter-and-divide over the preceding seven days.
Follow-up
- The count-weighted rate is flat while the dollar-weighted rate falls 80 basis points. What do you look at first?
- How would you choose the 15-minute window rather than inheriting it?
- A merchant moves from two retries to five. Which of your two rates moves, and is that a real change in approval quality?
Given a dataset, how would you perform feature selection in Python?
Medium · sql and data manipulation
Approach
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- State the window function and its partition and ordering out loud before writing it.
- Say which table is the grain you start from, and join outward from it.
Follow-up
- How would you verify this result without re-running the same query?
- What breaks if events arrive late or out of order?
Attribute chargebacks to transaction month without fanning out
Medium · fan-out · cohort attribution · maturity
Report the matured first-chargeback rate. Numerator: fct_card_dispute rows whose dispute_stage is first_chargeback, representment, pre_arbitration or arbitration, attributed to the requested_at month of the linked fct_payment_authorization row rather than to opened_at. Denominator: settled authorizations in that same transaction month, meaning settled_at is not null and is_reversal is false. One authorization can carry more than one dispute case. Report only transaction months whose last day is at least 120 days old, and mark the remainder incomplete rather than showing them as low.
Approach
- Compute the denominator from fct_payment_authorization alone, grouped by the requested_at month, before any dispute table is mentioned, so the join cannot touch it.
- Compute the numerator as a separate aggregate: join disputes to their authorization only to recover requested_at, then count dispute rows per transaction month with the stage filter.
- Join the two aggregates on the month key, which keeps the grain at one row per month and makes double counting structurally impossible rather than something you have to remember.
- Apply the maturity gate on the month's last day plus 120 days against the current date, because a transaction on the final day of the month is the least mature one in the cohort.
- Emit an explicit status column of 'final' or 'incomplete' instead of filtering immature months away silently, so a reader cannot mistake absence for zero.
Worked solution (30 min)
- CTE denom: SELECT date_trunc('month', requested_at) AS txn_month, COUNT(*) AS settled_txns FROM fct_payment_authorization WHERE settled_at IS NOT NULL AND NOT is_reversal GROUP BY 1.
- CTE numer: join fct_card_dispute to fct_payment_authorization on auth_id, filter dispute_stage to the four stages at or beyond first_chargeback, group by the same txn_month expression, count dispute rows.
- LEFT JOIN numer onto denom on txn_month so months with zero disputes appear as zero rather than disappearing.
- Derive the rate with a numeric cast and a status column driven by (txn_month + interval '1 month - 1 day') + interval '120 days' <= current_date.
Expected result: One row per transaction month with settled_txns, chargeback_cases, a rate in basis points, and a status of 'final' or 'incomplete'. The final months form a roughly flat series; the incomplete months at the right edge are visibly and correctly labelled rather than sloping to zero.
Verify it yourself
- settled_txns for a given month is identical whether or not the dispute CTE is joined, which is the direct test that no fan-out occurred.
- COUNT(DISTINCT auth_id) in the numerator is at or below the dispute row count; if they are equal, no authorization in the data carries two cases and the fan-out risk is latent rather than absent.
- Every month flagged 'final' has its last day at least 120 days before the current date.
- Attributing by opened_at instead shifts the series later and flattens the right edge, which demonstrates why the attribution key matters.
Follow-up
- Should the numerator count dispute cases or disputed authorizations? Which one reconciles to the loss line, and which one to the operations queue?
- Some reason codes allow filing well beyond 120 days. How would you estimate the tail on a month you have decided to call final?
- How would you present the incomplete months so that a weekly reader does not read the right-hand slope as an improvement?
How do you prioritize competing tasks and projects in a fast-paced environment?
Medium · product sense and metrics
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Restate the decision this analysis has to support, and who acts on the answer.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How would you approach optimizing a pricing strategy for a new insurance product?
Medium · product sense and metrics
Approach
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
- Decompose the metric into the rates that drive it, and say which one you would check first.
Follow-up
- Which segment would you cut first, and what would that rule out?
- What would you do if the primary metric and the guardrail moved in opposite directions?
If tasked with reducing claim processing time, what data would you analyze to identify bottlenecks?
Medium · product sense and metrics
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Fix the population and the time window before naming any metric.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
Given a dataset with customer transactions, how would you identify patterns in purchasing behavior?
Medium · product sense and metrics
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- State what result would change your recommendation, so the answer is falsifiable.
- Restate the decision this analysis has to support, and who acts on the answer.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- Which segment would you cut first, and what would that rule out?
How would you handle missing data in a dataset?
Medium
Approach
- Say what you would check first and why it is the highest-information step.
- Clarify what is being asked and what a complete answer would contain.
- State your assumptions explicitly before working the problem.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Design an authorization approval rate a team cannot game
Easy · approval rate · metric gaming · mix shift
A payments team is given authorization approval rate as its quarterly target, defined as approved authorizations over all authorizations in fct_payment_authorization. Using auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, auth_result, is_reversal, parent_auth_id, channel, issuer_country and requested_at, list three ways the team can raise that number without a single additional customer purchase succeeding. Then write the definition you would put on the dashboard instead, with its exclusions, and name the one guardrail you would always read beside it.
Approach
- Enumerate the rows in the table that are not purchases: zero-amount account verifications, reversals (is_reversal = true), incremental authorizations (parent_auth_id not null) and merchant retries of a declined attempt. Each is a lever on the numerator or the denominator that no customer would ever notice.
- Name the mix levers. Recurring and card-present traffic approve at higher rates than ecommerce and cross-border, so routing changes or a marketing push into an easy channel move the blended rate with no change in decisioning quality at all.
- Work the retry arithmetic rather than asserting it. A declined purchase that is retried three times and succeeds on the fourth contributes one approval and four denominator rows, which is below a healthy blended rate, so suppressing retries raises the published number while some purchases that would have succeeded on a retry now simply fail.
- Replace the definition: dollar-weighted, retry chains collapsed to a single attempt per (card_token_id, merchant_id, amount_minor) within a 15-minute window, excluding reversals, zero-amount verifications and incremental authorizations, with all amounts converted into one reporting currency before summing because amount_minor is in the minor unit of transaction_currency and that unit is not always two decimals. Cut by channel and issuer_country so mix shift shows up as mix shift.
- Guardrail: net fraud loss in basis points of settled volume, read only on transaction months with at least 120 days of dispute maturity. The fastest way to approve more is to decline less risk, and that bill arrives two to four months later.
Worked solution (25 min)
- Compute the naive rate for one week: approved rows over all rows.
- Apply the exclusions one at a time (zero-amount, is_reversal, parent_auth_id not null), recording the rate after each, so you can say what each row type was worth to the headline number.
- Add a retry-collapsing CTE keyed on (card_token_id, merchant_id, amount_minor) over a 15-minute window, keeping the last attempt in each chain, and recompute.
- Recompute dollar-weighted in a single reporting currency, then split by channel and issuer_country.
- Write the final one-line definition with its exclusions, plus the guardrail sentence.
Expected result: Four rates for the same week (naive count, count after exclusions, count after retry collapsing, dollar-weighted after both), with the largest single gap attributed to a named row type, and a published definition that states every exclusion and the currency it reports in.
Verify it yourself
- Excluding zero-amount verifications should lower the rate, since those rows approve at a higher rate than purchases. If it raises the rate, confirm that the amount_minor = 0 filter is not also catching rows with a null amount.
- Collapsing retry chains raises the rate and shrinks the denominator. The numerator falls only by the number of chains that contained more than one approval.
- Declines skew toward higher-value, cross-border and card-not-present transactions, so expect the dollar-weighted rate to sit below the count-weighted one. If it sits above, check for amounts summed across currencies before believing it.
Follow-up
- The blended rate is flat but ecommerce cross-border fell 200 basis points. How do you present that?
- Why weight by amount rather than by count, and when does the weighted version mislead?
- Your 15-minute retry window merges two genuine purchases of the same amount at the same merchant. How would you bound that error?
Fraud losses appear to halve in recent transaction months
Medium · right censoring · development factors · maturity
A weekly chart attributes fct_card_dispute cases to the requested_at month of the linked fct_payment_authorization row. The two most recent months show the first-chargeback rate falling by half, and a risk rule shipped six weeks ago. Columns: dispute_id, auth_id, dispute_category, dispute_stage, opened_at, disputed_amount_minor, liability_shift_flag, outcome, net_loss_minor, resolved_at. Decide whether the rule worked, and produce the version of the chart you would sign your name to.
Approach
- Separate the two dates explicitly. opened_at is when a case was filed, requested_at is when the transaction happened. Attributing by transaction month is the right causal choice and is exactly what makes the newest months structurally incomplete.
- Measure the filing lag rather than assuming it: the distribution of opened_at minus requested_at over fully developed months, split by dispute_category, and the age at which around 95 percent of cases have arrived.
- Build a development triangle of transaction month by months of development on cumulative case counts, and estimate age-to-age factors from the columns that are complete.
- Develop the immature months with those factors and plot the result as an estimate with a visible band, kept visually distinct from the matured series rather than blended into it.
- State the assumption the method needs: a stable development pattern across cohorts. A change in filing behaviour, merchant mix or the dispute team's own backlog breaks it, so inspect factor stability down each column before relying on the estimate.
- Only then evaluate the rule, comparing pre-change and post-change cohorts at equal development age.
Follow-up
- What leading indicator would you accept while the cohort matures, and what is its known bias?
- How does liability_shift_flag change which disputes you should expect to see in the first place?
- If the rule also blocked good transactions, where does that cost appear, and is any of it in this chart?
Suggested practice checklist
For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.
Day 1 — Metric anatomy
- For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
- For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
- Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.
Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.
Day 2 — Diagnosing a drop without guessing
- Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
- List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
- Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.
Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.
Day 3 — Should we build it
- Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
- Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
- Write the counter-metric that would make you kill the feature even if it wins on the primary metric.
Deliverable: A one-page product memo ending in a decision rather than a list of considerations.
Day 4 — The places aggregate numbers lie
- Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
- Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
- Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.
Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.
Day 5 — Technical maintenance, aimed at metrics
- Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
- Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
- Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.
Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.
Day 6 — Turning engineering work into data science stories
- Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
- For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
- Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.
Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.
Day 7 — Mock case and gap list
- Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
- Listen back and mark every moment you proposed a solution before the success metric existed.
- Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.
Deliverable: A recorded case plus a rewritten opening 90 seconds.
Stories where someone disagreed with you
Have two ready. In one, the data was on your side and you had to move someone who outranked you. In the other, the pushback was correct and you changed position. The second is the harder story and it lands better, because it shows you separate being right from being attached to an answer. Name the person's actual objection.
Tell me about a time you faced a significant challenge in a project. How did you overcome it?
Medium · behavioural and stakeholder questions
Approach
- Name the disagreement or constraint, and how you resolved it with evidence.
- Close with what you would do differently, concretely.
- State the situation in two sentences and spend the rest on your reasoning.
Follow-up
- What would you do differently if you ran that project again?
- What did you decide not to do, and why?
Explain an incomplete dispute chart to a non-technical executive
Easy · dispute maturity · stakeholder communication · right-censoring
A finance lead is looking at first-chargeback rate by transaction month, built from fct_card_dispute joined to fct_payment_authorization on auth_id and attributed to requested_at. The last three months slope sharply down and the lead wants to announce a fraud improvement at tomorrow's review. Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, so those months are not complete. In five minutes, with no statistics vocabulary, explain why the decline is not yet evidence and say exactly what you would put on the slide instead.
Approach
- Lead with the mechanism in the listener's own terms, not with the statistical name for it: a dispute is attributed to the month the transaction happened, but it can be filed up to roughly 120 days later, so recent months contain only the disputes filed so far.
- Show completeness rather than arguing about the rate: for each transaction month, plot the share of its eventual disputes already filed, estimated from months that are fully matured. The last three months will sit visibly below 100 percent.
- Replace the chart with two artefacts: a matured series that stops 120 days back and is labelled final, and a development-factor estimate for the immature months drawn as a dashed range and labelled an estimate.
- Hand over one sentence the executive can repeat without you in the room: the recent months look better because the disputes have not arrived yet, not because fewer will arrive.
- Offer a weekly signal they can watch instead, such as the risk-score mix of approved volume or the decline-rule hit rate, and state up front what it does and does not predict.
Follow-up
- The deck ships tomorrow regardless. What exactly goes on the slide, and what wording do you insist on?
- How would you estimate the development factors, and how would you notice if they had shifted?
State honestly what your cutoff change actually contributed
Hard · impact attribution · swap set · counterfactual
Six months ago your recommendation moved a credit cutoff, using fct_loan_application and fct_loan_performance_monthly. Since then approval rate rose four points and the 12-month vintage 90-plus rate on affected cohorts is flat. In the same window the bureau changed a score attribute, marketing shifted channel mix toward broker, and the internal funding rate moved. Your performance review asks for impact in currency terms. Give the number you would stand behind, the counterfactual it rests on, and the part of the observed movement you would not claim.
Approach
- Define the counterfactual before computing anything: the claim is not what happened after the change, it is what would have happened had the old cutoff scored the same applications, which means replaying the old threshold on the post-change population.
- Build the swap set: applications the new cutoff approves that the old one declined, applications the old one approved that the new one declines, and everyone else held out as unaffected. Only the swap groups carry your effect. Note that the swap-out group has no outcome under the new rule, because those loans were never funded, so its forgone margin must be estimated from matched pre-change approvals rather than observed.
- Price each swap group at months_on_book equal to 12 on the measure the cutoff was meant to move: interest and fees collected, minus net charge-offs, minus funding cost at the internal transfer rate.
- Strip the confounders explicitly. Cohorts affected by the bureau attribute change are either recomputed on the old attribute or excluded; channel mix is held fixed by reweighting to the pre-change mix; funding cost is charged at the rate in force each month rather than one blended average.
- State the residual you will not claim, with its size, and give a range rather than a point wherever cohorts have not yet reached 12 months on book.
Follow-up
- The swap-in group is only 6 percent of applications. How does that change the way you present the number, and to whom?
- What would you have needed to set up at launch to make this attribution clean, and why was a randomised band around the cutoff not used?