Interview guides · Kiavi

Kiavi Data Scientist Interview Questions & Guide 2026

Data scientist prep for Kiavi: practice defining a payments and lending metric, picking its guardrails and defending the tradeoff.

The 60-second brief

A Data Scientist at Kiavi is a highly strategic role positioned at the intersection of technology, real estate, and finance. Kiavi is one of the nation's largest technology-enabled lenders for real estate investors, and the data science team is the engine that drives its competitive advantage. In this role, you will build and deploy models that directly impact risk assessment, property valuation, capital allocation, and lead generation, shaping the core financial products that serve thousands of real estate developers.

The boundary with engineering varies enough to be worth asking about directly. Some seats end at the analysis and the writeup; others carry feature pipelines, scheduling and alerting, which decides whether production data plumbing belongs in your prep at all.

Focus areas

Reconcile amounts in minor units and currency
Set fraud thresholds by expected cost
Read vintage curves, not blended portfolio averages

Overview

The impact of this position is immediate and highly visible. Unlike traditional financial institutions where data science operates in a silo, Kiavi integrates data-driven decision-making into every phase of the lending lifecycle. You will work on complex, high-dimensional datasets that combine real estate market trends, property characteristics, macroeconomic indicators, and borrower behavior. Your models will not only predict default risks and property appreciation but also automate underwriting processes, making lending faster, safer, and more reliable.

To succeed as a Data Scientist at Kiavi, you must possess a rare blend of technical rigor and sharp business acumen. The business operates in a dynamic, capital-intensive market where understanding the "why" behind the data is just as important as the model's predictive accuracy. You will be expected to translate complex statistical outputs into actionable business strategies that help Kiavi scale its lending portfolio while maintaining strict risk controls.

The Kiavi interview process

Kiavi candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Phone Screen

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Take-Home Data Challenge

Your submission is read asynchronously by someone who cannot ask you a clarifying question, and who will usually skim it before reading it properly. That changes what good looks like. The conclusion belongs near the top with the supporting analysis beneath it, and the code should run start to finish on a clean machine without a manual step you forgot to document. A reviewer forced to reconstruct your reasoning from the order of notebook cells is already discounting the work. The submissions that land are the ones where a busy reader gets the answer immediately and can verify it if they want to.

What to demonstrate

Whether the answer arrives before the methodology, so a reader who stops after the first page still has the recommendation
Whether the code runs end to end from the submitted files, with dependencies and data paths declared rather than assumed
Whether each chart is legible on its own, carrying axis labels and units, and exists to support a claim made in the text

How to prepare

Restructure a past analysis so the opening paragraph holds the recommendation and the number behind it, then check that nothing later in the document quietly contradicts it
Copy your own submission into an empty directory, run it in a clean environment, and fix everything that breaks; hidden local state is caught here or by the reviewer
For every chart, write the one sentence it is meant to prove, and delete the chart if you cannot write that sentence

Feedback Call

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Onsite Interview Loop

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Recalibrating an underwriting cutoff on approved and funded applicants only

Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.

Reading the most recent months of fraud and dispute rates as final

Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.

Ending an analysis without a recommendation or next step

Close with what you would do and what would change your mind, stated as a condition you can check later. If the evidence is genuinely inconclusive, recommend the specific next measurement and say what it costs in time or exposure.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Questions & practice

What techniques do you use to handle highly imbalanced datasets when modeling default risk?

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Say what the estimate is of, and over what population it generalises.
Translate the result into the decision it informs, in one plain sentence.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

How would you use third-party data, such as public records or demographic APIs, to enrich a model predicting property appreciation?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

How would you build a model to forecast future lending opportunities in a specific geographical market?

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

How do you address the limitations of machine learning when trying to explain causal relationships rather than simple correlations?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Write integrity checks for the authorization and settlement lifecycle

Easy · data quality · minor units · fx reconciliation

You are given fct_payment_authorization as a pandas DataFrame with auth_id, requested_at, amount_minor, transaction_currency, auth_result, decline_reason_code, is_reversal, parent_auth_id, captured_at, captured_amount_minor, settled_at, settlement_amount_minor, settlement_currency and settlement_fx_rate. Write a function returning one row per integrity check with the check name, failing row count, failing share and up to five example auth_id values. Cover at least six checks, one of which reconciles captured_amount_minor against settlement_amount_minor through settlement_fx_rate. Partial capture, zero-amount verification and a decline with no capture are all legitimate and must not be flagged.

Approach

Separate contract violations from observations before writing any code: an approved row carrying a decline_reason_code is structurally impossible, while a capture two days after requested_at is merely slow and belongs in a different severity tier.
Express each check as a boolean mask over the whole frame and collect the masks in a dict, so the summary table is one comprehension over mask.sum() rather than a row loop.
For the reconciliation, leave minor units before comparing: expected = captured_amount_minor / 10exponent[transaction_currency] * settlement_fx_rate * 10exponent[settlement_currency]. Build the exponent table covering zero-decimal and three-decimal currencies instead of assuming two everywhere.
Guard the legitimate cases explicitly so each mask fires only on the genuine contradiction: captured_amount_minor below amount_minor is partial capture, amount_minor of zero on an approved row is account verification, a null captured_at on a declined row is correct.
Sort the output by failing share times a stated severity weight, because a check firing on 0.01 percent of rows can still be the one that breaks a ledger reconciliation.

Worked solution (25 min)

Assert auth_id is unique, then build a currency exponent lookup that includes the zero-decimal and three-decimal currencies present in the data.
Define masks for: approved with non-null decline_reason_code; declined with non-null captured_at; captured_amount_minor above amount_minor with parent_auth_id null; captured_at before requested_at; settled_at before captured_at; is_reversal true with parent_auth_id null; settlement_currency differing from transaction_currency while settlement_fx_rate is null.
Add the exponent-aware reconciliation mask with a tolerance of one minor unit plus a small relative term.
Assemble a frame of check_name, n_failing, pct_failing and up to five sample auth_id values, ordered by severity then share.
Read five flagged rows per check by hand and confirm each is genuinely contradictory before reporting any counts.

Expected result: A DataFrame of at least eight rows, one per check, each with n_failing, pct_failing and example auth_id values. The reconciliation check should fire on a whole currency at once rather than on scattered rows, because an exponent error is systematic while an FX error is not.

Verify it yourself

Every mask returns false on three hand-constructed legitimate rows: a partial capture, a zero-amount verification and a decline with no capture.
Each pct_failing uses the denominator that check actually applies to, with reversals and verifications excluded where they are out of scope.
The reconciliation residual is centred at zero with spread under one minor unit; residuals clustered at exactly 100x or 0.01x mean an exponent is wrong, not a rate.

Follow-up

Which of these would you run as a blocking pipeline assertion and which as a monitored metric, and why?
The FX check fails on 3 percent of rows, all in one settlement currency. How do you decide between a data bug and a rounding convention?
How would you detect that a currency's minor-unit exponent is wrong in your reference table, using only the transaction data?

Customers with no credit application, avoiding the NOT IN trap

Medium · null semantics · anti-join · scd type 2

Count current customers who have never submitted a credit application, broken out by segment. dim_customer is a slowly changing dimension type 2, so restrict to is_current = true, kyc_status = 'verified' and closed_at null. In fct_loan_application, customer_id is null for applicants who were not customers when they applied. Write the anti-join, return segment and customer_count, and state in one line what NOT IN (SELECT customer_id FROM fct_loan_application) returns against this table and why.

Approach

Pin the dimension to one row per customer first: is_current = true already guarantees that, but say so out loud, because forgetting it multiplies every count by the number of attribute versions a customer has accumulated.
Write the anti-join as NOT EXISTS with a correlated predicate on customer_id, which evaluates per row and is unaffected by nulls anywhere in the applications table.
Name the failure explicitly: NOT IN against a nullable column compares each candidate to a set containing NULL, the comparison yields UNKNOWN rather than TRUE, and the whole predicate is therefore never satisfied, so the query returns zero rows.
If NOT IN is required for some reason, add WHERE customer_id IS NOT NULL inside the subquery, which restores the intended semantics, and note that a LEFT JOIN with an IS NULL filter is equally safe.
Group by segment and sanity-check the total against the unfiltered current-customer count minus the count of distinct applying customers.

Worked solution (20 min)

SELECT segment, COUNT(*) FROM dim_customer c WHERE c.is_current AND c.kyc_status = 'verified' AND c.closed_at IS NULL.
Add AND NOT EXISTS (SELECT 1 FROM fct_loan_application a WHERE a.customer_id = c.customer_id).
Group by segment and order by the count descending.
Run the NOT IN variant alongside it and record that it returns zero rows, then run it again with IS NOT NULL added to the subquery and confirm the counts match the NOT EXISTS version.

Expected result: One row per segment present in the current dimension (retail, sme, corporate), each with a non-zero customer_count. The NOT EXISTS and the null-guarded NOT IN forms agree exactly; the unguarded NOT IN form returns no rows at all.

Verify it yourself

COUNT(DISTINCT customer_id) equals COUNT(*) on the filtered dimension, which confirms is_current really collapsed the type 2 versions.
The total across segments plus COUNT(DISTINCT customer_id) from fct_loan_application restricted to current verified open customers equals the full current verified open customer count.
Swapping NOT EXISTS for the unguarded NOT IN changes the result from a populated set to an empty one, which is the whole point of the exercise.

Follow-up

Rewrite it as a LEFT JOIN with IS NULL and say when you would prefer that form to NOT EXISTS.
How does the answer change if you want customers who never applied as of a historical date rather than today?
The applications table has 40,000 rows with a null customer_id. What are those rows, and are they a data quality problem or a product fact?

Attribute chargebacks to transaction month without fanning out

Medium · fan-out · cohort attribution · maturity

Report the matured first-chargeback rate. Numerator: fct_card_dispute rows whose dispute_stage is first_chargeback, representment, pre_arbitration or arbitration, attributed to the requested_at month of the linked fct_payment_authorization row rather than to opened_at. Denominator: settled authorizations in that same transaction month, meaning settled_at is not null and is_reversal is false. One authorization can carry more than one dispute case. Report only transaction months whose last day is at least 120 days old, and mark the remainder incomplete rather than showing them as low.

Approach

Compute the denominator from fct_payment_authorization alone, grouped by the requested_at month, before any dispute table is mentioned, so the join cannot touch it.
Compute the numerator as a separate aggregate: join disputes to their authorization only to recover requested_at, then count dispute rows per transaction month with the stage filter.
Join the two aggregates on the month key, which keeps the grain at one row per month and makes double counting structurally impossible rather than something you have to remember.
Apply the maturity gate on the month's last day plus 120 days against the current date, because a transaction on the final day of the month is the least mature one in the cohort.
Emit an explicit status column of 'final' or 'incomplete' instead of filtering immature months away silently, so a reader cannot mistake absence for zero.

Follow-up

Should the numerator count dispute cases or disputed authorizations? Which one reconciles to the loss line, and which one to the operations queue?
Some reason codes allow filing well beyond 120 days. How would you estimate the tail on a month you have decided to call final?
How would you present the incomplete months so that a weekly reader does not read the right-hand slope as an improvement?

How do you ensure data quality and consistency when merging internal transactional data with external third-party APIs?

Medium

Approach

Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

How do you define and measure risk when underwriting a loan for a fix-and-flip real estate project?

Medium

Approach

Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Design an authorization approval rate a team cannot game

Easy · approval rate · metric gaming · mix shift

A payments team is given authorization approval rate as its quarterly target, defined as approved authorizations over all authorizations in fct_payment_authorization. Using auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, auth_result, is_reversal, parent_auth_id, channel, issuer_country and requested_at, list three ways the team can raise that number without a single additional customer purchase succeeding. Then write the definition you would put on the dashboard instead, with its exclusions, and name the one guardrail you would always read beside it.

Approach

Enumerate the rows in the table that are not purchases: zero-amount account verifications, reversals (is_reversal = true), incremental authorizations (parent_auth_id not null) and merchant retries of a declined attempt. Each is a lever on the numerator or the denominator that no customer would ever notice.
Name the mix levers. Recurring and card-present traffic approve at higher rates than ecommerce and cross-border, so routing changes or a marketing push into an easy channel move the blended rate with no change in decisioning quality at all.
Work the retry arithmetic rather than asserting it. A declined purchase that is retried three times and succeeds on the fourth contributes one approval and four denominator rows, which is below a healthy blended rate, so suppressing retries raises the published number while some purchases that would have succeeded on a retry now simply fail.
Replace the definition: dollar-weighted, retry chains collapsed to a single attempt per (card_token_id, merchant_id, amount_minor) within a 15-minute window, excluding reversals, zero-amount verifications and incremental authorizations, with all amounts converted into one reporting currency before summing because amount_minor is in the minor unit of transaction_currency and that unit is not always two decimals. Cut by channel and issuer_country so mix shift shows up as mix shift.
Guardrail: net fraud loss in basis points of settled volume, read only on transaction months with at least 120 days of dispute maturity. The fastest way to approve more is to decline less risk, and that bill arrives two to four months later.

Worked solution (25 min)

Compute the naive rate for one week: approved rows over all rows.
Apply the exclusions one at a time (zero-amount, is_reversal, parent_auth_id not null), recording the rate after each, so you can say what each row type was worth to the headline number.
Add a retry-collapsing CTE keyed on (card_token_id, merchant_id, amount_minor) over a 15-minute window, keeping the last attempt in each chain, and recompute.
Recompute dollar-weighted in a single reporting currency, then split by channel and issuer_country.
Write the final one-line definition with its exclusions, plus the guardrail sentence.

Expected result: Four rates for the same week (naive count, count after exclusions, count after retry collapsing, dollar-weighted after both), with the largest single gap attributed to a named row type, and a published definition that states every exclusion and the currency it reports in.

Verify it yourself

Excluding zero-amount verifications should lower the rate, since those rows approve at a higher rate than purchases. If it raises the rate, confirm that the amount_minor = 0 filter is not also catching rows with a null amount.
Collapsing retry chains raises the rate and shrinks the denominator. The numerator falls only by the number of chains that contained more than one approval.
Declines skew toward higher-value, cross-border and card-not-present transactions, so expect the dollar-weighted rate to sit below the count-weighted one. If it sits above, check for amounts summed across currencies before believing it.

Follow-up

The blended rate is flat but ecommerce cross-border fell 200 basis points. How do you present that?
Why weight by amount rather than by count, and when does the weighted version mislead?
Your 15-minute retry window merges two genuine purchases of the same amount at the same merchant. How would you bound that error?

A single day of settled volume comes in thirty percent low

Medium · pipeline failure · null rates · data freshness

Yesterday's settled volume is 30 percent below the same weekday across the previous eight weeks, while authorization counts and approved counts for the day look normal. You have fct_payment_authorization with requested_at, captured_at, captured_amount_minor, settled_at, settlement_amount_minor, settlement_currency, created_at and updated_at, plus warehouse load metadata. Decide within the hour whether this is a real drop or a load problem, and say what you would post to the on-call channel in either case.

Approach

Check the data before the business. Compare max(updated_at) and row counts per load partition for the affected day against the prior eight same weekdays; a truncated load usually shows as a count deficit confined to a specific hour range.
Measure null rates per column by requested_at date. Authorization fields present while settled_at and settlement_amount_minor are null points at the settlement feed; a uniform deficit across all columns points at ingestion.
Separate late arrival from loss. Settlement lands after authorization by design, so re-measure the same day 24 and 48 hours later. A deficit that closes on the next load is lag, not a drop.
Reconcile against the independent source: tie the day's settled total to the settlement ledger or acquirer file. If the ledger has the money and the warehouse does not, the business number was never wrong.
Only after those four steps consider a business explanation, and then only if a segment cut produces a coherent story rather than a flat haircut spread evenly across every merchant and channel.
Close by splitting the alert: a freshness and completeness check on the settlement feed, separate from the volume alert, so the next occurrence is classified automatically.

Follow-up

How would you make the daily volume alert immune to settlement lag without also hiding a genuine drop?
What is the right way to restate a published number after a backfill lands?
Which check catches a partial load that is uniform across the day rather than concentrated in one window?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Bring numbers, including the ones you are unsure of

An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.

Describe a time when you received constructive feedback on a model or analysis. How did you iterate on your work to address those concerns?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
State the situation in two sentences and spend the rest on your reasoning.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

How would you explain what machine learning is to a business stakeholder who has no background in statistics or engineering?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Explain an incomplete dispute chart to a non-technical executive

Easy · dispute maturity · stakeholder communication · right-censoring

A finance lead is looking at first-chargeback rate by transaction month, built from fct_card_dispute joined to fct_payment_authorization on auth_id and attributed to requested_at. The last three months slope sharply down and the lead wants to announce a fraud improvement at tomorrow's review. Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, so those months are not complete. In five minutes, with no statistics vocabulary, explain why the decline is not yet evidence and say exactly what you would put on the slide instead.

Approach

Lead with the mechanism in the listener's own terms, not with the statistical name for it: a dispute is attributed to the month the transaction happened, but it can be filed up to roughly 120 days later, so recent months contain only the disputes filed so far.
Show completeness rather than arguing about the rate: for each transaction month, plot the share of its eventual disputes already filed, estimated from months that are fully matured. The last three months will sit visibly below 100 percent.
Replace the chart with two artefacts: a matured series that stops 120 days back and is labelled final, and a development-factor estimate for the immature months drawn as a dashed range and labelled an estimate.
Hand over one sentence the executive can repeat without you in the room: the recent months look better because the disputes have not arrived yet, not because fewer will arrive.
Offer a weekly signal they can watch instead, such as the risk-score mix of approved volume or the decline-rule hit rate, and state up front what it does and does not predict.

Follow-up

The deck ships tomorrow regardless. What exactly goes on the slide, and what wording do you insist on?
How would you estimate the development factors, and how would you notice if they had shifted?