Credit Risk Modeling Case Study Interview: Loan Vintages, Default Labels, and a Decision Memo

A credit-risk model can look excellent on a spreadsheet and still be unusable if recent loans have not had time to default, post-origination fields leaked into the features, or the recommendation cannot be translated into a governed lending decision. A strong credit risk modeling case study interview answer begins with the observation clock, not an algorithm leaderboard.

This exercise gives you six invented loans, a twelve-month default label, and a one-page decision memo. It is designed to test whether you can keep loan vintages, label maturity, out-of-time validation, calibration, and business constraints in one coherent explanation. Practice the related Explain PD model validation steps prompt after you have labeled the rows yourself.

Evidence boundary: The loans, dates, rates, and recommendation below are synthetic training material, not a lender's portfolio or an estimate of real default risk. “Default” is an exercise-specific event definition. Regulatory and policy obligations vary by jurisdiction and product; the cited official materials illustrate validation and explainability concerns, not legal advice or a universal approval rule.

State the target and as-of date before touching features

Assume the business asks for the probability that a newly originated loan experiences a defined default event within twelve months of origination. For the exercise, an event is the first recorded 90-plus-days-past-due status or charge-off, whichever occurs earlier. This is a modeling convention for the case, not an assertion that every lender or regulator uses the same threshold. The data extract is as of April 1, 2026. Use date arithmetic consistently and decide whether an event on the twelve-month anniversary is included; here it is.

The interviewer might ask, “Can we train on every loan in the file and call the others non-defaults?” No. A loan originated in October 2025 has only about six months of observation at the extract date. An absence of a default event so far is not proof of no default in twelve months. First ask about origination timestamp, event timestamp, data latency, observation end, early payoff, sale, and loss to follow-up. A paid-off loan can have a different risk trajectory and should not be silently treated as equivalent to an active loan without a defined policy.

Here is the complete toy extract. A blank event means none observed by the as-of date, not “never.”

Loan	Origination	First qualifying default event	Observed through	Twelve-month binary label as of April 1, 2026
L1	Jan 1, 2025	None	Apr 1, 2026	0: full window observed
L2	Jan 1, 2025	Sep 15, 2025	Apr 1, 2026	1: event within window
L3	Apr 1, 2025	None	Apr 1, 2026	0: full window observed
L4	Apr 1, 2025	Mar 1, 2026	Apr 1, 2026	1: event within window
L5	Oct 1, 2025	None	Apr 1, 2026	Unknown: window incomplete
L6	Oct 1, 2025	Feb 1, 2026	Apr 1, 2026	1 observed early; October vintage remains immature

L5 is not a zero. L6 already has a positive event within the target horizon, but including early positives from immature vintages while excluding their unknown negatives can bias a binary training sample. A simple full-window binary design uses mature cohorts for both classes. An event-time or survival formulation can make principled use of censored follow-up, but requires a different target, loss, and evaluation. Say which problem you are solving rather than combining them by convenience.

Read the vintage table without inventing a population estimate

For the toy data, the January 2025 vintage has one default among two fully observed loans; April 2025 also has one among two. Each is 50% in these two tiny rows, not a credible estimate of either month's true default rate. October 2025 has one observed early default and one unresolved loan. Reporting “October 50% twelve-month default” would pretend that L5's future is known. Reporting “October 100%” after dropping L5 would condition on event observability and be worse.

The useful vintage chart plots cumulative default incidence against months on book, with each cohort's denominator and exposure visible. Compare January at month six with October at month six if the same policy and observation rules hold; do not compare January at month twelve with October at month six as though the endpoints match. Stratify by product, underwriting policy, origination channel, risk band, and economic period where sample size permits. A vintage shift can reflect population mix or policy changes rather than model drift.

Inspect whether the event definition changed during the window. A new collections workflow, charge-off policy, or servicing vendor can move event timestamps without changing underlying borrower behavior. Also inspect whether the portfolio includes applications that were declined: observed defaults exist only for originated loans. A model trained on funded loans does not automatically estimate outcomes for applicants denied by the previous policy. That selection boundary belongs in the decision memo.

Design an out-of-time split that respects the decision moment

For a twelve-month binary target and April 1, 2026 extract, define a mature training window such as applications from January 2023 through September 2024, a validation window in October–December 2024, and a final out-of-time test in January–March 2025. The exact windows depend on volume, policy changes, and data latency; this is a defensible illustration, not a universal split. April 2025 applications sit at the maturity boundary and can be reserved until the team confirms complete event ingestion. A safety lag is often needed after the nominal anniversary.

Feature rows must represent information available when the lending decision was made. Candidate fields might include verified income, existing obligations, bureau attributes timestamped before the application, requested amount, and application channel. A current delinquency flag, collections note, later repayment count, or “ever charged off” feature is post-decision information and leaks the answer. Even a seemingly harmless aggregate can leak if it was recomputed using future data. Require a point-in-time feature join and inspect its lineage.

Keep borrower-linked records together where leakage is possible, and evaluate repeat applicants deliberately. If a borrower has a loan in train and another in test, decide whether the production scenario includes prior-loan history available at the second application. If yes, construct it point-in-time; if no, enforce an entity-disjoint split. Random row shuffling erases the temporal question. Scikit-learn's cross-validation guidance warns that ordinary schemes can be inappropriate for time-ordered data; a chronological holdout is the simplest final evaluation here.

The final test is not a sandbox for threshold tuning. Choose transformations, missing-value handling, hyperparameters, and candidate cutoffs on training and validation data. Freeze them, then report the out-of-time result once. If a policy change occurred between windows, stratify or run sensitivity analyses; do not smooth away a real regime change with a single overall AUC.

Compare models on ranking, probability, and decision cost

Start with an interpretable baseline such as regularized logistic regression. A more flexible candidate, perhaps gradient-boosted trees, earns its place only if it improves the relevant held-out outcomes without unacceptable instability, explanation burden, or operational cost. Ask what the model will do: rank cases for review, estimate twelve-month probability of default, or directly trigger a lending action. Those uses require different evidence.

Ranking metrics such as ROC AUC or precision-recall curves show discrimination, but a policy uses probabilities and costs. Check calibration by risk band and time period: among applications scored near a given PD, do observed mature outcomes align within uncertainty? The toy six-loan sample cannot support calibration; it merely illustrates label construction. On the full matured portfolio, also show score distribution shift, missingness, approval-rate impact, loss concentration, and uncertainty intervals. Evaluate material subgroups where lawful and appropriate, while avoiding unsupported claims about fairness from tiny cells.

A threshold calculation should use the business's loss and return assumptions. If expected loss is approximated as PD × loss given default × exposure at default, document how each component is estimated and when it is available. Add funding cost, servicing cost, revenue, and risk appetite for a real decision. Do not invent a single “safe PD cutoff” from this article. A cutoff that maximizes accuracy can still create poor outcomes when defaults are rare or errors have asymmetric costs.

Model validation also includes governance. The CFPB has stated in its US guidance that creditors using complex algorithms still need to provide specific, accurate adverse-action reasons when the rule applies. That does not mean “never use a complex model”; it means the explanation and decision process must be designed and tested, not improvised after deployment. Legal counsel and compliance owners must determine applicability for the actual product and jurisdiction.

Write the decision memo the interviewer can challenge

Use a memo with five short fields rather than a slide full of metrics:

Recommendation: Do not approve a production policy change from this toy extract. Build and freeze a point-in-time, mature-label baseline; compare the challenger on an untouched out-of-time period. Permit only a controlled shadow evaluation until calibration, stability, reason-code quality, and operational monitoring meet agreed gates.

Target and population: Twelve-month event from origination among funded loans under the exercise definition. Recent non-events are censored/unknown for the binary target. The model does not cover previously declined applicants without additional assumptions or data.

Evidence required: Label audit by vintage, event latency, feature availability at application, temporal split, discrimination and calibration with intervals, subgroup and policy-shift sensitivity, and decision-cost simulation under stated PD/LGD/EAD and revenue assumptions.

Deployment control: Version the score and policy, log decision inputs and reason codes, monitor live feature drift and mature-outcome performance, and define rollback or manual-review triggers. Validate any explanation process with compliance.

Open decision: What loss appetite, approval target, adverse-action obligations, and review capacity does the lender actually have? The model cannot choose these governance values.

This is intentionally conditional. It gives the team a safe next action and a clear gate. It does not imply that a model with a high AUC automatically deserves deployment or that an illustrative default rate should drive credit policy.

Common interviewer follow-ups

“Can we label L5 as zero for now and update it later?” Not for a static twelve-month binary training snapshot. That creates systematic false negatives in newer vintages. You can hold it out until maturity, or use an event-time method that encodes its follow-up and censoring explicitly. If you retrain later, version the as-of date so labels are reproducible.

“What if L6 defaulted after only four months?” Its twelve-month positive outcome is observed, but the October cohort's twelve-month rate remains unknown. For a mature-only binary analysis, keep a consistent eligibility rule for the cohort rather than including only outcomes that happened early. For a survival analysis, use time to event and censoring correctly.

“Why not random train/test?” The production decision is about future originations. A random split lets nearby policy regimes and duplicated entities appear on both sides and can hide temporal decay. A chronological test makes the intended generalization more visible, though it must still account for policy, product, and economic shifts.

“What if the new model improves AUC but worsens calibration?” Investigate the intended use. A ranking queue may tolerate recalibration; a PD-based expected-loss threshold cannot treat biased probabilities as harmless. Recalibration itself must be learned without peeking at the final test, then evaluated on a later period.

“Can this improve approvals?” Possibly, but the case lacks the counterfactual outcomes for declined applicants and the lender's risk constraints. Simulate on the observed population, state the selection limitation, and propose a governed pilot or exploration design rather than promising an approval-rate gain.

A concise answer to deliver aloud

“I would fix the twelve-month target and April 1 as-of date first. L1–L4 are mature; L5 has no observed event but is not a negative, and L6 has an early positive while its vintage remains immature. I would build a point-in-time feature set, train on older mature originations, tune on a later validation period, and keep a January–March 2025 out-of-time test untouched. I would compare an interpretable baseline with a challenger on both ranking and calibration, then simulate decisions using the lender's real loss, revenue, and policy constraints. I would not change approval policy from a six-row toy table; I would recommend a shadow evaluation and explicit governance gates.”

Then let the interviewer change the as-of date or event definition. Re-label the rows, explain what remains unknown, and update the memo. That is the central skill: the model decision should move only when the evidence and assumptions justify it.

Sources and Further Reading

• Scikit-learn User Guide: Cross-validation and time-series data
• CFPB Circular 2022-03: Adverse action notification and complex algorithms
• European Banking Authority: Guidelines on the definition of default

Research checked September 23, 2026. This is interview preparation, not lending, investment, or legal advice.