cswork / libraryOpen practice workspace

Interview guides · Hirebeat

Hirebeat Data Scientist Interview Questions & Guide 2026

Hirebeat data scientist prep on data pitfalls: selection bias, target leakage, and checking an outlier's provenance before you drop it.

The 60-second brief

As a Data Scientist at Hirebeat, you will play a pivotal role in leveraging data to drive strategic decisions and enhance product offerings. This position is crucial for interpreting complex data sets, crafting predictive models, and generating insights that directly influence the company’s direction. Your work will not only impact internal processes but also enhance user experiences, making the role both challenging and rewarding.

Product-sense cases reward reasoning from a mechanism to a testable prediction. Reciting every metric you can name reads as pattern matching; naming the single quantity that would move if your explanation were true reads as thinking.

Focus areas

  • Read vintage curves, not blended portfolio averages
  • Reconcile amounts in minor units and currency
  • Separate authorization, settlement and dispute outcomes cleanly

Overview

In this capacity, you will collaborate closely with product managers, engineers, and other stakeholders to translate data findings into actionable strategies. You'll engage in projects that involve analyzing user behavior, optimizing algorithms, and identifying trends that can lead to innovative solutions. The environment at Hirebeat is dynamic and encourages creative problem-solving, making this an exciting opportunity for aspiring data professionals.

The Hirebeat interview process

Hirebeat candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Technical Interview

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

  • Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
  • Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
  • Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

  • Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
  • Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
  • If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Behavioral Interview

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

  • Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
  • The counterfactual: what would have been done without your analysis, and why that default was worse
  • How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

  • Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
  • Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
  • Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Reading the most recent months of fraud and dispute rates as final

Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.

Recalibrating an underwriting cutoff on approved and funded applicants only

Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.

Writing SQL without stating NULL and tie-breaking behaviour

Before calling a query finished, say what it does with NULLs, ties and empty groups. NOT IN against a subquery containing a single NULL returns no rows at all, and RANK, DENSE_RANK and ROW_NUMBER differ precisely on ties, so name which one the question requires.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Questions & practice

Design a data model for predicting customer churn.

Medium · machine learning and modelling

Approach

  1. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  2. Set a baseline first, so any model has something honest to beat.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • How would you choose the decision threshold, and who owns that choice?

Can you explain overfitting and how to prevent it?

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Check what information would not exist at prediction time, and exclude it.
  3. Say how the offline result would be validated online before it is trusted.

Follow-up

  • Where could label leakage enter this setup?
  • What would you monitor after launch to know the model is still valid?

Estimate a delinquency roll-rate matrix and project twelve months

Hard · roll rates · markov chain · survivorship

fct_loan_performance_monthly gives loan_id, as_of_month_end, months_on_book, delinquency_bucket, charge_off_flag, prepaid_in_full_flag and restructured_flag. Build a month-to-month transition matrix over the five delinquency buckets plus absorbing charged_off and prepaid states. Loans that stop appearing must be routed to an absorbing state rather than dropped. Project the current book forward 12 months by repeated matrix multiplication and report the projected share reaching charge-off. Handle restructured_flag explicitly, and name one place the Markov assumption fails on this data.

Approach

  1. Build consecutive month pairs per loan by shifting as_of_month_end within loan_id, then verify the shifted value is exactly one month later. A gap is not a transition, it is an exit you have not resolved yet.
  2. Resolve exits before counting anything. A loan whose last row carries charge_off_flag moves to charged_off, one carrying prepaid_in_full_flag moves to prepaid, and one that disappears with neither is a data question to raise rather than silently discard, because discarding it is survivorship that inflates every cure rate.
  3. Count pairs into a 7 by 7 matrix and row-normalise. Assert every row sums to one and the two absorbing rows are the identity; a row that does not sum to one means exits were dropped.
  4. Decide and state the restructure rule. Restructuring resets days_past_due, so a dpd_60_89 to current move on a restructured loan is not a cure. Either give restructured loans their own state or carry the pre-restructure bucket, but do not let that move land in the cure cell.
  5. Project by taking the current bucket distribution as a row vector and multiplying by the matrix twelve times. Report the charged_off entry, and report it again from an all-current starting vector so the reader can see how much of the projection comes from loans that are already delinquent today.
  6. State the homogeneity failure plainly: transition rates depend strongly on months_on_book, so one pooled matrix applied to a book with a young mix understates early-life delinquency. If the mix is moving, estimate separate matrices by seasoning band.

Worked solution (45 min)

  1. Sort by loan_id and as_of_month_end, shift to form (from_state, to_state) pairs, and flag pairs whose month gap is not exactly one.
  2. For each loan's final row, assign the absorbing destination from charge_off_flag or prepaid_in_full_flag, and list loans that vanish with neither as an exception count to report.
  3. Apply the restructure rule, then build the 7 by 7 count matrix with a cross-tabulation over ordered state categories and row-normalise it.
  4. Assert row sums equal one and absorbing rows are the identity, then take the current month's bucket distribution as a row vector.
  5. Multiply twelve times, report the charged_off component, and repeat from an all-current vector for comparison.

Expected result: A 7 by 7 row-stochastic matrix with identity rows for charged_off and prepaid, a current-to-current diagonal typically above 0.95, roll rates rising with bucket depth, and a 12-month projected charge-off share of a few percentage points from an all-current start and materially higher from the actual book.

Verify it yourself

  • Every row sums to one within floating-point tolerance, and both absorbing rows are exactly the identity.
  • No mass sits in cells that skip a bucket upward, such as dpd_1_29 to dpd_60_89 in one month; if there is, the pairs are not actually consecutive months.
  • The projection from an all-current start is strictly below the projection from the real book distribution.
  • Backtest by estimating on months up to t minus 12 and comparing the projected distribution with the realised one at t; a systematic optimistic gap points at dropped exits or at seasoning, not at noise.

Follow-up

  • How would you validate the projection against what actually happened, and over what window?
  • The cure rate out of dpd_30_59 rose five points last quarter. What are the candidate explanations and how would you separate them?
  • When would you prefer a vintage curve to a roll-rate projection, and why?

Count-weighted and dollar-weighted approval rates on one currency

Easy · aggregation · denominators · minor units

Using fct_payment_authorization, report the trailing 7-day authorization approval rate two ways for transaction_currency = 'EUR': count-weighted, and dollar-weighted on amount_minor. Exclude is_reversal = true, exclude incremental authorizations (parent_auth_id not null), and exclude zero-amount account verifications. auth_result = 'approved' is the numerator; the four declined_* values make up the rest of the denominator. Return channel, attempts, approved_attempts, approval_rate_count and approval_rate_value. State every exclusion and its reason before you write the SELECT.

Approach

  1. Say the denominator out loud first: attempts on a single transaction currency, excluding reversals, incremental authorizations and zero-amount verifications, because none of those is a purchase attempt a merchant is trying to get approved.
  2. Filter requested_at against a half-open interval (>= start AND < end) so the boundary day is neither dropped nor double counted.
  3. Compute both rates in one pass with FILTER clauses: COUNT() FILTER (WHERE auth_result = 'approved') over COUNT(), and SUM(amount_minor) FILTER (WHERE auth_result = 'approved') over SUM(amount_minor).
  4. Cast one side of each ratio to numeric before dividing, since amount_minor and the counts are integers and integer division silently truncates to zero.
  5. Group by channel and sort by the value-weighted rate, then read the gap between the two rates as a statement about where the declines sit rather than as noise.

Worked solution (20 min)

  1. Write the exclusion list as comments above the query: is_reversal = false, parent_auth_id is null, amount_minor > 0, transaction_currency = 'EUR'.
  2. Build a single aggregate query over fct_payment_authorization with a half-open requested_at predicate and those four filters.
  3. Emit attempts, approved_attempts, approval_rate_count and approval_rate_value with FILTER clauses and a numeric cast on the numerator.
  4. Group by channel, order by approval_rate_value ascending so the worst channel is on top.

Expected result: One row per channel present in the window, with attempts and approved_attempts as integers and both rates as numerics strictly between 0 and 1. approval_rate_value is typically lower than approval_rate_count on ecommerce and moto, because declines concentrate in higher-value card-not-present traffic.

Verify it yourself

  • SUM(attempts) across channels equals a separate COUNT(*) run with the same four filters and no GROUP BY.
  • Dropping the amount_minor > 0 filter raises attempts and moves approval_rate_count while leaving approval_rate_value unchanged, which confirms the zero-amount rows were only ever affecting the count-weighted figure.
  • Both rates are strictly less than 1 unless a channel had no declines at all in the window.

Follow-up

  • The two rates diverge by four points on the ecommerce channel but agree on card_present. What does that tell you, and what would you cut next?
  • How would you extend this to all currencies without summing amount_minor across them?
  • Which of the four decline reasons belong in the denominator of a rate you would put in front of a risk team, and which are really the network's problem?

Reconcile captured authorizations against the daily settlement total

Easy · reconciliation · currency · rounding

fct_payment_authorization holds captured_amount_minor in transaction_currency, and settlement_amount_minor in settlement_currency with settlement_fx_rate applied at settlement rather than at authorization. The rate is quoted in major units of settlement_currency per major unit of transaction_currency, and dim_currency.minor_unit_exponent carries the ISO 4217 exponent for each code (0, 2 or 3 depending on the currency). Produce a daily reconciliation: for each settled_at date and settlement_currency, return settled_count, total settlement_amount_minor, and the sum of captured_amount_minor converted into settlement minor units. Flag any date and currency pair whose two totals differ by more than one minor unit per settled authorization. Do not sum amounts across currencies anywhere in the output.

Approach

  1. Restrict to rows that actually settled: settled_at is not null and settlement_amount_minor is not null, which is a smaller population than captured rows because a capture can still be in flight.
  2. Truncate settled_at to a date with an explicit time zone so the cut matches the ledger's cut, since settled_at is timestamptz and date_trunc on timestamptz silently uses the session time zone.
  3. Join dim_currency twice, once on transaction_currency and once on settlement_currency, so both exponents are on the row. Minor units are not a common scale: a bare captured_amount_minor * settlement_fx_rate is correct only when the two exponents are equal, and a zero-decimal currency settling into a two-decimal one is wrong by a factor of 100.
  4. Convert per row as ROUND(captured_amount_minor::numeric / POWER(10::numeric, exp_txn) * settlement_fx_rate * POWER(10::numeric, exp_settle)) — minor units to major in the transaction currency, apply the major-per-major rate, then back to minor units in the settlement currency. The collapsed form ROUND(captured_amount_minor::numeric * settlement_fx_rate * POWER(10::numeric, exp_settle - exp_txn)) is the same expression. Round per row and then sum, not SUM(...) * an average rate, because the rate varies row by row and rounding per row is what the settlement file did.
  5. Group by the settlement date and settlement_currency together, never by date alone, and carry the currency into every output column name or row.
  6. Compare the two totals with a tolerance scaled by settled_count, since per-row rounding accumulates linearly in the number of rows rather than being a fixed constant.

Follow-up

  • A partial capture means captured_amount_minor is less than amount_minor. Where does that show up in this reconciliation, and where does it not?
  • On one currency pair the converted total is consistently about one hundredth of the settlement total, on every date, while the other pairs reconcile. Which two columns do you inspect first, and what single change fixes it?
  • The rate is documented as major-per-major. If a feed started publishing it minor-per-minor instead, which pairs would still reconcile and which would break?
  • How would you present a total across currencies to a finance partner who has asked for one number?

How would you evaluate the effectiveness of a new feature in an application?

Medium · product sense and metrics

Approach

  1. State what result would change your recommendation, so the answer is falsifiable.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • Which segment would you cut first, and what would that rule out?

Discuss a project where you had to analyze a large dataset. What was your approach?

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

How do you prioritize tasks when managing multiple projects?

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

How would you approach estimating the number of rides in New York City on a given day?

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • Which segment would you cut first, and what would that rule out?

How would you handle missing data in a dataset?

Medium

Approach

  1. Work from the decision backwards to the evidence you would need.
  2. State your assumptions explicitly before working the problem.
  3. Say what you would check first and why it is the highest-information step.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Measure an underwriting automation programme without hiding cost

Medium · metric gaming · swap set · operations metrics

An operations programme aims to cut manual underwriting. The proposed metric is manual review rate: decided_by = 'manual' over applications with decision in ('approve','decline') in fct_loan_application. The team controls the referral rules recorded in policy_rule_hits. Explain how they can halve that metric in one release while making the book worse, then design the readout that catches it: primary metric, the cuts it must always be reported in, and the guardrails. Say what evidence you would require before declaring a batch of referrals safe to auto-decide.

Approach

  1. Name the exploit precisely. Referrals can be converted to automatic declines by a rule change, which removes them from the numerator at no operational cost while they stay in the denominator, so the rate falls and applicants a human would have approved are rejected instead. Auto-approving the same population moves the metric identically with the mirror-image failure mode.
  2. Refuse the bare rate as a primary. Make it conditional on outcomes: manual review rate measured at a held-constant approval rate within each model_pd_12m band, so a reduction is creditable only when the decisions themselves did not move.
  3. Require fixed cuts on every report: by model_pd_12m band and by the rule identifier inside policy_rule_hits. A single blended operations number cannot be decomposed after the fact, and the decomposition is the entire diagnostic.
  4. Set guardrails on the things the exploit disturbs: approval rate within each score band, the override rate where manual decisions disagree with the automated recommendation, and the vintage 90+ rate at months_on_book = 12 for the newly auto-decided population against the referrals of the prior regime.
  5. Fix the evidence bar for retiring a referral rule: a shadow period where the rule fires and the case is still worked manually, recording what the automation would have decided, what the human decided, and, for the disagreement group alone, the realised performance. Agreement on the easy majority establishes nothing about the disputed minority.
  6. State the conflict honestly rather than designing around it. The referral population exists because the model is least certain there, so automating it trades loss for cost by construction, and the readout's job is to price that trade rather than to make it disappear.

Worked solution (30 min)

  1. Compute manual review rate for a recent month overall, then by model_pd_12m band, then by rule identifier extracted from policy_rule_hits.
  2. For each rule, compute the approve share and decline share of the applications it touches, so the rules whose removal would actually change decisions are identified before anyone proposes removing them.
  3. Simulate the exploit: reclassify one high-volume referral rule's applications as automatic declines, then recompute the blended rate and the band-level approval rates side by side.
  4. Write the readout spec: the primary with its conditioning, the two mandatory cuts, the three guardrails, and the shadow-period evidence bar.

Expected result: A table in which the simulated rule change lowers the blended manual review rate materially while approval rate in the affected model_pd_12m bands falls at the same time, together with a readout spec that makes it impossible to publish the first effect without the second.

Verify it yourself

  • The simulated change lowers the manual review rate and the band-level approval rate together. If band approval rate is unchanged, that rule was not binding and its removal is genuinely free.
  • Rule-level counts reconcile to the blended rate once applications hitting several rules are counted once. A jsonb array of rule identifiers is the usual place this double-counts.
  • The newly auto-decided population and the prior manual referrals are compared at equal months on book, not at equal calendar dates.

Follow-up

  • The override rate falls to near zero after the change. Is that good?
  • How long must the shadow period run, given the outcome matures at twelve months on book?
  • The rule fires on 200 applications a month. What can and cannot be established at that volume?

Manual review rate doubles on the day two changes ship

Hard · definition change · model version · population stability

Manual review rate on fct_loan_application, defined as decided_by = 'manual' over decisions in ('approve','decline'), stepped from 9 to 18 percent on a single day. That day a new model_version went live, and the same release changed which referral paths are recorded as manual. Columns: submitted_at, channel, bureau_score, model_pd_12m, model_version, policy_rule_hits, decision, decided_by, decision_at. Attribute the step across definition, model and population, with a number against each.

Approach

  1. Establish the shape before the cause. A definition or code change produces a step at the deploy timestamp; a population change produces a ramp. Plot at hourly granularity either side of the release and read which one this is.
  2. Dual-run the definition. Recompute the metric under both the old and the new recording rules across several months of history. If the historical series under the new rule sits near 18 percent throughout, that part of the step is definitional and should be restated, not investigated.
  3. Hold the model constant and decompose the residual by policy_rule_hits identifier. A referral rule keyed to a score threshold shows its own step, and naming the rule is what makes the finding actionable to the policy owner.
  4. Test the score, not the applicants. Compare model_pd_12m distributions by model_version on the same applications where shadow scores exist; otherwise compare adjacent weeks with a population stability index, treating the customary 0.1 and 0.25 marks as heuristics rather than tests, and assess calibration separately from ranking.
  5. Test the applicants, not the score. Compare channel mix, bureau_score distribution and null-bureau share across the boundary. If those are stable, population is not a driver and should be reported as approximately zero rather than left unquantified.
  6. Deliver three components that sum to the observed step, each with the team that owns it.

Follow-up

  • If the new model is better calibrated, is a higher referral rate the correct outcome or a threshold nobody retuned?
  • How would you have caught the recording change before it reached the dashboard?
  • What is the swap set here, and which swap group would you inspect first?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

  • For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
  • For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
  • Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

  • Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
  • List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
  • Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

  • Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
  • Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
  • Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

  • Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
  • Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
  • Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

  • Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
  • Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
  • Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

  • Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
  • For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
  • Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

  • Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
  • Listen back and mark every moment you proposed a solution before the success metric existed.
  • Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Tell me about a time you faced a significant challenge in a project. How did you handle it?

Medium · behavioural and stakeholder questions

Approach

  1. State the situation in two sentences and spend the rest on your reasoning.
  2. Close with what you would do differently, concretely.
  3. Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

  • How did you know the outcome was caused by your change?
  • What did you decide not to do, and why?

What is your experience with statistical analysis?

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Quantify the outcome, including what you would not claim credit for.

Follow-up

  • What would you do differently if you ran that project again?
  • How did you know the outcome was caused by your change?

State honestly what your cutoff change actually contributed

Hard · impact attribution · swap set · counterfactual

Six months ago your recommendation moved a credit cutoff, using fct_loan_application and fct_loan_performance_monthly. Since then approval rate rose four points and the 12-month vintage 90-plus rate on affected cohorts is flat. In the same window the bureau changed a score attribute, marketing shifted channel mix toward broker, and the internal funding rate moved. Your performance review asks for impact in currency terms. Give the number you would stand behind, the counterfactual it rests on, and the part of the observed movement you would not claim.

Approach

  1. Define the counterfactual before computing anything: the claim is not what happened after the change, it is what would have happened had the old cutoff scored the same applications, which means replaying the old threshold on the post-change population.
  2. Build the swap set: applications the new cutoff approves that the old one declined, applications the old one approved that the new one declines, and everyone else held out as unaffected. Only the swap groups carry your effect. Note that the swap-out group has no outcome under the new rule, because those loans were never funded, so its forgone margin must be estimated from matched pre-change approvals rather than observed.
  3. Price each swap group at months_on_book equal to 12 on the measure the cutoff was meant to move: interest and fees collected, minus net charge-offs, minus funding cost at the internal transfer rate.
  4. Strip the confounders explicitly. Cohorts affected by the bureau attribute change are either recomputed on the old attribute or excluded; channel mix is held fixed by reweighting to the pre-change mix; funding cost is charged at the rate in force each month rather than one blended average.
  5. State the residual you will not claim, with its size, and give a range rather than a point wherever cohorts have not yet reached 12 months on book.

Follow-up

  • The swap-in group is only 6 percent of applications. How does that change the way you present the number, and to whom?
  • What would you have needed to set up at launch to make this attribution clean, and why was a randomised band around the cutoff not used?