Freddie Mac Data Scientist Interview Questions & Guide 2026

Interviewing at Freddie Mac

• Practice bank: 8+ questions
• Rounds: 2
• Typical prep: 1–2 weeks

Interview rounds

Round	Questions
Technical Screen	3
Onsite	5

How hard is the Freddie Mac Data Scientist interview?

Difficulty	Share	Questions
Easy	0%	0
Medium	50%	4
Hard	50%	4

What Freddie Mac actually tests for

Topic	Share	Questions
Data Manipulation (SQL/Python)	38%	3
Analytics & Experimentation	13%	1
Behavioral & Leadership	13%	1
Coding & Algorithms	13%	1
Machine Learning	13%	1
Statistics & Math	13%	1

The 60-second brief

As a Data Scientist at Freddie Mac, you sit at the vital intersection of financial services, risk management, and advanced technology. Freddie Mac plays a foundational role in the US housing market by keeping capital flowing to mortgage lenders. Within this ecosystem, the data science team is responsible for building the predictive models, risk-assessment frameworks, and automation pipelines that keep the secondary mortgage market stable and efficient.

Freddie Mac candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

What a Data Scientist does at Freddie Mac

Your work in this position directly influences multi-billion-dollar portfolios. Whether you are placed in the Multifamily division, Credit Risk Transfer, or Single-Family analytics, you will tackle highly complex datasets. You will transition traditional econometric models into scalable, modern machine learning systems, and increasingly leverage generative AI to automate the extraction of critical financial insights.

This role is highly visible and intellectually demanding. It requires a rare blend of deep statistical rigor, modern programming skills, and business acumen. If you enjoy solving high-stakes financial puzzles and deploying models that impact millions of homeowners, this position offers an unparalleled platform for professional growth.

The Freddie Mac interview process

Freddie Mac candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Phone Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What it evaluates

• Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
• Whether you name what you have not done instead of stretching to cover every line of the posting
• Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

• Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
• Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
• Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Comprehensive Interviews

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What it evaluates

• Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
• Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
• How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

• Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
• For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
• Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Project Presentation

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What it evaluates

• Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
• Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
• Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

• For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
• Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
• For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Technical Sessions

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What it evaluates

• Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
• Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
• Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

• Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
• Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
• If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Questions Freddie Mac candidates report

Product sense and metrics

Decomposition questions are arithmetic before they are judgement. A top-line number is a product or a sum of smaller terms, and a tree is only useful if those terms recombine into the thing at the top and do not overlap: users times sessions per user times actions per session, or this month's actives as last month's carried forward, plus new, plus resurrected, minus churned. Build the identity first and attach levers to the leaves second. Trees come apart when a branch is a metric someone likes rather than a term in the arithmetic, because one underlying change then shows up in two branches and neither explains it.

• What strategies do you use to mitigate hallucinations when using Large Language Models (LLMs) to extract financial metrics?
• How do you prioritize your work when multiple business lines are requesting analytical support with competing deadlines?
• Explain how you would optimize a slow-running Pandas data transformation pipeline.

SQL and data manipulation

Decide what one row of the output means before typing a single join. Nearly every question of this kind resolves into a spine at some grain, one row per user or per session or per user-day, with measures attached to it. Build the spine first, collapse every other table down to that grain, then attach. What most often separates a strong answer from a weak one is refusing to join one fact table straight onto another: pre-aggregate the second one, or test it with EXISTS. A duplicate row on the many side multiplies a measure you already computed, and nothing in the result looks broken.

• Write a Python function to clean and merge two highly unstructured datasets containing property lease information.
• How do you implement cross-validation when dealing with highly imbalanced credit default data?

Statistics and probability

Most of these questions end, explicitly or not, with a stakeholder who wants a decision and does not want a lecture, which makes precision of language the thing actually under test. A p-value is the probability of data at least this extreme if the null were true. It is not the probability that the null is true, and not the chance the result is a fluke. A 95% interval is a property of the procedure across repeated samples, not a 95% chance that this particular interval contains the parameter. Give the plain sentence that is still true, then the caveat that would change the decision.

• How do you detect and resolve multicollinearity in a high-dimensional dataset?
• Explain the mathematical difference between L1 (Lasso) and L2 (Ridge) regularization.

Machine learning and modelling

A model that clearly improves offline can move nothing online, and the useful question is why. The candidate set feeding it may already exclude the cases it would have caught. The score may enter a surface where a cap, a fallback or a business rule blunts it. Training labels may describe a population the live traffic no longer matches, because they were collected under an earlier policy. Treat the offline number as a filter rather than as proof, plan the launch as a measured rollout with a guardrail and a rollback, and say which of those gaps you would check first.

• Prove the Gauss-Markov theorem and explain its significance in ordinary least squares (OLS) regression.

Behavioural and stakeholder questions

Ownership questions usually arrive disguised as something else: a project that slipped, a pipeline that failed quietly, a number people had been reading wrong for months. What is being tested is what you did in the window before anyone told you it was yours. Be concrete about the first hour. Who you told, and what you checked before you knew how bad it was. The answer that falls short is the one that jumps from discovery straight to resolution and skips the part where you sized the damage, because sizing it is the decision everything else depended on.

• Describe a time when you had to explain a highly complex statistical model to a non-technical business stakeholder. How did you adapt your message?

Other topics

A few of these have no retrievable answer at all. They ask what you would prioritise with one analyst and two weeks, whether a model is good enough to ship, whether a data source is appropriate to use. Answer by naming the decision, then the cost of being wrong in each direction. A false positive that sends an email and a false positive that suspends an account do not deserve the same threshold, and any argument about the threshold only makes sense once those costs are on the table. Say what you would do, then say what evidence would change it.

• How do you design and structure AI prompts to convert unstructured lease agreements into highly structured JSON data?

Practice exercises with worked solutions

The prompts above tell you what gets asked.

Rebuild sessions from raw events with a thirty-minute gap

From fct_event (event_id, visitor_id, occurred_at_utc) alone, rebuild sessions: a new session begins when the gap from that visitor's previous event exceeds 30 minutes, and every session is force-closed at UTC midnight so none spans two calendar dates. Return one row per session with visitor_id, session_start, session_end, event_count and session_date. Do not read fct_session; the point is to reproduce it. Assume duplicate occurred_at_utc values exist for the same visitor.

How a strong candidate works it

• Get the previous timestamp per visitor with LAG(occurred_at_utc) OVER (PARTITION BY visitor_id ORDER BY occurred_at_utc, event_id). The event_id tiebreaker is required, not stylistic: with duplicate timestamps an unstable ordering makes the boundary flags non-deterministic between runs.
• Set a boundary flag when prev IS NULL, or occurred_at_utc - prev > interval '30 minutes', or occurred_at_utc::date <> prev::date. The third disjunct is the midnight rule, expressed as a date change rather than a clock comparison so it holds across any gap length.
• Number the islands with SUM(flag::int) OVER (PARTITION BY visitor_id ORDER BY occurred_at_utc, event_id ROWS UNBOUNDED PRECEDING). Because event_id is unique the ordering is total, so no two rows are peers and RANGE UNBOUNDED PRECEDING would compute exactly the same numbers here. Write ROWS anyway: it is the half of the guard that survives someone later simplifying the ORDER BY back to occurred_at_utc alone, at which point the default RANGE frame gives every row sharing a timestamp one shared running total.
• GROUP BY visitor_id and the island number, then MIN(occurred_at_utc) AS session_start, MAX(...) AS session_end, COUNT(*) AS event_count, session_start::date AS session_date.
• Reconcile against fct_session on one sample day. The counts should agree except for server-emitted events carrying no client session, so a systematic difference beyond those is a bug in the gap rule or in the ordering.

Expected result (30 min): One row per (visitor_id, island). SUM(event_count) equals the input row count exactly, with no events lost or duplicated. No session_end falls on a different date from its session_start.

Check yourself

• Total events in equals total events out.
• MAX(session_end::date - session_start::date) is 0 across all rows.
• For a visitor ordered 09:00, 09:10, 09:55, 23:58, 00:02, the result is four sessions, not three.
• No two sessions for the same visitor have overlapping [session_start, session_end] ranges.
• Swapping ROWS for RANGE while event_id stays in the ORDER BY changes nothing; dropping event_id from both ORDER BY clauses does change the boundaries for a visitor with duplicate timestamps, which locates the guard in the tiebreaker rather than in the frame keyword.

Follow-ups

• Why 30 minutes? What does a 5-minute rule do to sessions-per-visitor and to any per-session conversion rate?
• The same person uses phone then laptop. Two visitor_ids, two sessions. What breaks if you sessionise on user_id instead?
• The midnight rule splits an overnight session. Which metrics does that bias, and in which direction?

Rebuild per-visitor ordering without groupby convenience methods

You have a DataFrame of 2 million fct_event rows with visitor_id, occurred_at_utc and event_id, unsorted and containing duplicate timestamps within a visitor. Produce three new columns: event_rank, the 1-based position of the event within its visitor ordered by occurred_at_utc; seconds_since_prev, the gap to that visitor's previous event, NULL for the first; and is_first_for_visitor. You may use sort_values, shift, cumsum, numpy and boolean masking. You may not use groupby.transform, groupby.apply, groupby.cumcount, groupby.rank or merge_asof. Break timestamp ties on event_id.

How a strong candidate works it

• Sort once by ['visitor_id', 'occurred_at_utc', 'event_id'] and reset the index. The whole exercise reduces to row arithmetic on a sorted frame, and the tiebreak on event_id is what makes the result reproducible across runs.
• Mark visitor boundaries with is_first = df['visitor_id'].ne(df['visitor_id'].shift()). This is the single fact every other column derives from.
• Compute seconds_since_prev as the diff of the timestamp column, then overwrite it with NaT/NaN wherever is_first is True. The shift crosses the boundary between visitors and will otherwise hand the first row of each visitor the last event of the previous one.
• Build event_rank from a running counter that resets at boundaries: take a global cumulative position (np.arange(len(df))) and subtract, per row, the global position at which that visitor started. Get the start position by forward-filling the positions where is_first is True, which is a cumsum-free reset and is O(n).
• Verify against the forbidden method once, as a test rather than as the implementation, and confirm the two agree on every row.

Expected result (20 min): Three columns on the sorted frame: event_rank starting at 1 for every visitor and increasing by 1 with no gaps, seconds_since_prev null exactly where is_first_for_visitor is True, and is_first_for_visitor summing to df['visitor_id'].nunique().

Check yourself

• is_first_for_visitor.sum() equals the number of distinct visitor_id values.
• seconds_since_prev.isna().sum() equals that same number.
• event_rank.max() per visitor equals that visitor's row count, and event_rank is non-negative everywhere.
• A visitor with two events at the identical timestamp gets ranks 1 and 2 with a gap of 0.0, ordered by event_id.

Follow-ups

• The frame does not fit in memory. How does your approach change if you can only process one visitor-partitioned chunk at a time?
• occurred_at_utc is client-supplied and sometimes runs backwards within a visitor. Does your seconds_since_prev go negative, and should it?
• How would you extend this to reset the counter at every change of surface as well as visitor?

Define success for halving the free trial length

A proposal cuts the free trial from 30 days to 14. Trial-to-paid conversion is defined on fct_subscription_period: numerator, subscription_id whose first row with is_first_paid_period = TRUE has period_status IN ('active','past_due') and period_start_utc no later than 14 days after trial end; denominator, subscription_id whose first row has period_status = 'trialing' and period_start_utc in the cohort week. dim_account (account_id, account_created_at_utc, signup_surface) is also available and joins to fct_subscription_period on account_id. Explain why conversion rate alone cannot decide this, define the metric that can, and state the horizon and lag your readout needs. Deliverable: the decision metric and the readout schedule.

How a strong candidate works it

• Show the incomparability numerically before arguing about it: at a fixed calendar readout date, a larger share of the 14-day arm's cohorts have completed trial end plus 14 days plus settlement, so the short arm leads mechanically and the lead shrinks as both arms mature.
• Fix the censoring first, and do not mistake a rescaling for a fix. The given conversion rate already has trial starts as its denominator, so 'paid accounts per 1,000 trial starts' is that same ratio multiplied by 1,000 and answers nothing new. What the censoring needs is a fixed cohort age: read both arms at account_created_at_utc plus 60 days, which clears the long arm's full 30 + 14 + 3 = 47-day path to a settled first payment, so neither arm is censored at readout.
• Then move the denominator upstream, which is a separate fix and the only one that catches a higher share of a smaller population: paid accounts per 1,000 dim_account rows created in the cohort week. Precondition — this diverges from the conversion rate only when trial length is visible before the trial starts, on pricing pages, in ads or in the signup flow. If the arms are assigned at trial start and nothing upstream differs, trial starts per 1,000 new accounts is equal across arms by construction and the two metrics are the same comparison rescaled. Report that ratio per arm and say which regime you are in rather than assuming one.
• Add the leg neither denominator can see: a shorter trial converts users who had less time to reach activation, so read month-3 gross revenue churn and week-4 retention of the converted cohort, and read activation at trial day 7 to see whether the converted population changed composition.
• Audit 'past_due' before comparing, since the definition counts it as converted: report the rate with and without it per arm, because a gain that sits entirely inside past_due is a billing artefact rather than a conversion effect.
• Be explicit that the twelve-month consequence is not observable inside a planning cycle: pre-register that the decision is made on the 60-day metric, that the twelve-month cohort read will be published as a check, and what action follows if the two disagree.

Expected result (35 min): A decision metric stated as paid accounts per 1,000 new accounts at a fixed 60-day cohort age measured from account creation, the trial-start rate published per arm so it is explicit whether that denominator change carries any information here, a readout date computed from the enrolment window, and at least two quality metrics capable of overturning a conversion-rate win.

Check yourself

• Both arms must have identical elapsed time at readout: compute min and max of (readout_date minus account_created_at_utc) per arm and confirm the two ranges match.
• Trial starts per 1,000 new accounts must be reported per arm. If that ratio is equal across arms, the conversion rate and the per-new-account metric are the same comparison rescaled and cannot diverge — so never present them as two independent confirmations. Only a gap in that ratio gives the per-new-account metric information the conversion rate does not carry.
• The past_due share of the numerator should be similar across arms; a gap there is a payment-failure effect and does not belong in a conversion claim.

Follow-ups

• What happens to a user who would have converted on day 20, and how would you detect that population in the data?
• The short arm has lower day-7 activation but higher conversion. Reconcile those two facts into one story.
• What randomisation unit do you use here, and what goes wrong with the obvious alternatives?

Gross revenue churn doubled with no cancellations behind it

Gross monthly revenue churn computed from fct_subscription_period doubled from 1.8% to 3.6% in one month. The support queue shows no rise in cancellations and renewals look normal. You have fct_subscription_period with subscription_id, account_id, period_start_utc, period_end_utc, mrr_cents, mrr_cents_constant_fx, seats_billed, period_status, change_reason and canceled_at_utc, plus dim_account. Decompose the 1.8-point rise into named mechanisms, size each in points of the headline, and state the remainder you cannot explain.

How a strong candidate works it

• Reconstruct the numerator row by row and group it by change_reason before arguing about causes. A mid-period plan or seat change closes the current period row and opens a new one, so any implementation that treats a closed row as lost revenue books upgrades, downgrades and seat changes as churn; the change_reason breakdown of the numerator makes that visible in one query.
• Check the recognition timestamp. Churn belongs at period_end_utc, because revenue continues until the period ends, not at canceled_at_utc when the button was pressed. Then count period_end_utc rows per month across thirteen months: annual cohorts concentrate their period ends in the month twelve months after they were signed, so a spike that repeats in the same month last year is seasonality in the book, not an event this month.
• Recompute the whole numerator and denominator on mrr_cents_constant_fx. If the constant-currency figure is materially flatter, the move is an exchange-rate translation and belongs nowhere in a churn narrative.
• Check period_status handling. Rows with period_status = 'past_due' are dunning, not cancellation; a status-based rule counts them as loss while a period-end rule does not, and a dunning backlog can move the number by itself.
• Express every mechanism in points of the headline, sum them, and print the residual explicitly next to the month-to-month standard deviation of the prior twelve months. A decomposition without a stated remainder is a story, not an accounting.

Follow-ups

• Which of these mechanisms should be fixed in the metric definition and which should be reported as a genuine business fact?
• How would you present a month whose churn is dominated by an annual cohort anniversary without the audience concluding the business is deteriorating?
• What would you change so that an upgrade can never enter the churn numerator again, and how would you test that it worked?

Explain a wide interval to a non-technical executive

A pricing change is under consideration. Your best estimate of its effect on trial-to-paid conversion is a 1.8pp drop, with a 95% interval from a 4.6pp drop to a 1.0pp rise, read from a geo holdout rather than a randomised test. An executive preparing a board slide asks you for 'the number'. You have ninety seconds and one slide, and the words confidence interval, p-value and significance are not usable with this audience. Deliver the slide headline, the single supporting line, and what you say aloud.

How a strong candidate works it

• Recognise what is being probed: whether you can carry uncertainty into a decision instead of either hiding it or hiding behind it. The generic answer promises to explain the interval in plain English; the strong one replaces the question 'what is the number' with 'across this range, where does the decision change'.
• Find the threshold before you draft anything. Ask what the pricing case assumes, then compute the conversion drop at which the higher price stops adding revenue: price uplift on the conversions kept against the revenue lost from conversions forgone. That single figure is what makes the range legible.
• Restate the estimate and both bounds in the unit the audience already reasons in. Convert percentage points into monthly first-paid conversions at current trial volume, then into mrr_cents_constant_fx, so the slide reads as money per month rather than as statistics.
• Place the range against the break-even and say which part of it sits on each side. If most of the range clears the threshold, that is a recommendation to proceed with a monitoring plan; if the range straddles it, that is a recommendation to narrow the range first.
• Name what would narrow it and what that costs in weeks, then give one recommendation with an explicit condition for revisiting it. Uncertainty stated without a next step is read as indecision and the midpoint gets used anyway.

Follow-ups

• The executive says to give the midpoint and they will manage the risk. What do you do?
• How does the slide change if the interval were a 4.6pp to 0.2pp drop, with no positive outcomes in range?
• Why is a geo holdout the credible read here rather than the attributed channel numbers you already have?

Disagree with a product manager's roadmap claim using data

A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.

How a strong candidate works it

• Recognise what is being probed: whether you can separate a number being right from an inference being wrong, and do it without costing the PM face. The generic answer recites that correlation is not causation; the strong one names the specific confound and proposes the cheapest design that could distinguish the explanations.
• State the alternative concretely. Accounts that connect an integration in week one are accounts that already have a workflow and a technical owner, so week-one intent plausibly drives both the connection and week-four retention. The selection is on intent, which no amount of post-hoc adjustment observes.
• Order the discriminating analyses by cost. First, condition on pre-connection activity by comparing retention within strata of week-one core-action count, which removes the crude version of the confound but not unobserved intent. Second, look for variation in integration availability that was unrelated to intent, such as a staggered release or an outage window. Third, an encouragement design that randomises a prompt to connect and reads the intent-to-treat effect on week-four retention, which is the only version that identifies an effect.
• Run the timing check, because it is nearly free and it is the most persuasive single piece of evidence. If the retention advantage among connectors is already visible before any of them connected, the causal story is largely finished.
• Pre-commit to what would change your mind and say it before you show anything: if the gap survives stratification and the encouragement arm moves week-four retention at all, the feature has a case and you will say so.
• Open the 1:1 by agreeing with the true part, that the correlation is real and worth chasing, then ask what effect size the roadmap plan assumes. That makes the size of the claim the topic instead of its authorship.

Follow-ups

• The encouragement test needs six weeks and the roadmap locks in two. What do you recommend in the interim?
• Stratifying on week-one activity closes half the gap. What do you conclude, and what do you still not know?
• How would you word this in the roadmap document so the PM's original number is reframed rather than deleted?

Where candidates lose points

Watching an experiment daily and stopping when it crosses significance

A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.

Treating last-touch attribution as the causal value of a channel

The attribution label on dim_user is the output of a rule that assigns full credit to whichever touch happened to be recorded last inside a lookback window, and that rule systematically rewards channels that sit close to the conversion, especially branded search and retargeting, which largely intercept demand that already existed. Reallocating spend on those labels moves budget toward the channels that are best at being last, which is why attributed return on ad spend often improves while total signups do not. Nothing in the touchpoint data can settle this, because the counterfactual of not running the channel was never observed. The credible reads are a geo holdout or a scheduled pause, sized in advance on the total-signups metric rather than on the attributed one, and the honest framing in the meantime is that the label describes correlation with conversion and not incremental contribution.

Generalising beyond the population the sample actually supports

State the frame the sample was drawn from and where it diverges from the population you want to talk about: time window, platform, geography, opt-in. If a group is excluded from the frame, either weight to known margins or narrow the claim rather than quietly extending it.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Weakness triage, driven by a day-one diagnostic

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

• Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
• Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
• Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

• Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
• Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
• Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

• Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
• Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
• Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

• Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
• Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
• Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

• Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
• Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
• Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

• Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
• Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
• Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

• Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
• Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
• Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.