Interview guides · Kabbage

Kabbage Data Scientist Interview Questions & Guide 2026

Data scientist interview prep for Kabbage: original practice on experiment design, SQL and metric reasoning, with worked solutions.

The 60-second brief

As a Data Scientist at Kabbage, you play a pivotal role in leveraging data to drive business intelligence and enhance decision-making processes. This position is critical to the company, as it directly influences products and services by providing actionable insights derived from complex datasets. You will work on projects that involve predictive modeling, machine learning, and data analysis, all aimed at improving user experiences and optimizing operational efficiency.

Most of the loop measures decision-making under uncertainty rather than recall. You are scored on whether you state your assumptions, commit to an estimate you can defend, and say explicitly what evidence would change it.

Focus areas

Measure catalogue breadth beyond head consumption
Diagnose rebuffering by device, network and POP
Define a qualified stream and defend it

Overview

At Kabbage, you will engage with various teams, including engineering, product management, and customer service, to solve real-world problems using data. The work is both challenging and rewarding, as you will be tasked with developing algorithms that help small businesses access funding more efficiently. Your contribution will not only impact the company's bottom line but also empower entrepreneurs, making your role both strategic and impactful.

Expect to navigate a fast-paced environment where data-driven decisions are paramount. The complexity of the datasets and the scale at which Kabbage operates provide an exciting landscape for innovative thinking and problem-solving.

The Kabbage interview process

Kabbage candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Phone Screen

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Assessment

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Final Interview

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Testing hours, revenue or completion with a difference in means on a heavy-tailed distribution.

Listening and viewing hours per account are strongly right-skewed and content popularity is close to power-law, so the variance of a sample mean is dominated by a few accounts and the central limit approximation converges slowly at realistic sample sizes. A t-test on mean hours can flip sign when one heavy account's week changes, and an experiment can appear significant because a single title released into one arm's window. Capping at a pre-registered percentile, or decomposing into a rate (did they stream at all) and a conditional intensity, controls the variance, at the stated cost that capping biases toward zero exactly when the true effect lives in the tail.

Collapsing cancellation and payment failure into one churn number.

Involuntary churn from expired or declined payment instruments is a large and volatile share of gross churn, and it responds to retry schedules, card-updater coverage and billing provider, not to anything in the product. It also resolves late, so a period that looks involuntary today can be a successful retry next week, and reading the split before the dunning horizon closes overstates it. Compounding this, cancel-at-period-end means the cancellation request and the entitlement end are different timestamps on different rows, so a churn curve keyed on cancel_requested_ts and one keyed on churn_ts disagree by a full billing period.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Defining the cohort on a post-treatment condition

Ask how rows entered the table. Filtering on something that treatment itself influences, such as users who finished onboarding or accounts still active at ninety days, breaks comparability between arms; define the population at an entry point that precedes exposure and keep everyone in it.

Questions & practice

Can you explain the bias-variance tradeoff?

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Translate the result into the decision it informs, in one plain sentence.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

Explain how you would architect a machine learning model for scalability.

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Discuss the considerations for deploying a model into production.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.
Check what information would not exist at prediction time, and exclude it.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Permutation test for hours per account across two ranker arms

Medium · permutation-test · heavy-tails · experiment-analysis

arm_hours holds one row per account: account_id, arm in {control, treatment}, qualified_hours over a seven-day window. Roughly 40 thousand accounts per arm, about 38 percent of them at zero hours, and the non-zero tail is long. Without calling a library test function, write a permutation test on the difference in mean hours with 10,000 relabellings. Then run it as two parts: the difference in the share of accounts with any hours, and the difference in mean hours among accounts with hours. Report all three and say which belongs in the readout.

Approach

Shuffle the labels, not the data. Draw a permutation of the arm indicator over accounts, which is the unit that was randomised, and hold the hours vector fixed.
Make each replication O(n): precompute the grand sum and the arm sizes, so a shuffled difference is the treated subset sum over n_t minus (grand sum minus that subset sum) over n_c. Ten thousand replications then take seconds instead of a minute.
Use the two-sided p-value (1 + count of permuted absolute differences at or above the observed) divided by (B + 1). The plus one is not cosmetic: it makes the p-value valid rather than optimistic, and it means the smallest reportable value here is 1/10001, not zero.
For the two-part version, run the same machinery on the 0/1 indicator for the rate, then on the non-zero subset for the conditional mean, and say plainly that conditioning on a post-treatment outcome breaks the randomisation, so the conditional arm is descriptive rather than causal.
Report the rate test and the overall mean test as the result, with the conditional mean as colour, and give the effect size in hours beside each p-value, because at 80 thousand accounts almost anything is detectable.

Worked solution (30 min)

Pull hours into a float array and arm into a boolean array; record n_t, n_c, the grand sum and the observed difference of means.
Loop B = 10,000 times, taking np.random.permutation(n)[:n_t] as the treated index set and accumulating hours[idx].sum().
Convert each shuffled treated sum into a difference of means, compare its absolute value against the observed, count, and apply the plus-one form.
Repeat with hours replaced by (hours > 0).astype(float) for the rate test.
Subset to hours above zero, re-run for the conditional mean, and assemble a three-row result carrying effect, p-value and the unit each is defined on.

Expected result: Three rows: the mean-hours difference with a permutation p-value bounded below by 1/10001; the streaming-rate difference in percentage points with its own p-value; and the conditional mean difference among non-zero accounts, labelled descriptive because it conditions on a post-treatment outcome.

Verify it yourself

The permutation distribution of the mean difference is centred on zero within its own Monte Carlo error. A centre away from zero means the labels were not actually being reshuffled between replications.
The permutation p-value on the rate agrees to two decimal places with a two-proportion z-test, which is a fair cross-check because a 0/1 indicator is not heavy-tailed.
Doubling B leaves the p-value stable to within roughly 1 over sqrt(B). A p-value that moves a lot between runs is not yet resolved.
Multiplying every hours value in both arms by 60 leaves all three p-values identical under the same seed and scales only the effect sizes, since a permutation test is invariant to a common positive rescaling.

Follow-up

The permutation p-value on the mean is 0.03 and the rate test is flat. What is the most likely explanation, and does it change the decision?
How would CUPED on pre-period hours change your power here, and what would disqualify a covariate?
Accounts are households. Does that affect the validity of this test, or only its interpretation?

Solve a coding challenge related to data manipulation and analysis.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Sessionise playback into listening sessions with an idle gap

Medium · window functions · gaps and islands · sessionisation

fct_stream has profile_id, stream_id, content_version_id, started_at, ended_at (nullable when the client stopped reporting) and played_seconds. Collapse consecutive streams by the same profile into listening sessions: a new session begins when a stream starts more than 30 minutes after the latest end reached by any of that profile's earlier streams. One profile can play on two devices at once, so streams overlap and 'the previous stream' is not well defined — the idle gap is measured against the running maximum end, not against the row immediately before. Treat a NULL ended_at as started_at plus played_seconds. Return one row per session with profile_id, session_start, session_end, stream_count, distinct content_version_id count and total played_seconds. Use window functions; a self-join on fct_stream is not an acceptable answer.

Approach

In a first CTE compute effective_end = COALESCE(ended_at, started_at + played_seconds * INTERVAL '1 second'). This both fills the abandoned-client case and bounds it, since played_seconds is advancing playback only and cannot run past the true end.
Take prev_max_end = MAX(effective_end) OVER (PARTITION BY profile_id ORDER BY started_at, stream_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING), and flag is_new_session = (prev_max_end IS NULL OR started_at > prev_max_end + INTERVAL '30 minutes'). LAG(effective_end) is the reflex here and it is wrong under concurrent devices: with A 10:00-12:00, B 10:05-10:10 and C starting 10:45, LAG hands C the end of B at 10:10, so C opens a new session 35 minutes into a session still running until 12:00. The NULL branch is not defensive noise either: the first row of every profile has no preceding frame and would otherwise fall outside any session.
Number the sessions with SUM(is_new_session::int) OVER (PARTITION BY profile_id ORDER BY started_at, stream_id ROWS UNBOUNDED PRECEDING). A running count of boundaries is the gaps-and-islands device. Specify ROWS rather than letting the default RANGE frame apply, because RANGE lumps peer rows that share a started_at into one frame position — the same reason the ORDER BY carries stream_id as a tie-break.
Group by (profile_id, session_number) and aggregate: MIN(started_at), MAX(effective_end), COUNT(*), COUNT(DISTINCT content_version_id), SUM(played_seconds).
The running maximum makes the sessions disjoint in wall-clock time, but it does not make SUM(played_seconds) an elapsed-time figure: two devices playing at once contribute overlapping seconds, so the total can exceed session_end minus session_start. Report it as consumption and say which of the two the requester asked for.

Worked solution (30 min)

Materialise the effective_end CTE and check for rows where effective_end < started_at, which means negative played_seconds or a clock-skewed ended_at and must be quarantined before anything else.
Add the running MAX and the boundary flag; verify that exactly one row per profile has prev_max_end IS NULL and that it is flagged as a session start.
Add the running SUM to number sessions; confirm the numbers are dense and begin at 1 within each profile.
Aggregate and reconcile: SUM(stream_count) across all sessions must equal the input row count.
Run the overlap fixture — one profile with A 10:00-12:00, B 10:05-10:10 and C at 10:45 — and confirm it collapses to one session. With LAG(effective_end) in place of the running MAX it splits into two, and the second session's start, 10:45, sits inside the first session's span.

Expected result: One row per (profile_id, session_number). Session numbers are dense from 1 within each profile, SUM(stream_count) equals the input row count exactly, and a profile's sessions are disjoint in wall-clock time: each session_start exceeds the previous session's session_end by more than 30 minutes.

Verify it yourself

Input row count equals SUM(stream_count); a shortfall means rows fell out of the window partition or the NULL branch of the boundary flag.
For any profile, each session's session_start exceeds the previous session's session_end by more than 30 minutes. This holds only because the boundary is measured against the running maximum end — under LAG(effective_end) the overlap fixture above produces a session whose session_start, 10:45, precedes the prior session_end of 12:00.
Changing the gap from 30 to 60 minutes reduces the session count or leaves it unchanged, and never raises it.

Follow-up

Two devices on one profile play simultaneously. Is that one session or two, and what changes in the query for each answer?
Re-run at a 10-minute and a 60-minute idle gap. How far does session count move, and which number do you publish and defend?
Can SUM(played_seconds) for a session ever exceed session_end minus session_start, and what does it mean when it does?

How do you prioritize tasks when faced with multiple deadlines?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

What key performance indicators (KPIs) would you track for a new product launch?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

If given a dataset, what steps would you take to analyze it?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you design a data pipeline for real-time analytics?

Medium

Approach

Say what you would check first and why it is the highest-information step.
Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Randomise on account or profile for a personalisation test

Medium · randomisation unit · interference · clustered design

You are testing a personalisation change to the home row. Assignment could be on profile_id or on account_id. From dim_profile and fct_stream: 38 percent of accounts have two or more non-deleted profiles, those accounts hold 61 percent of qualified hours, and on multi-profile accounts 23 percent of qualified streams start on a profile other than the one that device last used. The primary metric is qualified hours per account-week. Choose the randomisation unit, name the interference you accept or avoid, and quantify what the choice costs in power.

Approach

Match the assignment unit to the estimand. The metric denominator is the account-week, so profile-level assignment splits a single metric row across both arms and leaves no defensible way to attribute that row.
Name the leakage mechanism concretely rather than gesturing at it: a shared TV or speaker plays on whatever profile is active, continue-watching and queue state are visible account-wide, and 23 percent cross-profile starts means a treated profile physically changes what a control profile on the same account is shown.
State the direction of the damage. Leakage makes control partially treated, so the estimate is attenuated toward zero and a null becomes uninterpretable, which is worse than a wider confidence interval.
Price the alternative with the clustered design effect, DEFF = 1 + (mbar - 1) * rho, where mbar is mean profiles per account and rho the intra-account correlation of profile-week hours, estimated from a one-way ANOVA on the pre-period.
Decide on the asymmetry: the clustering cost is bounded and computable, the dilution bias is neither. Randomise on account_id and serve the treated model to every profile inside a treated account.

Worked solution (25 min)

Compute mbar exactly from dim_profile with is_deleted = false rather than assuming it; with 38 percent of accounts at two or more profiles, roughly 1.5 profiles per account is the working figure.
Estimate rho as the intra-class correlation of profile-week qualified hours within account over four pre-period weeks: rho = between-account variance / (between + within). Around 0.4 is typical when a household shares taste and a device.
DEFF = 1 + (1.5 - 1) * 0.4 = 1.20, so account randomisation needs 20 percent more accounts for the same power, and at fixed sample size the MDE is sqrt(1.20) = 1.095 times larger, about 9.5 percent worse.
Write the decision in one sentence: accept a 9.5 percent wider MDE to avoid an attenuation of unknown size on the 61 percent of hours that sit on multi-profile accounts.

Expected result: Randomise on account_id. With mbar = 1.5 and rho = 0.4 the design effect is 1.20, costing 20 percent more accounts or a 9.5 percent larger MDE. The alternative is a profile-randomised test whose effect estimate is attenuated toward zero exactly on the accounts carrying most of the hours, with no way to correct it after the fact.

Verify it yourself

Recompute rho on accounts with three or more profiles. A rho near zero would mean profiles inside an account behave independently, so the clustering cost approaches nil, though it would not weaken the interference argument at all.
The account-level metric must be identical under either assignment scheme. If the reported denominator changes when you switch units, the metric definition is leaking the assignment unit.
Simulate a profile-randomised A/A on the pre-period and confirm that account rows are split across arms at roughly the rate implied by 38 percent of accounts holding two or more profiles.

Follow-up

Bound the leakage rather than assuming it: how would you use single-profile accounts as a comparison group to estimate the dilution in a profile-randomised test?
The serving layer keys on profile_id and cannot read account_id at request time. How do you get account-level assignment anyway?
Some households share one TV across two billing accounts. Does that break account randomisation, and how would you detect it?

Revenue per engaged account slid with no price change

Medium · mix-shift · segmentation · revenue

Net revenue per active account-month fell 7% across two closed months. No list price changed and no discount campaign ran. You have fct_subscription_period (account_id, period_start_ts, period_end_ts, plan_tier, billing_interval, net_amount_usd, currency_code, is_promotional, payment_status), dim_account (signup_country, plan_tier, billing_provider, first_paid_ts) and the qualified-stream denominator from fct_stream. Decide whether revenue per paying account fell or the denominator changed, and where. Deliverable: a decomposition across numerator and denominator naming the two largest contributing segments and whether each is reversible.

Approach

Split the ratio before splitting any segment. Revenue per engaged account equals revenue per paying account multiplied by paying accounts over engaged accounts. A growing free_ad_supported tier moves only the second factor and looks identical to a pricing problem in the headline.
Cut the numerator by signup_country and currency_code and recompute at both period-of-record and fixed exchange rates. If local list prices are unchanged, a simultaneous USD fall across many countries is FX, and the fixed-rate series says so in one line instead of a meeting.
Decompose revenue per paying account into within-segment change and mix over (plan_tier x billing_interval x signup_country) cells. Annual periods recognise pro rata across months, so a monthly-to-annual shift changes recognised revenue per account-month without changing what anybody paid over a year — a real accounting move with no economic content.
Treat the denominator's composition as its own problem. Engaged accounts are distinct accounts with a qualified stream, so an acquisition push into a low-price market or into the free tier adds denominator months before it adds revenue and dilutes the ratio mechanically for a knowable number of months.
Report the two segments carrying most of the move with their weight change and within-segment change shown side by side, and label which reverse on their own and which do not. That distinction is the actionable part of the answer; the 7% by itself is not.

Follow-up

The ratio dilutes for some months after an acquisition push. How many, and how do you present the metric so a marketing success is not read as a regression every time?
If the shift into annual billing is permanent, is the 7% real? What does the promotional-share guardrail say about it?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
On the third attempt, write the grain of every CTE as a comment before writing its body.
Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
Re-read only your own notes from this week, and open no new material.
Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

The project you argued against

Saying no well is a senior skill and it is rarely rehearsed. Think of a time you told someone their analysis was not worth doing, or that the experiment could not answer their question at the sample size available. Explain what you offered instead. Refusal without an alternative reads as obstruction rather than judgement.

Give an example of how you worked with a team to achieve a common goal.

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

State the impact of your last year without inflation

Hard · self-assessment · attribution · counterfactual reasoning

You are asked what your work was worth over the last year. Two candidates are on the table. First: a dunning-schedule change you analysed, shipped in March with no holdout, after which the involuntary share of gross churn fell three points. Second: a metric-definition change you drove, which nobody can attribute revenue to. Deliverable: the impact claim you make for each, the counterfactual attached to each, and which one you lead with in a performance review — with the reason.

Approach

The probe is whether you apply the same causal standard to your own work as to a product experiment. Attach the counterfactual before the claim: involuntary share of gross churn responds to card-updater coverage, billing_provider mix, retry schedules and the dunning horizon you read it at, any of which can move three points with the schedule change contributing nothing.
Say which evidence would separate those, then say plainly which you actually have. A staggered rollout by billing_provider, a holdout, or at minimum a flat pre-period series would each support a different strength of claim; with none of them, the honest claim is a contribution to a favourable movement, stated as such.
Check the classification itself before claiming the movement is real: the split between cancelled_voluntary and failed_involuntary must be computed only after the dunning horizon has closed for every period in the month, or retries still in flight are counted as churn and the share reads too involuntary in one direction and corrects in the other.
Price your contribution rather than the outcome. The value of analysis that changed a decision is the decision's delta multiplied by the probability the decision would not have been taken without it, and being explicit about that second factor is most of what honesty means here.
Make the definition-change claim concrete rather than apologetic: name the decisions that would have been taken on the wrong number, the reports it reconciled, the recurring argument it closed, with dates. Then lead with whichever claim survives questioning, not whichever carries the larger number, because an inflated first claim makes the second unbelievable.

Follow-up

Your manager writes up the churn improvement and credits it to you. Do you correct it, and to whom?
How would you have designed the March rollout so that attribution was possible, at what cost in delay?
Name something you worked on last year that had no impact, and say what you learned from that rather than from the wins.

Scope a one-line request about home row performance

Easy · scoping · stakeholder communication · metric definition

A director messages you: "Is the new home row working?" Nothing else. A new ranker_version has been serving a fraction of profiles for eleven days. You have fct_impression (surface, slate_position, ranker_version, is_exploration_slot, logging_propensity, experiment_assignment_id, was_clicked) and fct_stream (impression_id, start_source, is_qualified, played_seconds). You get a fifteen-minute call before they go into a rollout meeting. Deliverable: the three questions you ask before writing any SQL, the single primary metric you commit to with its guardrail, and the questions you tell them this data cannot answer.

Approach

The probe is whether you convert a vague request into a decision before producing a number. Ask what happens at each answer — rollback, widen, iterate — because a question whose answer changes nothing is a report request and should be scoped as one.
Pin the unit of analysis out loud. fct_impression is at (profile, slate, slot) grain and experiment_assignment_id is per assignment, so the comparison must be aggregated to the assignment unit first; comparing impression-level rates lets a change in slate length move the metric on its own.
Commit to one primary metric from the tree — qualified hours per active account-week for assigned accounts — and name the guardrail pair explicitly: share of qualified streams with start_source = 'autoplay_continuation', and median completion_ratio within content_type. A ranker can lift qualified stream counts by queueing short items that clear the 30-second threshold, and the guardrail is the only thing that catches it.
State the refusals with structural reasons, not time reasons: eleven days gives no matured cohort, so month-6 retention and net revenue per active account-month are unanswerable; and logging_propensity is populated only where is_exploration_slot = true, so the positivity condition for an off-policy estimate fails outside those slots.
Write the scope back in one paragraph — decision, metric, guardrail, the date the read becomes valid — and get it agreed in the thread before querying, so the number that arrives is the number that was asked for.

Follow-up

They reply "just give me click-through by slate position." What do you say, and what would that number actually tell them?
The eleven days include a weekend and a large release landing on day six. Does that change the metric you commit to, or only the read date?
What would have to be true for you to be willing to answer the retention question from this experiment?