Interview guides · Chaos Industries

Chaos Industries Data Scientist Interview Questions & Guide 2026

Case practice for Chaos Industries data scientist candidates: scope a vague prompt into a measurable question before you analyze it.

The 60-second brief

As a Data Scientist within the Mission Engineering team at Chaos Industries, you are tasked with bridging the gap between raw, complex data and high-stakes decision-making. You will work at the intersection of defense technology and advanced analytics, building models and analytical frameworks that directly impact the performance and reliability of our systems. Your work is not just about generating reports; it is about providing the quantitative backbone for mission-critical operations.

The boundary with engineering varies enough to be worth asking about directly. Some seats end at the analysis and the writeup; others carry feature pipelines, scheduling and alerting, which decides whether production data plumbing belongs in your prep at all.

Focus areas

Reconcile amounts in minor units and currency
Decompose expected loss into PD, LGD, EAD
Report only matured cohorts for loss metrics

Overview

This role requires a unique blend of technical rigor and product-sense. You will spend your time designing experiments, diagnosing performance drops in real-time, and translating ambiguous operational requirements into clear, measurable metrics. Because Chaos Industries operates in a fast-paced, high-impact environment, you will need to demonstrate the ability to navigate uncertainty while maintaining a deep commitment to statistical integrity.

Candidates who thrive here are those who view data as a tool for engineering excellence. You will collaborate closely with hardware and software engineers to ensure that every decision—from sensor calibration to system deployment—is backed by solid empirical evidence. It is a challenging role that demands high ownership, a proactive mindset, and a genuine passion for solving complex, real-world problems.

The Chaos Industries interview process

Chaos Industries candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Deep Dives

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Collaborative Problem Solving

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Behavioral Assessments

Most of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.

What to demonstrate

Whether you can state the other side's argument accurately before you explain why you disagreed
What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
Whether you distinguish being overruled from being wrong, and can give an example of each

How to prepare

Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.

Where candidates lose points

Averaging delinquency across a book that is growing

A loan three months old cannot be 90 days past due, so a portfolio with many recent originations reports a low blended 90+ rate purely from age mix. The blended rate falls fastest exactly when originations grow fastest, which is precisely when credit quality most needs watching, so the metric moves in the reassuring direction during the riskiest period. Only comparisons at equal months on book are valid, which is what a vintage or roll-rate view enforces.

Recalibrating an underwriting cutoff on approved and funded applicants only

Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Solving silently instead of narrating the reasoning

Say which branch you are taking and why you chose it over the alternative, for example checking the denominator first because it changes what the comparison means. A correct answer that arrives with no visible path scores below a rigorous one that needed a hint.

Questions & practice

How do you balance the trade-off between model accuracy and system latency in a real-time environment?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Say how the offline result would be validated online before it is trusted.
Check what information would not exist at prediction time, and exclude it.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Bootstrap a fraud loss rate that clusters within merchant

Medium · bootstrap · clustered resampling · heavy tails

You have a per-transaction frame with auth_id, merchant_id, settled_amount_reporting and net_loss_reporting, both already in one reporting currency. Most rows carry zero loss, a few carry large ones, and losses cluster within merchant. Using only numpy's random generator and no resampling helper from any library, write a bootstrap that returns a 95 percent interval for net fraud loss in basis points of settled volume, resampling merchants with replacement and taking all rows belonging to each drawn merchant. Also produce the naive row-level interval and state which you would report.

Approach

State the estimator before writing it: total net loss divided by total settled volume, times 10,000. It is a ratio of sums, so each replicate recomputes both sums. Averaging per-transaction loss rates instead would weight a five-unit transaction like a five-thousand-unit one.
Pre-aggregate loss and volume to merchant level once. For a ratio of sums, drawing merchants and taking all their rows is arithmetically identical to drawing merchant-level (loss_sum, volume_sum) pairs, so a replicate becomes one integer draw plus two vectorised sums rather than a groupby inside the loop.
Draw B replicates of M merchant indices with replacement, where M is the observed merchant count, compute the ratio per replicate, and take the 2.5th and 97.5th percentiles. Say explicitly that this is a percentile interval and that BCa would correct the skew-induced bias if the decision is close.
Repeat with independent row draws for the naive interval and compare widths on the same replicate count.
Report the clustered interval. Rows within a merchant share an acceptance profile, a category code and a fraud exposure, so they are not independent, and the row-level interval understates variance by roughly the design effect.

Follow-up

Your clustered interval is three times wider. How do you explain that to someone who wanted a tighter number?
One merchant accounts for 40 percent of losses. What does that do to the interval, and what would you do about it?
How does this change if the question is whether two months differ rather than what this month's rate is?

Collapse retry chains and compute a dollar-weighted approval rate

Medium · sessionisation · window functions · dollar-weighted rates

fct_payment_authorization gives auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, requested_at, auth_result, is_reversal, channel and issuer_country. Two reference frames give the minor-unit exponent per currency and a daily rate to one reporting currency. Collapse retry chains first: attempts sharing card_token_id, merchant_id and amount_minor whose consecutive gaps are under 15 minutes form a single attempt, whose outcome is its last row. Exclude reversals and zero-amount verifications. Return a 7-day rolling dollar-weighted approval rate by channel and issuer_country.

Approach

Filter before grouping: drop is_reversal rows and zero-amount verifications, since neither is a purchase attempt and both would otherwise sit in the denominator.
Sort by card_token_id, merchant_id, amount_minor and requested_at, take the gap to the previous row within that key, mark a chain start where the gap exceeds 15 minutes or the key changes, and label chains with a cumulative sum of that flag. This is a gap rule between consecutive attempts, not a fixed clock bucket, so a chain may span more than 15 minutes in total.
Keep each chain's terminal row by requested_at. If a retry was approved, the purchase was approved; keeping the first row reports the decline that caused the retry as the outcome.
Convert amounts exactly once: amount_minor divided by 10 to the power of the currency exponent, multiplied by the reference rate for the authorization date. Do not reach for settlement_fx_rate, which is null on precisely the declined rows the denominator needs.
Build the rolling window as a ratio of two rolling sums, approved value over total value, per channel and issuer_country. A rolling mean of daily ratios weights a quiet Sunday the same as a busy Friday.

Worked solution (35 min)

Filter out reversals and zero-amount rows, then sort by the chain key and requested_at.
Compute the within-key time difference, derive the chain start flag and the chain id, and take the last row per chain with groupby(chain_id).tail(1) after sorting.
Join the exponent and daily rate tables, compute value_reporting, and assert no nulls remain after the join.
Aggregate approved value and total value to a daily grain by channel and issuer_country, reindex to a complete date range per group so missing days are zero rather than absent.
Take 7-day rolling sums of both columns and divide, then confirm one hand-picked group-day against a direct filter.

Expected result: A DataFrame keyed by date, channel and issuer_country with approved_value, total_value and approval_rate. The collapsed attempt count is materially below the raw row count, with the gap concentrated in declined ecommerce rows, and ecommerce sits below card_present.

Verify it yourself

Within every chain, all consecutive requested_at gaps are under 15 minutes; across chains with the same key, at least one gap is not.
Chain count is strictly below raw attempt count, and the difference is concentrated in declined rows rather than spread evenly.
Total converted value on a single-currency single-day subset equals the raw amount_minor sum divided by that currency's exponent power, times that day's rate.
The rolling rate on one chosen day matches a direct filter-and-divide over the preceding seven days.

Follow-up

The count-weighted rate is flat while the dollar-weighted rate falls 80 basis points. What do you look at first?
How would you choose the 15-minute window rather than inheriting it?
A merchant moves from two retries to five. Which of your two rates moves, and is that a real change in approval quality?

Describe how you would use SQL window functions to calculate rolling averages or identify time-series anomalies.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Given two tables, one containing raw telemetry and the other event logs, how would you join and aggregate them for a comparative analysis?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Count-weighted and dollar-weighted approval rates on one currency

Easy · aggregation · denominators · minor units

Using fct_payment_authorization, report the trailing 7-day authorization approval rate two ways for transaction_currency = 'EUR': count-weighted, and dollar-weighted on amount_minor. Exclude is_reversal = true, exclude incremental authorizations (parent_auth_id not null), and exclude zero-amount account verifications. auth_result = 'approved' is the numerator; the four declined_* values make up the rest of the denominator. Return channel, attempts, approved_attempts, approval_rate_count and approval_rate_value. State every exclusion and its reason before you write the SELECT.

Approach

Say the denominator out loud first: attempts on a single transaction currency, excluding reversals, incremental authorizations and zero-amount verifications, because none of those is a purchase attempt a merchant is trying to get approved.
Filter requested_at against a half-open interval (>= start AND < end) so the boundary day is neither dropped nor double counted.
Compute both rates in one pass with FILTER clauses: COUNT() FILTER (WHERE auth_result = 'approved') over COUNT(), and SUM(amount_minor) FILTER (WHERE auth_result = 'approved') over SUM(amount_minor).
Cast one side of each ratio to numeric before dividing, since amount_minor and the counts are integers and integer division silently truncates to zero.
Group by channel and sort by the value-weighted rate, then read the gap between the two rates as a statement about where the declines sit rather than as noise.

Worked solution (20 min)

Write the exclusion list as comments above the query: is_reversal = false, parent_auth_id is null, amount_minor > 0, transaction_currency = 'EUR'.
Build a single aggregate query over fct_payment_authorization with a half-open requested_at predicate and those four filters.
Emit attempts, approved_attempts, approval_rate_count and approval_rate_value with FILTER clauses and a numeric cast on the numerator.
Group by channel, order by approval_rate_value ascending so the worst channel is on top.

Expected result: One row per channel present in the window, with attempts and approved_attempts as integers and both rates as numerics strictly between 0 and 1. approval_rate_value is typically lower than approval_rate_count on ecommerce and moto, because declines concentrate in higher-value card-not-present traffic.

Verify it yourself

SUM(attempts) across channels equals a separate COUNT(*) run with the same four filters and no GROUP BY.
Dropping the amount_minor > 0 filter raises attempts and moves approval_rate_count while leaving approval_rate_value unchanged, which confirms the zero-amount rows were only ever affecting the count-weighted figure.
Both rates are strictly less than 1 unless a channel had no declines at all in the window.

Follow-up

The two rates diverge by four points on the ecommerce channel but agree on card_present. What does that tell you, and what would you cut next?
How would you extend this to all currencies without summing amount_minor across them?
Which of the four decline reasons belong in the denominator of a rate you would put in front of a risk team, and which are really the network's problem?

How would you define the success of our new mission-critical platform?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

Design a metric to measure the reliability of a sensor array.

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

If a key performance metric drops suddenly, what is your systematic approach to diagnosing the root cause?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Walk me through the steps to design an A/B test for a system update.

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
State the primary metric and the minimum effect worth shipping, then size the test.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
How would you handle interference between treated and control units?

What are the most common experimentation pitfalls that could lead to false positives?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

How would you evaluate the effectiveness of a feature update before a full deployment?

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.
Work from the decision backwards to the evidence you would need.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Design an authorization approval rate a team cannot game

Easy · approval rate · metric gaming · mix shift

A payments team is given authorization approval rate as its quarterly target, defined as approved authorizations over all authorizations in fct_payment_authorization. Using auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, auth_result, is_reversal, parent_auth_id, channel, issuer_country and requested_at, list three ways the team can raise that number without a single additional customer purchase succeeding. Then write the definition you would put on the dashboard instead, with its exclusions, and name the one guardrail you would always read beside it.

Approach

Enumerate the rows in the table that are not purchases: zero-amount account verifications, reversals (is_reversal = true), incremental authorizations (parent_auth_id not null) and merchant retries of a declined attempt. Each is a lever on the numerator or the denominator that no customer would ever notice.
Name the mix levers. Recurring and card-present traffic approve at higher rates than ecommerce and cross-border, so routing changes or a marketing push into an easy channel move the blended rate with no change in decisioning quality at all.
Work the retry arithmetic rather than asserting it. A declined purchase that is retried three times and succeeds on the fourth contributes one approval and four denominator rows, which is below a healthy blended rate, so suppressing retries raises the published number while some purchases that would have succeeded on a retry now simply fail.
Replace the definition: dollar-weighted, retry chains collapsed to a single attempt per (card_token_id, merchant_id, amount_minor) within a 15-minute window, excluding reversals, zero-amount verifications and incremental authorizations, with all amounts converted into one reporting currency before summing because amount_minor is in the minor unit of transaction_currency and that unit is not always two decimals. Cut by channel and issuer_country so mix shift shows up as mix shift.
Guardrail: net fraud loss in basis points of settled volume, read only on transaction months with at least 120 days of dispute maturity. The fastest way to approve more is to decline less risk, and that bill arrives two to four months later.

Worked solution (25 min)

Compute the naive rate for one week: approved rows over all rows.
Apply the exclusions one at a time (zero-amount, is_reversal, parent_auth_id not null), recording the rate after each, so you can say what each row type was worth to the headline number.
Add a retry-collapsing CTE keyed on (card_token_id, merchant_id, amount_minor) over a 15-minute window, keeping the last attempt in each chain, and recompute.
Recompute dollar-weighted in a single reporting currency, then split by channel and issuer_country.
Write the final one-line definition with its exclusions, plus the guardrail sentence.

Expected result: Four rates for the same week (naive count, count after exclusions, count after retry collapsing, dollar-weighted after both), with the largest single gap attributed to a named row type, and a published definition that states every exclusion and the currency it reports in.

Verify it yourself

Excluding zero-amount verifications should lower the rate, since those rows approve at a higher rate than purchases. If it raises the rate, confirm that the amount_minor = 0 filter is not also catching rows with a null amount.
Collapsing retry chains raises the rate and shrinks the denominator. The numerator falls only by the number of chains that contained more than one approval.
Declines skew toward higher-value, cross-border and card-not-present transactions, so expect the dollar-weighted rate to sit below the count-weighted one. If it sits above, check for amounts summed across currencies before believing it.

Follow-up

The blended rate is flat but ecommerce cross-border fell 200 basis points. How do you present that?
Why weight by amount rather than by count, and when does the weighted version mislead?
Your 15-minute retry window merges two genuine purchases of the same amount at the same merchant. How would you bound that error?

Fraud losses appear to halve in recent transaction months

Medium · right censoring · development factors · maturity

A weekly chart attributes fct_card_dispute cases to the requested_at month of the linked fct_payment_authorization row. The two most recent months show the first-chargeback rate falling by half, and a risk rule shipped six weeks ago. Columns: dispute_id, auth_id, dispute_category, dispute_stage, opened_at, disputed_amount_minor, liability_shift_flag, outcome, net_loss_minor, resolved_at. Decide whether the rule worked, and produce the version of the chart you would sign your name to.

Approach

Separate the two dates explicitly. opened_at is when a case was filed, requested_at is when the transaction happened. Attributing by transaction month is the right causal choice and is exactly what makes the newest months structurally incomplete.
Measure the filing lag rather than assuming it: the distribution of opened_at minus requested_at over fully developed months, split by dispute_category, and the age at which around 95 percent of cases have arrived.
Build a development triangle of transaction month by months of development on cumulative case counts, and estimate age-to-age factors from the columns that are complete.
Develop the immature months with those factors and plot the result as an estimate with a visible band, kept visually distinct from the matured series rather than blended into it.
State the assumption the method needs: a stable development pattern across cohorts. A change in filing behaviour, merchant mix or the dispute team's own backlog breaks it, so inspect factor stability down each column before relying on the estimate.
Only then evaluate the rule, comparing pre-change and post-change cohorts at equal development age.

Follow-up

What leading indicator would you accept while the cohort matures, and what is its known bias?
How does liability_shift_flag change which disputes you should expect to see in the first place?
If the rule also blocked good transactions, where does that cost appear, and is any of it in this chart?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Bring numbers, including the ones you are unsure of

An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.

How do you handle missing or noisy data in a production pipeline?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

State honestly what your cutoff change actually contributed

Hard · impact attribution · swap set · counterfactual

Six months ago your recommendation moved a credit cutoff, using fct_loan_application and fct_loan_performance_monthly. Since then approval rate rose four points and the 12-month vintage 90-plus rate on affected cohorts is flat. In the same window the bureau changed a score attribute, marketing shifted channel mix toward broker, and the internal funding rate moved. Your performance review asks for impact in currency terms. Give the number you would stand behind, the counterfactual it rests on, and the part of the observed movement you would not claim.

Approach

Define the counterfactual before computing anything: the claim is not what happened after the change, it is what would have happened had the old cutoff scored the same applications, which means replaying the old threshold on the post-change population.
Build the swap set: applications the new cutoff approves that the old one declined, applications the old one approved that the new one declines, and everyone else held out as unaffected. Only the swap groups carry your effect. Note that the swap-out group has no outcome under the new rule, because those loans were never funded, so its forgone margin must be estimated from matched pre-change approvals rather than observed.
Price each swap group at months_on_book equal to 12 on the measure the cutoff was meant to move: interest and fees collected, minus net charge-offs, minus funding cost at the internal transfer rate.
Strip the confounders explicitly. Cohorts affected by the bureau attribute change are either recomputed on the old attribute or excluded; channel mix is held fixed by reweighting to the pre-change mix; funding cost is charged at the rate in force each month rather than one blended average.
State the residual you will not claim, with its size, and give a range rather than a point wherever cohorts have not yet reached 12 months on book.

Follow-up

The swap-in group is only 6 percent of applications. How does that change the way you present the number, and to whom?
What would you have needed to set up at launch to make this attribution clean, and why was a randomised band around the cutoff not used?

Allocate one analyst-week across three competing risk requests

Medium · prioritisation · decision deadlines · stakeholder negotiation

Three requests land in the same week and you have one analyst-week. Payments wants a merchant-level decline teardown before a contract renewal in nine days. Credit wants a swap-set analysis on a cutoff change scheduled to ship in six weeks. Insurance wants accident-quarter loss ratios at 12 months development for a reserving review with no fixed date. Each sponsor believes theirs is first, and each has escalated before. Produce the allocation, the reasoning you would say out loud to all three at once, and what you explicitly drop.

Approach

Score each request on the decision it unblocks rather than on effort or on how loudly it arrived: what changes if it is late, and is that change reversible.
Separate deadline from value. The nine-day renewal is a hard, irreversible date with a bounded prize; the six-week cutoff has slack but a much larger downside if it ships unmeasured; the reserving number has no date but feeds external reporting, which is its own kind of hard.
Hunt for the cheap partial in each: a decline teardown restricted to the top merchants by declined value usually answers the contract question at a fraction of the full cut.
Sequence by hard date first, then by largest irreversible downside, and deliver the trade-off to all three sponsors in one message rather than three, so nobody negotiates privately against a version you told someone else.
Name what is dropped and who now owns that consequence, in writing, so the trade-off is visible rather than silently absorbed by you.

Follow-up

The credit sponsor escalates to your manager. What do you change, and what do you refuse to change?
How would you make this allocation reproducible so the next contested week is a rule application rather than a negotiation?