Capital on Tap Data Scientist Interview Questions & Guide 2026

The 60-second brief

A Data Scientist at Capital on Tap occupies a high-impact position at the intersection of financial technology and data-driven product strategy. Your work is critical to the company’s mission of providing seamless financing solutions to small businesses. By leveraging large-scale transaction data, you will build models and insights that directly influence how the company manages risk, detects fraud, and optimizes product features.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

• Set fraud thresholds by expected cost
• Separate authorization, settlement and dispute outcomes cleanly
• Report only matured cohorts for loss metrics

Overview

This role is not merely about running queries; it is about acting as a strategic partner to the business. You will be expected to bridge the gap between complex statistical analysis and actionable product decisions. Whether you are analyzing user behavior to improve retention or developing robust monitoring systems to mitigate sanctions and fraud risk, your contributions will have a tangible effect on the company’s bottom line and the success of its customers.

The provided compensation data reflects the total package expectations for this role in the UK market. Candidates should interpret these figures as a blend of base salary and potential performance incentives, noting that seniority significantly influences the upper bounds of these ranges. Use this as a benchmark to ensure your expectations align with the market standard for high-growth fintech environments.

The Capital on Tap interview process

Capital on Tap candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screening Call

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

• Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
• Whether you name what you have not done instead of stretching to cover every line of the posting
• Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

• Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
• Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
• Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Assessments

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

• Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
• Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
• Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

• Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
• Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
• If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Situational Assessments

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

• Whether the query you write matches the plan you just described
• What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
• Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

• Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
• Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
• Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Where candidates lose points

Averaging delinquency across a book that is growing

A loan three months old cannot be 90 days past due, so a portfolio with many recent originations reports a low blended 90+ rate purely from age mix. The blended rate falls fastest exactly when originations grow fastest, which is precisely when credit quality most needs watching, so the metric moves in the reassuring direction during the riskiest period. Only comparisons at equal months on book are valid, which is what a vintage or roll-rate view enforces.

Assuming a model is fair because protected attributes are not among its inputs

Postcode, device, tenure, income proxies and even transaction patterns correlate with protected characteristics, so a model can produce a disparate outcome without ever reading the attribute. Credit decisions additionally carry an explainability obligation in many jurisdictions, since a denial has to be accompanied by its principal reasons, which constrains model form and feature engineering rather than being a reporting afterthought. Treating fairness testing and reason-code generation as design constraints from the first model version is far cheaper than retrofitting them to a deployed one.

Explaining an aggregate move without decomposing the mix shift

Split the change in the aggregate into within-segment movement and movement in segment weights before you explain it. Every segment's rate can fall while the overall rate rises, purely because volume shifted toward segments that already had higher rates.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Questions & practice

Estimate a delinquency roll-rate matrix and project twelve months

Hard · roll rates · markov chain · survivorship

fct_loan_performance_monthly gives loan_id, as_of_month_end, months_on_book, delinquency_bucket, charge_off_flag, prepaid_in_full_flag and restructured_flag. Build a month-to-month transition matrix over the five delinquency buckets plus absorbing charged_off and prepaid states. Loans that stop appearing must be routed to an absorbing state rather than dropped. Project the current book forward 12 months by repeated matrix multiplication and report the projected share reaching charge-off. Handle restructured_flag explicitly, and name one place the Markov assumption fails on this data.

Approach

1. Build consecutive month pairs per loan by shifting as_of_month_end within loan_id, then verify the shifted value is exactly one month later. A gap is not a transition, it is an exit you have not resolved yet.
2. Resolve exits before counting anything. A loan whose last row carries charge_off_flag moves to charged_off, one carrying prepaid_in_full_flag moves to prepaid, and one that disappears with neither is a data question to raise rather than silently discard, because discarding it is survivorship that inflates every cure rate.
3. Count pairs into a 7 by 7 matrix and row-normalise. Assert every row sums to one and the two absorbing rows are the identity; a row that does not sum to one means exits were dropped.
4. Decide and state the restructure rule. Restructuring resets days_past_due, so a dpd_60_89 to current move on a restructured loan is not a cure. Either give restructured loans their own state or carry the pre-restructure bucket, but do not let that move land in the cure cell.
5. Project by taking the current bucket distribution as a row vector and multiplying by the matrix twelve times. Report the charged_off entry, and report it again from an all-current starting vector so the reader can see how much of the projection comes from loans that are already delinquent today.
6. State the homogeneity failure plainly: transition rates depend strongly on months_on_book, so one pooled matrix applied to a book with a young mix understates early-life delinquency. If the mix is moving, estimate separate matrices by seasoning band.

Worked solution (45 min)

1. Sort by loan_id and as_of_month_end, shift to form (from_state, to_state) pairs, and flag pairs whose month gap is not exactly one.
2. For each loan's final row, assign the absorbing destination from charge_off_flag or prepaid_in_full_flag, and list loans that vanish with neither as an exception count to report.
3. Apply the restructure rule, then build the 7 by 7 count matrix with a cross-tabulation over ordered state categories and row-normalise it.
4. Assert row sums equal one and absorbing rows are the identity, then take the current month's bucket distribution as a row vector.
5. Multiply twelve times, report the charged_off component, and repeat from an all-current vector for comparison.

Expected result: A 7 by 7 row-stochastic matrix with identity rows for charged_off and prepaid, a current-to-current diagonal typically above 0.95, roll rates rising with bucket depth, and a 12-month projected charge-off share of a few percentage points from an all-current start and materially higher from the actual book.

Verify it yourself

• Every row sums to one within floating-point tolerance, and both absorbing rows are exactly the identity.
• No mass sits in cells that skip a bucket upward, such as dpd_1_29 to dpd_60_89 in one month; if there is, the pairs are not actually consecutive months.
• The projection from an all-current start is strictly below the projection from the real book distribution.
• Backtest by estimating on months up to t minus 12 and comparing the projected distribution with the realised one at t; a systematic optimistic gap points at dropped exits or at seasoning, not at noise.

Follow-up

• How would you validate the projection against what actually happened, and over what window?
• The cure rate out of dpd_30_59 rose five points last quarter. What are the candidate explanations and how would you separate them?
• When would you prefer a vintage curve to a roll-rate projection, and why?

Collapse retry chains and compute a dollar-weighted approval rate

Medium · sessionisation · window functions · dollar-weighted rates

fct_payment_authorization gives auth_id, card_token_id, merchant_id, amount_minor, transaction_currency, requested_at, auth_result, is_reversal, channel and issuer_country. Two reference frames give the minor-unit exponent per currency and a daily rate to one reporting currency. Collapse retry chains first: attempts sharing card_token_id, merchant_id and amount_minor whose consecutive gaps are under 15 minutes form a single attempt, whose outcome is its last row. Exclude reversals and zero-amount verifications. Return a 7-day rolling dollar-weighted approval rate by channel and issuer_country.

Approach

1. Filter before grouping: drop is_reversal rows and zero-amount verifications, since neither is a purchase attempt and both would otherwise sit in the denominator.
2. Sort by card_token_id, merchant_id, amount_minor and requested_at, take the gap to the previous row within that key, mark a chain start where the gap exceeds 15 minutes or the key changes, and label chains with a cumulative sum of that flag. This is a gap rule between consecutive attempts, not a fixed clock bucket, so a chain may span more than 15 minutes in total.
3. Keep each chain's terminal row by requested_at. If a retry was approved, the purchase was approved; keeping the first row reports the decline that caused the retry as the outcome.
4. Convert amounts exactly once: amount_minor divided by 10 to the power of the currency exponent, multiplied by the reference rate for the authorization date. Do not reach for settlement_fx_rate, which is null on precisely the declined rows the denominator needs.
5. Build the rolling window as a ratio of two rolling sums, approved value over total value, per channel and issuer_country. A rolling mean of daily ratios weights a quiet Sunday the same as a busy Friday.

Follow-up

• The count-weighted rate is flat while the dollar-weighted rate falls 80 basis points. What do you look at first?
• How would you choose the 15-minute window rather than inheriting it?
• A merchant moves from two retries to five. Which of your two rates moves, and is that a real change in approval quality?

Build a vintage delinquency table without pivot or unstack

Easy · vintage analysis · cohorts · groupby · cumulative max

fct_loan_performance_monthly gives loan_id, origination_month, months_on_book, days_past_due, charge_off_flag and restructured_flag. Produce a DataFrame with one row per origination_month and columns for months_on_book 0 through 12, each cell holding the share of that vintage's funded loans that had ever reached 90 or more days past due, or charge-off, by that age. You may not use pivot, pivot_table, crosstab or unstack. Cells for ages a cohort has not yet reached must be NaN rather than zero.

Approach

1. Define the per-row indicator as days_past_due >= 90 or charge_off_flag, then take a cumulative maximum of it per loan ordered by months_on_book, because the metric is reached-by-age-m, not in-that-state-at-age-m.
2. Deal with restructuring before the cumulative max. Restructuring resets days_past_due, so a restructured loan re-enters at current and, without the cumulative maximum carrying its pre-restructure worst state, reads as a cure.
3. Fix the denominator once as the count of distinct loan_id per origination_month across the whole cohort. Prepaid and charged-off loans stop producing rows, so a denominator recomputed at each age silently shrinks exactly where losses land.
4. Aggregate with groupby(['origination_month','months_on_book'])['ever_90'].sum(), then pre-build the output frame indexed by sorted origination months with integer columns 0 to 12 and assign from the grouped Series by .loc on its index.
5. Mask cells beyond each cohort's maximum observed months_on_book so an immature cell reads NaN instead of an artificially low rate.

Follow-up

• Two adjacent vintages diverge at months_on_book 6. How would you separate seasoning, mix shift and a genuine credit-quality change?
• The three most recent vintages look best on this table. What do you check before saying so?
• How does the table change if charge-off policy moved from 180 to 120 days past due partway through the series?

When would you choose to use a Common Table Expression (CTE) over a temporary table or a subquery?

Medium · sql and data manipulation

Approach

1. Check whether any join is one-to-many before aggregating, or the sums inflate.
2. State the window function and its partition and ordering out loud before writing it.
3. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

• What breaks if events arrive late or out of order?
• How would you verify this result without re-running the same query?

How do you optimize a query that is performing slowly on a dataset with millions of transactions?

Medium · sql and data manipulation

Approach

1. State the window function and its partition and ordering out loud before writing it.
2. Compute rates by summing numerator and denominator separately, never by averaging rates.
3. Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

• How would you verify this result without re-running the same query?
• What breaks if events arrive late or out of order?

Explain the practical differences between row_number, rank, and dense_rank in window functions.

Medium · sql and data manipulation

Approach

1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
2. Check whether any join is one-to-many before aggregating, or the sums inflate.
3. State the window function and its partition and ordering out loud before writing it.

Follow-up

• What breaks if events arrive late or out of order?
• How would you verify this result without re-running the same query?

Collapse merchant retry chains before measuring authorization approval

Medium · window functions · gaps and islands · sessionisation

Merchant retry logic turns one declined purchase into several rows in fct_payment_authorization. Collapse attempts into chains: inside a partition of (card_token_id, merchant_id, amount_minor), consecutive rows whose requested_at gap is 15 minutes or less belong to the same chain. Return one row per chain with chain_started_at, attempt_count, amount_minor, channel, and chain_outcome set to 'approved' when any attempt in the chain was approved. Then report a 7-day rolling dollar-weighted approval rate by channel over chains rather than attempts. Exclude is_reversal rows.

Approach

1. In a CTE, take LAG(requested_at) OVER (PARTITION BY card_token_id, merchant_id, amount_minor ORDER BY requested_at) and set is_new_chain when the previous value is null or the gap exceeds 15 minutes.
2. Turn the flag into a chain identifier with SUM(is_new_chain::int) OVER (same partition, same order, rows unbounded preceding to current row), which is the standard gaps-and-islands construction.
3. Aggregate to one row per chain: MIN(requested_at), COUNT(*), and BOOL_OR(auth_result = 'approved') as the chain outcome, because the purchase succeeded if any attempt in the chain did.
4. Roll the chains to a daily grain per channel, then apply the rolling window as PARTITION BY channel ORDER BY chain_date RANGE BETWEEN INTERVAL '6 days' PRECEDING AND CURRENT ROW. An interval-offset RANGE frame is value-based: it admits every row in the partition whose chain_date falls in the closed window [D - 6 days, D], so a channel with no chains on some dates still gets the right seven-day sums — sparsity is exactly what this construct tolerates. ROWS BETWEEN 6 PRECEDING AND CURRENT ROW is the one that needs a dense grain, because it counts rows and on a sparse grain reaches back further than seven days.
5. Left-join a dense date spine only if the output needs a row on days with no chains. That is a presentation requirement, not a correctness one, and it leaves every rolling sum unchanged. The RANGE frame's preconditions are that chain_date is a date or timestamp (an interval offset needs an ordering column it can be added to) and that the aggregation left one row per (channel, chain_date).
6. Divide approved chain value by total chain value inside the rolling frame, never the average of daily rates, since averaging rates weights a quiet day the same as a heavy one.

Worked solution (40 min)

1. CTE 1: filter out is_reversal rows, then compute prev_requested_at with LAG and the is_new_chain flag.
2. CTE 2: build chain_id with a running SUM of the flag over the same partition and order.
3. CTE 3: group by partition keys plus chain_id to emit chain_started_at, attempt_count, amount_minor, channel and chain_outcome.
4. CTE 4: aggregate chains to (chain_date, channel) with approved value and total value. Add the date spine here only if the report must show empty days.
5. Final SELECT: two rolling SUMs over PARTITION BY channel ORDER BY chain_date RANGE BETWEEN INTERVAL '6 days' PRECEDING AND CURRENT ROW, divided at the end.

Expected result: A rolling series with one row per date and channel, where the chain count is strictly below the attempt count on ecommerce and recurring, and the chain-level approval rate is at or above the attempt-level rate for every channel, since collapsing removes failed attempts that later succeeded.

Verify it yourself

• SUM(attempt_count) over all chains equals the filtered attempt row count exactly; if it does not, the chain identifier is not partitioned the same way as the LAG.
• Every chain has exactly one row whose is_new_chain flag is true, so COUNT(DISTINCT chain_id) equals the number of flagged rows.
• A chain whose attempt_count is 1 must have chain_outcome equal to that single row's approved status.
• The rolling denominator on day D equals the sum of the daily denominators over every date present between D-6 and D inclusive — seven terms on a dense grain, fewer on a sparse one, and the same total either way.
• Adding or removing the date spine leaves every rolling sum identical, because spine rows contribute zero. Swapping RANGE for ROWS changes them on any channel whose grain has a gap, which is the direct test of which construct tolerates sparsity.

Follow-up

• This definition chains off the previous attempt, so a chain can span an hour if attempts arrive every 14 minutes. When is that right, and when would you anchor the window to the first attempt instead?
• Two genuine purchases of the same amount at the same merchant, four minutes apart, collapse into one chain. How large is that error and how would you bound it?
• A channel goes dark for three days mid-window. Which of the two frame types would silently widen, and what would the resulting rate look like?
• What changes if the merchant retries with a slightly different amount to dodge a velocity rule?

How would you design a metric to measure the success of a new feature for small business owners?

Medium · product sense and metrics

Approach

1. Restate the decision this analysis has to support, and who acts on the answer.
2. Fix the population and the time window before naming any metric.
3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

• What would you do if the primary metric and the guardrail moved in opposite directions?
• How would you detect that the metric is being gamed rather than genuinely improving?

How do you balance the trade-off between user experience and risk management in a financial product?

Medium · product sense and metrics

Approach

1. State what result would change your recommendation, so the answer is falsifiable.
2. Decompose the metric into the rates that drive it, and say which one you would check first.
3. Fix the population and the time window before naming any metric.

Follow-up

• What would you do if the primary metric and the guardrail moved in opposite directions?
• Which segment would you cut first, and what would that rule out?

If we notice a sudden, unexplained drop in a key product metric, what steps do you take to diagnose the root cause?

Medium · product sense and metrics

Approach

1. State what result would change your recommendation, so the answer is falsifiable.
2. Fix the population and the time window before naming any metric.
3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

• How would you detect that the metric is being gamed rather than genuinely improving?
• Which segment would you cut first, and what would that rule out?

How do you determine the required sample size for an experiment to ensure statistical significance?

Medium · a/b testing and experimentation

Approach

1. Say whether units interfere with each other, and switch design if they do.
2. State the primary metric and the minimum effect worth shipping, then size the test.
3. Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

• What would you conclude if the result is positive but the test is underpowered?
• How would you handle interference between treated and control units?

What are the most common biases you watch for when designing experiments for transaction-based data?

Medium · a/b testing and experimentation

Approach

1. Decide the analysis before seeing data, including how long it runs and when you look.
2. State the primary metric and the minimum effect worth shipping, then size the test.
3. Say whether units interfere with each other, and switch design if they do.

Follow-up

• What would you do if you could not randomise at all?
• How would you handle interference between treated and control units?

Judge a rate increase without letting price flatter the ratio

Hard · mix shift · adverse selection · loss development

A rating change raises premium on one product_line's renewal book. Using fct_policy_period_monthly (earned_premium_minor, written_premium_minor, exposure_units, paid_loss_minor, case_reserve_minor, ibnr_reserve_minor, loss_adjustment_expense_minor, rating_tier, policy_status, renewal_flag, policy_term_end), the accident-period loss ratio improves two points over the following year. Explain why that number alone cannot tell you the change worked, decompose the movement into the effects you would separate, and specify the primary metric and guardrails you would commit to before the next rating change.

Approach

1. Start with the arithmetic. A rate increase raises earned premium per exposure unit, so the loss ratio falls even if every policyholder behaves identically and every claim is unchanged. Part of the two points is mechanical and carries no information about risk at all.
2. Switch the risk measure to one price cannot move: pure premium, being incurred losses (paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, with loss_adjustment_expense_minor included or excluded consistently and the choice stated) divided by exposure_units, evaluated by accident period at a fixed development age. Flat pure premium alongside an improved loss ratio means the improvement was entirely price.
3. Decompose the loss ratio movement into three named components: the price effect at constant exposure and mix, the mix effect from which rating_tiers renewed and which lapsed, and the residual change in pure premium within tier. Only the third is evidence about risk selection, and it is usually the smallest.
4. Name the adverse-selection risk directly. Price sensitivity and loss propensity are not independent, and the policyholders most able to leave after a rate rise are often the ones worth keeping. Retention by rating_tier is therefore a guardrail with teeth, and it has to be read at tier level because a flat blended retention hides tiers moving in opposite directions.
5. Add prior-period reserve development as the second guardrail. The same two points are producible by setting case reserves or IBNR light, which surfaces only later as adverse development, so the reserve guardrail is what stops the primary metric being satisfiable by an accounting choice.
6. Commit the primary before the next change: underwriting margin per exposure unit, being earned premium minus incurred losses minus loss adjustment expense minus allocated expense, over exposure_units, at a fixed development age, reported by accident period and by rating_tier with exposure volume printed beside it so that improving margin by shrinking the book is visible in the same table.

Worked solution (40 min)

1. By accident quarter at a fixed twelve-month development age, compute both the loss ratio (incurred over earned_premium_minor) and the pure premium (incurred over exposure_units), before and after the change.
2. Build the three-way decomposition: move price only at constant mix and exposure, then move the tier mix to the post-change distribution at constant price, then take the remainder as the within-tier pure premium change.
3. Compute renewal retention by rating_tier from policies reaching policy_term_end, excluding cancelled_midterm and terms where no renewal offer was made, and cross each tier's retention with its prior pure premium.
4. Pull prior-period reserve development for the periods used and state whether the improvement survives it.

Expected result: A decomposition splitting the two-point improvement into price, mix and within-tier pure premium components that reconcile to the total, alongside retention by rating_tier and an explicit statement of whether the within-tier component is distinguishable from zero.

Verify it yourself

• The three components reconcile to the observed change up to the interaction term. Decide in advance where the interaction is allocated and say so on the chart.
• Pure premium contains no price term by construction, so a step change in it exactly at the rate effective date points to an exposure or earning-pattern error rather than to a risk effect.
• Recent accident periods have no value at the fixed development age, which means unavailable rather than favourable, so any period younger than that age is excluded from the comparison.
• Retention measured at tier level reconciles to the blended figure once terms with no renewal offer are excluded from both.

Follow-up

• Retention is flat overall but fell nine points in the lowest-loss tier. What do you expect next year's pure premium to do?
• Why not use written premium as the denominator, and where would you still legitimately see it used?
• How would you separate a genuine underwriting improvement from a year of mild weather?

Settled volume jumps after a new acceptance corridor launches

Medium · minor units · currency conversion · reconciliation

Settled payment volume per active customer rose 12 percent in the month a new acceptance corridor went live. The pipeline sums settlement_amount_minor from fct_payment_authorization and divides by 100 to get major units. Columns available: amount_minor, transaction_currency, captured_amount_minor, settlement_amount_minor, settlement_currency, settlement_fx_rate, settled_at, is_reversal, issuer_country, acquirer_country. Confirm or refute the 12 percent, and specify exactly the conversion the metric should be using.

Approach

1. Split the month-over-month increase by settlement_currency. If one currency carries nearly all of it while its transaction count barely moved, the problem is arithmetic rather than demand, and that is a two-minute check.
2. Look up each currency's ISO 4217 exponent. Dividing a three-decimal currency by 100 instead of 1000 overstates it tenfold; dividing a zero-decimal currency by 100 understates it hundredfold. Neither is a rounding issue.
3. Rewrite the conversion to scale settlement_amount_minor by ten to the power of that currency's exponent, then convert to the reporting currency. Do not reuse settlement_fx_rate for this; it is the rate applied between transaction and settlement currency at settlement time, not a reporting-currency rate.
4. Check the other half of the series too. settlement_amount_minor is denominated in settlement_currency while amount_minor is in transaction_currency, so any series that mixes the two is uninterpretable regardless of the exponent fix.
5. Exclude reversals and refunds, recompute, then reconcile the corrected total to the settlement ledger for the same period and show the tie-out in the write-up.
6. Recompute the per-customer denominator on the corrected month, since a new corridor adds customers as well as volume and the ratio can move either way once the numerator is right.

Follow-up

• Where else in the warehouse does a hardcoded divide-by-100 appear, and how would you find every instance?
• What test would fail the build the next time a currency with a different exponent is onboarded?
• The reporting-currency rate is itself a choice. Which rate, on which date, and why does finance care about the answer?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

• Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
• Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
• State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

• Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
• Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
• Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

• Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
• Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
• Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

• Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
• Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
• Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

• Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
• Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
• List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

• Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
• Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
• Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

• Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
• Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
• Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Getting an analysis acted on

Work that nobody used is a common and unflattering pattern in data careers, and interviewers probe for it. Have a story about an analysis that changed a decision, and be specific about how you got it in front of the person who could act. Also have one about work that went nowhere, with your reading of why.

Tell me about a time you proactively improved a technical process to make it faster or more efficient.

Medium · behavioural and stakeholder questions

Approach

1. Name the disagreement or constraint, and how you resolved it with evidence.
2. Close with what you would do differently, concretely.
3. Pick a story where you drove the decision, not one where you observed it.

Follow-up

• How did you know the outcome was caused by your change?
• What did you decide not to do, and why?

Turn a one-line fraud-number request into a scoped brief

Easy · scoping · metric definition · denominators

A stakeholder messages: what is our fraud rate, and is it going up? You have fct_payment_authorization, fct_card_dispute and dim_customer. At least four defensible answers exist: count-weighted or value-weighted, attributed to the transaction month or to the dispute filing month, and gross or net of recoveries and successful representments. You get one reply before someone else produces an uncaveated number. Write that reply: the clarifying questions you ask, the single default you will produce if nobody answers, and what the default excludes.

Approach

1. Establish the decision behind the question first, because a risk-rule change, a board number and a merchant contract negotiation need different denominators, and asking which one is not stalling.
2. Offer a short menu rather than an open question: a stakeholder can choose between two named options but cannot specify a denominator from scratch.
3. Commit to a default so the reply is useful even if nobody answers, for example net fraud loss in basis points of settled volume, attributed to the requested_at month, matured months only.
4. State the exclusions in the same breath as the default: non-fraud dispute categories, transaction months with less than 120 days of maturity, and first-party abuse that arrives coded as consumer_dispute.
5. Give a delivery time for the default and a longer one for the fuller cut, so the choice between them carries a visible cost.

Follow-up

• They come back wanting it by merchant for a contract negotiation. What changes in the definition and in the maturity rule?
• How would you separate first-party abuse from third-party fraud in this data, and what would you refuse to conclude from the split?

Retract a published number after finding a currency bug

Medium · error disclosure · minor units · process repair

Two weeks ago you published an interchange and fraud analysis that summed amount_minor across fct_payment_authorization without converting currencies. Minor units are not two decimals everywhere: some currencies carry none and some carry three, so the sum has no interpretation. A pricing decision is already in flight on the back of it. You now have corrected figures. Produce the retraction: what you send, to whom, in what order, and what you change in the process so this class of error is caught next time rather than trusted next time.

Approach

1. Size the error before announcing it, because saying the number is wrong without a magnitude and a direction forces every reader to assume the worst case.
2. Check whether the conclusion actually flips: if the ranking that drove the pricing decision is unchanged, that belongs in the first sentence beside the correction rather than buried at the end.
3. Tell the person acting on it first and directly, then the wider distribution, using the same text, so nobody learns about it secondhand.
4. Write the correction as four parts: the old number, the cause in one clause, the effect on the pending decision, and the new number. Leave out self-flagellation, which makes the reader do emotional work instead of acting.
5. Fix the class rather than the instance: a rule that a sum over amount_minor either groups by transaction_currency or passes through both conversion steps, exponent scaling and then a dated rate into one named reporting currency, plus a standing reconciliation of the settled subset to the settlement ledger inside each settlement_currency.

Follow-up

• The corrected figures do not change the decision. Do you still send the correction, and what does that choice signal?
• What automated check would have caught this, where would it live, and what would it cost in false alarms?