cswork / libraryOpen practice workspace

Interview guides · Altair Engineering

Altair Engineering Data Scientist Interview Questions & Guide 2026

Case practice for Altair Engineering data scientist candidates: scope a vague prompt into a measurable question before you analyze it.

The 60-second brief

At Altair Engineering, we believe in transforming the future through the convergence of simulation, high-performance computing (HPC), and data analytics. While the broader Data Scientist title often implies purely digital or algorithmic work, this specific role operates at the cutting edge of physical science and data as a Fuel Cell Research Scientist. You will be working on site with our client’s Research Institute of North America (TRINA) in Ann Arbor, Michigan, supporting the research and development of next-generation fuel cell technologies.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

  • Reconcile amounts in minor units and currency
  • Report only matured cohorts for loss metrics
  • Separate authorization, settlement and dispute outcomes cleanly

Overview

This position is critical because it bridges the gap between theoretical materials science and practical, data-driven engineering. Your work directly impacts the viability, durability, and efficiency of clean energy solutions like low-temperature fuel cells and PEM water electrolyzers. By synthesizing electrocatalysts and rigorously testing Membrane Electrode Assemblies (MEAs), you provide the foundational data that drives broader simulation and product development.

Expect a highly collaborative, multidisciplinary environment where your deep domain expertise in chemistry or materials science intersects with rigorous data evaluation. This is not a standard desk-bound analytics role; it is a hands-on, highly specialized position where your physical experiments generate the data that powers tomorrow’s sustainable innovations.

The Altair Engineering interview process

Altair Engineering candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

  • Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
  • Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
  • The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

  • Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
  • Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
  • Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Technical Deep-Dive

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

  • Whether your row counts survive each join, and whether you notice on your own when they do not
  • Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
  • Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
  • Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

  • Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
  • Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
  • Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Behavioral Interviews

Behavioural answers from data candidates get audited in a way that answers from other roles do not. When you say a model lifted retention, the next question is the denominator, the window, and how you knew the lift was not seasonal. So attach the measurement to each claim while you tell it: what the metric was before, over what period, and against what comparison. Numbers with no baseline read as rounded-up memory, and one unsupported figure tends to make the rest of the story sound rehearsed.

What to demonstrate

  • Whether each impact number arrives with a baseline, a window and a comparison, or as a bare percentage
  • Whether you can name the method that attributed the effect to your work (an experiment, a staged rollout, a seasonal control) or concede the link was correlational
  • Whether the magnitudes stay internally consistent when the interviewer multiplies them against the scale you described earlier

How to prepare

  • For each story, write the impact line as metric, value before, value after, window, and how attribution was established. Any line missing two of those five is a follow-up you will answer badly.
  • Re-derive one headline number from the source table rather than the deck that reported it. Resume numbers drift upward across retellings.
  • Decide in advance which figures you cannot share, and prepare the ratio or relative change you can give instead, so a confidentiality limit does not read as evasion.

Final Decision

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

  • Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
  • Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
  • Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

  • Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
  • Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
  • Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
  • Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Using written premium as the denominator of a loss ratio

Premium is written at inception and earned pro rata across the exposure period, so in a growing book written premium runs ahead of earned premium and the loss ratio comes out too low, with the error reversing when the book shrinks. The numerator has the mirror-image problem if it omits incurred-but-not-reported reserves, since recent accident periods then look profitable twice over. Both sides must refer to the same exposure period, which is what an accident-period view at a fixed development age enforces.

Reading the most recent months of fraud and dispute rates as final

Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Questions & practice

How do you control the nanoparticle size and distribution during the synthesis of PGM ORR catalysts?

Medium · statistics and probability

Approach

  1. Sanity-check the answer against a simple bound or a simulated case.
  2. Translate the result into the decision it informs, in one plain sentence.
  3. Quantify uncertainty explicitly rather than reporting a point estimate alone.

Follow-up

  • How would you explain this result to someone who does not know statistics?
  • Which assumption here is most likely to be violated in practice?

Estimate a delinquency roll-rate matrix and project twelve months

Hard · roll rates · markov chain · survivorship

fct_loan_performance_monthly gives loan_id, as_of_month_end, months_on_book, delinquency_bucket, charge_off_flag, prepaid_in_full_flag and restructured_flag. Build a month-to-month transition matrix over the five delinquency buckets plus absorbing charged_off and prepaid states. Loans that stop appearing must be routed to an absorbing state rather than dropped. Project the current book forward 12 months by repeated matrix multiplication and report the projected share reaching charge-off. Handle restructured_flag explicitly, and name one place the Markov assumption fails on this data.

Approach

  1. Build consecutive month pairs per loan by shifting as_of_month_end within loan_id, then verify the shifted value is exactly one month later. A gap is not a transition, it is an exit you have not resolved yet.
  2. Resolve exits before counting anything. A loan whose last row carries charge_off_flag moves to charged_off, one carrying prepaid_in_full_flag moves to prepaid, and one that disappears with neither is a data question to raise rather than silently discard, because discarding it is survivorship that inflates every cure rate.
  3. Count pairs into a 7 by 7 matrix and row-normalise. Assert every row sums to one and the two absorbing rows are the identity; a row that does not sum to one means exits were dropped.
  4. Decide and state the restructure rule. Restructuring resets days_past_due, so a dpd_60_89 to current move on a restructured loan is not a cure. Either give restructured loans their own state or carry the pre-restructure bucket, but do not let that move land in the cure cell.
  5. Project by taking the current bucket distribution as a row vector and multiplying by the matrix twelve times. Report the charged_off entry, and report it again from an all-current starting vector so the reader can see how much of the projection comes from loans that are already delinquent today.
  6. State the homogeneity failure plainly: transition rates depend strongly on months_on_book, so one pooled matrix applied to a book with a young mix understates early-life delinquency. If the mix is moving, estimate separate matrices by seasoning band.

Worked solution (45 min)

  1. Sort by loan_id and as_of_month_end, shift to form (from_state, to_state) pairs, and flag pairs whose month gap is not exactly one.
  2. For each loan's final row, assign the absorbing destination from charge_off_flag or prepaid_in_full_flag, and list loans that vanish with neither as an exception count to report.
  3. Apply the restructure rule, then build the 7 by 7 count matrix with a cross-tabulation over ordered state categories and row-normalise it.
  4. Assert row sums equal one and absorbing rows are the identity, then take the current month's bucket distribution as a row vector.
  5. Multiply twelve times, report the charged_off component, and repeat from an all-current vector for comparison.

Expected result: A 7 by 7 row-stochastic matrix with identity rows for charged_off and prepaid, a current-to-current diagonal typically above 0.95, roll rates rising with bucket depth, and a 12-month projected charge-off share of a few percentage points from an all-current start and materially higher from the actual book.

Verify it yourself

  • Every row sums to one within floating-point tolerance, and both absorbing rows are exactly the identity.
  • No mass sits in cells that skip a bucket upward, such as dpd_1_29 to dpd_60_89 in one month; if there is, the pairs are not actually consecutive months.
  • The projection from an all-current start is strictly below the projection from the real book distribution.
  • Backtest by estimating on months up to t minus 12 and comparing the projected distribution with the realised one at t; a systematic optimistic gap points at dropped exits or at seasoning, not at noise.

Follow-up

  • How would you validate the projection against what actually happened, and over what window?
  • The cure rate out of dpd_30_59 rose five points last quarter. What are the candidate explanations and how would you separate them?
  • When would you prefer a vintage curve to a roll-rate projection, and why?

Simulate false alarms in a merchant chargeback monitoring rule

Medium · simulation · rare events · monitoring thresholds

Baseline matured first-chargeback rate is 12 per 10,000 settled transactions. A monitoring rule alerts when a merchant's observed monthly rate exceeds twice baseline. For monthly settled transaction counts of 500, 2,000, 10,000 and 50,000, simulate the false-alarm probability per merchant-month under the baseline, and the power to detect a merchant whose true rate is 30 per 10,000. Then, for a portfolio of 4,000 merchants split 60, 25, 10 and 5 percent across those four counts, give the expected number of false alarms per month.

Approach

  1. Recognise the rule is a threshold on an integer count, not on a continuous rate. At n = 500, twice baseline is 24 per 10,000, so the first observable value above it is 2 chargebacks, or 40 per 10,000. Derive the trigger count for every n before simulating anything.
  2. Draw binomial counts with numpy at p = 0.0012 and take the share at or above the trigger for the false-alarm rate, then repeat at p = 0.0030 for power. Use at least 200,000 draws per cell so a probability near 0.001 has a usable standard error.
  3. Cross-check every simulated cell against the Poisson approximation with lambda = n*p, which is tight here because p is tiny. A mismatch almost always means the trigger count is off by one.
  4. Weight the per-merchant false-alarm probabilities by the portfolio mix, and report the share of expected alerts contributed by each size band rather than only the total.
  5. Close on the operating consequence: a fixed multiplicative threshold is not a constant false-alarm rate across merchant sizes, so either the threshold scales with n or small merchants need a minimum volume before the rule applies.

Follow-up

  • How would you set a threshold that holds the false-alarm rate roughly constant across merchant size?
  • The rule reads the transaction month, but disputes arrive for up to 120 days afterwards. What does that do to the alert and how would you fix it?
  • What does a month of these false alarms cost, and how would you decide whether it is worth paying?

Attribute chargebacks to transaction month without fanning out

Medium · fan-out · cohort attribution · maturity

Report the matured first-chargeback rate. Numerator: fct_card_dispute rows whose dispute_stage is first_chargeback, representment, pre_arbitration or arbitration, attributed to the requested_at month of the linked fct_payment_authorization row rather than to opened_at. Denominator: settled authorizations in that same transaction month, meaning settled_at is not null and is_reversal is false. One authorization can carry more than one dispute case. Report only transaction months whose last day is at least 120 days old, and mark the remainder incomplete rather than showing them as low.

Approach

  1. Compute the denominator from fct_payment_authorization alone, grouped by the requested_at month, before any dispute table is mentioned, so the join cannot touch it.
  2. Compute the numerator as a separate aggregate: join disputes to their authorization only to recover requested_at, then count dispute rows per transaction month with the stage filter.
  3. Join the two aggregates on the month key, which keeps the grain at one row per month and makes double counting structurally impossible rather than something you have to remember.
  4. Apply the maturity gate on the month's last day plus 120 days against the current date, because a transaction on the final day of the month is the least mature one in the cohort.
  5. Emit an explicit status column of 'final' or 'incomplete' instead of filtering immature months away silently, so a reader cannot mistake absence for zero.

Worked solution (30 min)

  1. CTE denom: SELECT date_trunc('month', requested_at) AS txn_month, COUNT(*) AS settled_txns FROM fct_payment_authorization WHERE settled_at IS NOT NULL AND NOT is_reversal GROUP BY 1.
  2. CTE numer: join fct_card_dispute to fct_payment_authorization on auth_id, filter dispute_stage to the four stages at or beyond first_chargeback, group by the same txn_month expression, count dispute rows.
  3. LEFT JOIN numer onto denom on txn_month so months with zero disputes appear as zero rather than disappearing.
  4. Derive the rate with a numeric cast and a status column driven by (txn_month + interval '1 month - 1 day') + interval '120 days' <= current_date.

Expected result: One row per transaction month with settled_txns, chargeback_cases, a rate in basis points, and a status of 'final' or 'incomplete'. The final months form a roughly flat series; the incomplete months at the right edge are visibly and correctly labelled rather than sloping to zero.

Verify it yourself

  • settled_txns for a given month is identical whether or not the dispute CTE is joined, which is the direct test that no fan-out occurred.
  • COUNT(DISTINCT auth_id) in the numerator is at or below the dispute row count; if they are equal, no authorization in the data carries two cases and the fan-out risk is latent rather than absent.
  • Every month flagged 'final' has its last day at least 120 days before the current date.
  • Attributing by opened_at instead shifts the series later and flattens the right edge, which demonstrates why the attribution key matters.

Follow-up

  • Should the numerator count dispute cases or disputed authorizations? Which one reconciles to the loss line, and which one to the operations queue?
  • Some reason codes allow filing well beyond 120 days. How would you estimate the tail on a month you have decided to call final?
  • How would you present the incomplete months so that a weekly reader does not read the right-hand slope as an improvement?

Monthly delinquency roll rates with honest exiting-account denominators

Hard · roll rates · window functions · self joins

From fct_loan_performance_monthly, compute month-over-month roll rates: for each as_of_month_end and delinquency_bucket, the share of loans in that bucket that are in a worse state at the following month end. Use this disposition convention and apply it without exception: charge-off is the terminal state one rank past dpd_90_plus, so a loan that charges off has rolled forward, not exited; loans that leave the book any other way — prepaid in full, sold, matured, or simply absent at the next month end — are exited. Bucket order is current, dpd_1_29, dpd_30_59, dpd_60_89, dpd_90_plus, then charged_off. Return from_month, from_bucket, accounts, rolled_forward, cured, stayed, exited and roll_rate, and show that the four dispositions sum to accounts on every row.

Approach

  1. Map the five delinquency_bucket values to ranks 0 through 4 in a small inline VALUES list, and reserve rank 5 for charged_off, because 'worse' is an ordering statement and enum text does not sort that way reliably. Read charge-off from charge_off_flag on the next month's row rather than from delinquency_bucket, which keeps carrying whatever delinquency value the loan held when it was written off.
  2. Use LEAD(as_of_month_end), LEAD(bucket_rank) and LEAD(charge_off_flag) OVER (PARTITION BY loan_id ORDER BY as_of_month_end), then verify the leading month end is exactly one month after the current one, since a loan missing a month would otherwise appear to cure or roll across a gap it never traversed.
  3. Classify each loan-month into exactly one of four labels off the adjacent next row, whose rank is 5 when its charge_off_flag is true and its mapped bucket rank otherwise. rolled_forward: the next rank is strictly greater. cured: strictly lower. stayed: equal. exited: there is no adjacent next row at all, which is where prepayment, sale, maturity and a gap in the monthly history land.
  4. Define rolled_forward as 'any strictly worse rank', not 'rank plus one'. Under normal monthly aging the only reachable worse state is the next bucket, so the two definitions coincide — except at charge-off, where a loan can go from dpd_60_89 straight to rank 5. A strict plus-one test leaves those loan-months matching no branch and breaks the exhaustiveness invariant.
  5. Be explicit about why charge-off is a roll rather than an exit: routed to exited, dpd_90_plus has no worse state left, so its roll_rate is identically zero by construction — arithmetic, not a credit finding. At rank 5 the dpd_90_plus row reports roll-to-loss, which is the number a loss forecast actually consumes.
  6. Aggregate by from_month and from_bucket and require that rolled_forward + cured + stayed + exited equals accounts on every row, which is the invariant that proves the classification is exhaustive and mutually exclusive.

Follow-up

  • Project the next three months of dpd_90_plus inflow from these roll rates. What assumption does that projection make, and when does it break?
  • A restructure resets days_past_due to zero. What does that do to the cure rate out of dpd_60_89, and how would you separate a real cure from a reset?
  • Argue the other convention: prepayment in exited but charge-off also in exited, with roll rates reported only for the four non-terminal buckets. What does that series answer better, and what does it lose?
  • The charge-off policy changed from 180 to 120 days. Show where that appears in this table and what it does to the dpd_90_plus roll rate specifically.

How do you account for uncompensated resistance (iR drop) during your half-cell RDE measurements?

Medium · a/b testing and experimentation

Approach

  1. Name the guardrails that would stop a launch even on a positive primary result.
  2. State the primary metric and the minimum effect worth shipping, then size the test.
  3. Say whether units interfere with each other, and switch design if they do.

Follow-up

  • What would you do if you could not randomise at all?
  • How would you handle interference between treated and control units?

If a full-cell test fails prematurely, what is your diagnostic process to determine if the failure was due to the catalyst, the membrane, or the assembly process?

Medium · a/b testing and experimentation

Approach

  1. Say whether units interfere with each other, and switch design if they do.
  2. State the primary metric and the minimum effect worth shipping, then size the test.
  3. Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • What would you do if you could not randomise at all?

Describe the specific Accelerated Stress Test (AST) protocols you use to evaluate carbon support corrosion.

Medium · a/b testing and experimentation

Approach

  1. Name the randomisation unit first; it decides the variance and what the test can detect.
  2. Decide the analysis before seeing data, including how long it runs and when you look.
  3. Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • How would you handle interference between treated and control units?

Walk me through your step-by-step process for preparing an ink and fabricating an MEA.

Medium · a/b testing and experimentation

Approach

  1. Name the randomisation unit first; it decides the variance and what the test can detect.
  2. State the primary metric and the minimum effect worth shipping, then size the test.
  3. Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

  • What would you conclude if the result is positive but the test is underpowered?
  • How would you handle interference between treated and control units?

What characterization techniques do you rely on most to verify the structure of a newly synthesized catalyst?

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Say what you would check first and why it is the highest-information step.
  3. Clarify what is being asked and what a complete answer would contain.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Test a referral rule when reviewers are a shared queue

Medium · interference · switchback · cluster randomisation

A proposed risk rule raises the share of ecommerce authorizations routed to manual review. Reviewers work a shared pool of queues serving both arms, so extra referrals from treatment lengthen the wait for control cases too. The current plan randomises by customer_id and reads decision latency plus matured fraud basis points. Explain why that design is biased and in which direction, propose a design that is not, and say what governs its power.

Approach

  1. Name the violated assumption precisely: a unit's outcome depends on other units' assignments through the shared reviewer capacity, so the stable unit treatment value assumption fails. Customer-level randomisation then estimates a contrast between a degraded treatment and a degraded control, not between treatment and the status quo.
  2. State the direction. Treatment pushes work into the shared queue, control absorbs part of that wait, so the measured latency difference understates the true effect of full rollout. The test can look acceptable while the rolled-out state is materially worse, which is the expensive failure mode here.
  3. Move randomisation up to a unit that contains the interference. Either cluster-randomise whole queues or sites, or run a switchback that flips the rule for an entire queue over time blocks. Switchback is usually the better choice here because queue count is small and each queue serves as its own control, removing between-queue heterogeneity.
  4. Specify the switchback concretely. Block length should be several times the queue sojourn time; discard a burn-in after each switch equal to the 95th percentile sojourn so carryover cases from the previous regime do not contaminate the new block; balance assignment within each day so the daily arrival pattern cannot correlate with arm.
  5. Analyse at the randomisation unit. Aggregate to queue-block means, include queue and time-block fixed effects, and use randomisation inference or a wild cluster bootstrap rather than cluster-robust standard errors, which are anti-conservative with few clusters. Check residual autocorrelation between adjacent blocks; if it is material, widen the blocks or model it.
  6. Separate the two readouts by maturity. Latency and referral precision at case close are available inside the test window; matured fraud basis points require at least 120 days of dispute maturity from the transaction month, so it is a deferred confirmatory read and must not be presented as a low number on immature cohorts.

Worked solution (30 min)

  1. Write the interference channel explicitly: reviewer capacity is fixed per queue per hour, arrivals are the sum of both arms, so control's wait is a function of treatment's assignment.
  2. Choose a queue-by-four-hour-block switchback across 14 queues for 30 days, giving 14 * 6 * 30 = 2,520 blocks, with within-day balanced assignment and a burn-in equal to the 95th percentile sojourn discarded at each switch.
  3. Compute power at the block level. With a residual block-level standard deviation of decision latency of 6 minutes after removing queue and day-of-week fixed effects, and half the blocks per arm, MDE = 2.8016 * sqrt(2 * 36 / 1260).
  4. State the caveat that adjacent blocks are autocorrelated, so the effective block count is below 2,520 and the analytic MDE is a floor; estimate the inflation from historical block-to-block autocorrelation before committing.
  5. Split the readout: latency and referral precision as the in-window primary, matured fraud basis points as a deferred read at 120 days with immature months marked incomplete rather than plotted.

Expected result: A queue-by-time-block switchback with burn-in exclusion and within-day balanced assignment, analysed on block means with queue and time fixed effects and randomisation inference. With 2,520 blocks and a 6-minute residual block standard deviation, the analytic MDE on decision latency is 2.8016 * sqrt(72 / 1260) = 2.8016 * 0.2390 = 0.67 minutes, stated as a floor pending the autocorrelation correction. Power is governed by block count and between-block residual variance, not by transaction volume.

Verify it yourself

  • An A/A switchback on historical data should produce block-level differences centred on zero with a standard deviation matching the residual block standard deviation used in the power calculation.
  • Plot the effect against time since the last switch. If it is not flat after the burn-in window, the burn-in is too short and carryover is still present.
  • Confirm that arm assignment is orthogonal to hour of day and day of week in the realised schedule, since queue load is strongly periodic.
  • Cluster count, not row count, should appear in the degrees of freedom of the final inference; if the standard error implies thousands of independent observations, the analysis is at the wrong grain.

Follow-up

  • How do you set the block length when the queue sojourn time itself changes under treatment?
  • You have 14 queues and 30 days. Compare a queue-level cluster design against a queue-day switchback on power and on what each can estimate.
  • The shared resource is capacity. What happens to your estimate if reviewers work faster when the queue is long?

Settled volume jumps after a new acceptance corridor launches

Medium · minor units · currency conversion · reconciliation

Settled payment volume per active customer rose 12 percent in the month a new acceptance corridor went live. The pipeline sums settlement_amount_minor from fct_payment_authorization and divides by 100 to get major units. Columns available: amount_minor, transaction_currency, captured_amount_minor, settlement_amount_minor, settlement_currency, settlement_fx_rate, settled_at, is_reversal, issuer_country, acquirer_country. Confirm or refute the 12 percent, and specify exactly the conversion the metric should be using.

Approach

  1. Split the month-over-month increase by settlement_currency. If one currency carries nearly all of it while its transaction count barely moved, the problem is arithmetic rather than demand, and that is a two-minute check.
  2. Look up each currency's ISO 4217 exponent. Dividing a three-decimal currency by 100 instead of 1000 overstates it tenfold; dividing a zero-decimal currency by 100 understates it hundredfold. Neither is a rounding issue.
  3. Rewrite the conversion to scale settlement_amount_minor by ten to the power of that currency's exponent, then convert to the reporting currency. Do not reuse settlement_fx_rate for this; it is the rate applied between transaction and settlement currency at settlement time, not a reporting-currency rate.
  4. Check the other half of the series too. settlement_amount_minor is denominated in settlement_currency while amount_minor is in transaction_currency, so any series that mixes the two is uninterpretable regardless of the exponent fix.
  5. Exclude reversals and refunds, recompute, then reconcile the corrected total to the settlement ledger for the same period and show the tie-out in the write-up.
  6. Recompute the per-customer denominator on the corrected month, since a new corridor adds customers as well as volume and the ratio can move either way once the numerator is right.

Follow-up

  • Where else in the warehouse does a hardcoded divide-by-100 appear, and how would you find every instance?
  • What test would fail the build the next time a currency with a different exponent is onboarded?
  • The reporting-currency rate is itself a choice. Which rate, on which date, and why does finance care about the answer?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

  • For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
  • For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
  • Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

  • Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
  • List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
  • Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

  • Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
  • Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
  • Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

  • Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
  • Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
  • Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

  • Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
  • Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
  • Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

  • Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
  • For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
  • Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

  • Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
  • Listen back and mark every moment you proposed a solution before the success metric existed.
  • Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

Describe a situation where you had to collaborate with a multidisciplinary team (e.g., mechanical engineers or data analysts) to solve a complex problem.

Medium · behavioural and stakeholder questions

Approach

  1. Name the disagreement or constraint, and how you resolved it with evidence.
  2. State the situation in two sentences and spend the rest on your reasoning.
  3. Close with what you would do differently, concretely.

Follow-up

  • How did you know the outcome was caused by your change?
  • What did you decide not to do, and why?

Tell me about a time when your research hypothesis was proven wrong by your data. How did you pivot?

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. Pick a story where you drove the decision, not one where you observed it.
  3. Quantify the outcome, including what you would not claim credit for.

Follow-up

  • What would you do differently if you ran that project again?
  • How did you know the outcome was caused by your change?

Recommend a decision whose true outcome matures a year later

Hard · decision under censoring · leading indicators · staged rollout

An underwriting rule change must be decided in six weeks. Its real outcome, the vintage 90-plus rate at months_on_book 12 in fct_loan_performance_monthly, matures in a year. The executive wants a yes or no, not a range. Randomising the credit decision across the whole population is not available. Name the leading indicator you would accept, state its bias and the direction of that bias, define the decision rule and stopping condition before any rollout starts, and say what reading would make you recommend reversing the change.

Approach

  1. Fix the readout before the rollout, because a readout chosen after the data arrives is a story rather than a decision rule: indicator, window, threshold and reversal condition all go in writing first.
  2. Choose the leading indicator on its measured relationship to the matured outcome in historical vintages rather than on availability. Early delinquency, typically the share reaching dpd_1_29 or missing a first scheduled payment by months_on_book 3, is the usual candidate, and you quantify how well it predicted the 12-month rate across past cohorts.
  3. State the bias and its direction plainly: early delinquency under-represents default that emerges later and is contaminated by servicing and payment-date effects, so treat it as a floor on risk rather than an estimate of it.
  4. Buy identification where full randomisation is unavailable: a narrow randomised approval band around the cutoff, or a staged rollout by channel or region read as a difference-in-differences, with the parallel-trends assumption stated and checked in the pre-period rather than assumed.
  5. Give the executive the binary they asked for with the trigger attached in the same sentence: yes, conditional on the month-3 indicator staying inside a stated band, with an automatic hold if it breaches.

Follow-up

  • How would you validate that the month-3 indicator predicts the 12-month outcome, and what evidence would invalidate it mid-rollout?
  • Compliance refuses a randomised band. What is your next-best identification strategy, and what precision do you lose by taking it?