Interview guides · Belvedere Trading

Belvedere Trading Data Scientist Interview Questions & Guide 2026

Belvedere Trading Data Scientist interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.

The 60-second brief

At Belvedere Trading, the Data Scientist (often aligned with the Quantitative Researcher title) serves as a critical bridge between raw market data and actionable trading strategies. You are responsible for transforming complex, high-frequency datasets into competitive advantages by identifying alpha, modeling market behavior, and refining execution logic. This role is not merely about running models; it is about deeply understanding the mechanics of the markets in which Belvedere Trading operates.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

Attach uncertainty to every Sharpe claim
Separate forecast decay from execution cost
Build point-in-time panels without lookahead

Overview

Your impact is direct and measurable. By developing and testing new signals, you influence the firm’s bottom line and operational efficiency. You will work in a high-stakes environment where precision, mathematical rigor, and the ability to articulate your logic clearly are paramount. Successful candidates are those who combine technical depth in statistics and probability with a pragmatic, trader-like mindset focused on risk-adjusted returns.

Where to focus your preparation

Treat the sections below as preparation areas and confirm the format with your recruiter.

Preparation focus

No round sequence has been reported for this company, so work the categories below and confirm the format with your recruiter.

What to demonstrate

Breadth across SQL, experimentation and product reasoning
Ability to state assumptions before choosing a method

How to prepare

Drill the practice exercises below and time yourself
Prepare three quantified stories about decisions you drove

Where candidates lose points

Judging execution quality against interval VWAP and treating a favourable number as proof of good trading.

Interval VWAP is a benchmark the trader partly determines: trading in line with volume tracks VWAP almost by construction, and stretching an order over a longer interval makes the benchmark easier while exposing the position to price drift that the benchmark never charges. Arrival price is the benchmark aligned with the decision, because it charges both the spread and the drift between decision and completion, including the unfilled remainder. Reporting both, and reporting the opportunity cost of unfilled quantity, is what separates a real TCA from a flattering one.

Reporting the best backtest out of many trials as if it were a single pre-registered test.

The maximum of N noisy Sharpe estimates grows roughly like the standard error times sqrt(2 ln N) even when every underlying strategy has zero edge, so with a few hundred variants an in-sample Sharpe near 1 is the expected result of pure noise. Worse, the search is rarely counted honestly: parameter sweeps, universe changes, date-range choices and feature variants all count as trials. Quote the number of configurations tried, deflate the Sharpe for it, and keep a genuinely untouched holdout period. Note also that the asymptotic standard error of a Sharpe estimate is approximately sqrt((1 + SR^2/2)/T) for i.i.d. normal returns, which for three years of daily data is roughly 0.33, so two strategies differing by 0.3 in Sharpe are not distinguishable.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Averaging per-user rates to produce a population rate

Decide which quantity you want: the mean of per-user ratios and the ratio of summed numerator to summed denominator are different estimands, and heavy users dominate one but not the other. For a ratio metric, aggregate numerator and denominator separately and use the delta method for its variance.

Questions & practice

Solve a classic probability brain-teaser involving conditional expectations.

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Write down the assumption the method needs before you use the method.
Translate the result into the decision it informs, in one plain sentence.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

How would you calculate the probability of a specific outcome in a game involving multiple independent variables?

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Translate the result into the decision it informs, in one plain sentence.
Quantify uncertainty explicitly rather than reporting a point estimate alone.

Follow-up

Which assumption here is most likely to be violated in practice?
How would you explain this result to someone who does not know statistics?

Explain the concept of Expected Value (EV) in the context of a trading decision.

Medium · statistics and probability

Approach

Sanity-check the answer against a simple bound or a simulated case.
Translate the result into the decision it informs, in one plain sentence.
Write down the assumption the method needs before you use the method.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

Given a fair coin, what is the expected number of flips to get two heads in a row?

Medium · statistics and probability

Approach

Quantify uncertainty explicitly rather than reporting a point estimate alone.
Say what the estimate is of, and over what population it generalises.
Write down the assumption the method needs before you use the method.

Follow-up

How would you explain this result to someone who does not know statistics?
Which assumption here is most likely to be violated in practice?

Build a point-in-time as-of join without merge_asof

Medium · pandas · point-in-time · vectorization

Two frames. sig: instrument_id (int64), knowledge_ts (tz-aware UTC, irregular, several rows per instrument per day), zscore_xs, about 7.5M rows across 5,000 instruments. bars: instrument_id, bar_close_ts (tz-aware UTC, one row per instrument per trading day), fwd_ret_5d, about 7.5M rows. For each bar row attach the most recent zscore_xs whose knowledge_ts is at or before that instrument's bar_close_ts, or NaN when the newest such score is more than 10 calendar days old. You may not use pd.merge_asof.

Approach

Concatenate both frames with a source flag, sort by (instrument_id, ts, is_bar) so signal rows precede bar rows at an identical timestamp, then groupby('instrument_id')[[...]].ffill() and keep the bar rows. One sort and one forward fill, with no per-group Python.
Carry the signal's knowledge_ts through the forward fill as its own column. It is the only way to apply the 10-day staleness cap afterwards, and it is the audit trail showing which score each bar actually used.
The alternative is two searchsorteds. Factorize instrument_id to dense codes, compress both timestamp arrays to a shared integer rank, form key = code*(n_unique_ts+1) + rank so lexicographic order becomes numeric order, then np.searchsorted(sig_key, bar_key, side='right') - 1. Validate that the found row belongs to the same instrument, or the last row of one instrument is silently attached to the first bar of the next.
Normalize timezones before sorting. A tz-naive column compared against a tz-aware one raises in current pandas and compared wrongly in older ones; convert both with dt.tz_convert('UTC') and assert the dtype rather than trusting the loader.
Apply the cap as (bar_close_ts - carried_knowledge_ts) > Timedelta('10D') and set those to NaN. Report the NaN share by month: a rising share is usually a vendor outage or a delisted name still present in the bar file.

Worked solution (30 min)

Rename sig.knowledge_ts and bars.bar_close_ts to a common 'ts', add is_bar as 0/1, keep the original signal timestamp in its own column, concat with ignore_index=True.
combined = combined.sort_values(['instrument_id','ts','is_bar'], kind='mergesort') so equal keys preserve input order.
combined[['z_ff','kts_ff']] = combined.groupby('instrument_id', sort=False)[['zscore_xs','sig_ts']].ffill()
out = combined[combined.is_bar == 1]; set z_ff to NaN where out.ts - out.kts_ff exceeds Timedelta('10D').
Spot-check 200 randomly chosen bar rows against a brute-force lookup over the raw sig frame.

Expected result: Exactly one output row per input bar row, no duplication, with z_ff populated wherever a score exists within 10 calendar days of that bar. len(out) equals len(bars).

Verify it yourself

len(out) == len(bars), and out[['instrument_id','ts']] matches bars row for row once both are sorted; a changed row count means the concat introduced or dropped keys.
For 200 random bar rows, sig[(sig.instrument_id == i) & (sig.knowledge_ts <= t)].iloc[-1] must equal z_ff.
kts_ff <= ts must hold everywhere; a row where it does not is a sort-order bug, not a data bug.
Raising the cap to 10 years must make the NaN count equal the number of bars with no prior score at all.

Follow-up

The signal file contains rows with is_backfilled = TRUE, written after as_of_date by a rerun. Does your join still produce a point-in-time panel, and what would you change?
How would you extend this to a forward-looking join, the first score strictly after the bar, and where in a research pipeline is that the correct thing to want?
What changes when bar_close_ts and knowledge_ts can be exactly equal because the signal is computed off that same close?

Deduplicate a restated signal table into a leak-free panel

Medium · window-functions · dedup · point-in-time

signal_score may hold several rows per (signal_id, instrument_id, as_of_date) because reruns rewrite history; each row carries model_version, knowledge_ts, computed_at, is_backfilled, zscore_xs, decile_rank, coverage_flag and universe_id. Build the panel a backtest may legitimately use on trading date D: for every instrument in universe_id = 'liquid_us_1500', the single most recent score whose knowledge_ts is at or before D 21:00 UTC, preferring a live row over a backfilled one. Return instrument_id, as_of_date, zscore_xs, decile_rank.

Approach

Filter on knowledge_ts, never on as_of_date. as_of_date says what the score describes; knowledge_ts says the earliest instant every input was observable. Put it in the WHERE so it prunes before the window is evaluated.
Rank within the key: ROW_NUMBER() OVER (PARTITION BY signal_id, instrument_id ORDER BY as_of_date DESC, is_backfilled ASC, knowledge_ts DESC, computed_at DESC) and keep rank = 1. In Postgres FALSE sorts before TRUE, so is_backfilled ASC is the live-row preference, written down rather than assumed.
Prefer ROW_NUMBER to a MAX(computed_at) group-then-rejoin: the rejoin duplicates rows whenever two reruns share a computed_at, which is exactly what a batch job produces.
Decide what coverage_flag IN ('stale','imputed') means for this panel and encode it. Dropping those rows silently changes the universe size day to day, which surfaces later as an unexplained jump in measured IC rather than as a missing-data problem.
Express the cutoff as a parameter and the whole thing as a CTE so the backtest loop reuses one query text per date instead of a hand-edited copy.

Follow-up

Write the query that proves the panel is leak-free. What would a violation look like in measured IC, and roughly how large would you expect the inflation to be?
Produce every trading date in one pass instead of one query per date. What does that cost in plan shape?
Two model_version values are live at once during a migration. How does the tie-break change, and who decides?

Continuous holding spells from a daily position panel

Hard · gaps-and-islands · window-functions · calendar · panel-data

position_daily holds one row per (business_date, account_id, strategy_id, instrument_id) with signed quantity, weight_pct_nav, realized_pnl_base_day and unrealized_pnl_base_day. Rows exist only for business dates, and a flat day may appear as quantity = 0 or not at all. For one account_id, return every continuous holding spell — consecutive business dates with non-zero quantity and unchanged sign — as one row: instrument_id, strategy_id, side, start date, end date, length in trading days, peak absolute weight and total P&L. Weekends and exchange holidays must not split a spell.

Approach

Build a trading-day index before anything else: DENSE_RANK() OVER (ORDER BY business_date) across the distinct business dates in the panel, or against a calendar table if the panel itself can have gaps. Differencing raw dates counts calendar days and breaks every spell at a weekend; differencing the index counts sessions, which is the definition in play.
Restrict to quantity <> 0 and derive SIGN(quantity). A long that flips to short is two spells, so sign belongs in the island's partition key, not in the aggregate — otherwise a reversal is reported as one continuous position with a misleading average.
Apply the islands construction on the index: day_idx - ROW_NUMBER() OVER (PARTITION BY account_id, strategy_id, instrument_id, qty_sign ORDER BY day_idx) is constant within a run of consecutive sessions, so grouping by it collapses each run to one row.
Aggregate per island: MIN and MAX of business_date, COUNT(*) as the session length, MAX(ABS(weight_pct_nav)) for peak size, and SUM(realized_pnl_base_day + unrealized_pnl_base_day) for spell P&L.
Fix the gap policy before writing the predicate and state it: a single flat session that splits a spell into two is a different answer from one that is tolerated. If tolerated, merge adjacent islands separated by one session in a second pass, rather than loosening the non-zero filter, which would also swallow genuine exits.

Worked solution (40 min)

Create the session index CTE with DENSE_RANK over business_date across the account's panel.
Filter to non-zero quantity, add SIGN(quantity), and compute the island key with ROW_NUMBER.
Group by (strategy_id, instrument_id, qty_sign, island key) and aggregate the five output measures.
Check for overlap: self-join spells of one (instrument, sign) and assert no pair has intersecting date ranges.
Reconcile: summed spell lengths per (strategy, instrument) must equal that key's non-zero row count.

Expected result: One row per continuous spell; spells for a given (strategy_id, instrument_id, sign) are disjoint and ordered; lengths are in sessions, not calendar days; a position held across a long weekend produces one spell.

Verify it yourself

Summed spell length per (strategy_id, instrument_id) equals the count of non-zero rows for that key.
No two spells for one (instrument_id, sign) have overlapping [start, end] ranges.
A name known to be held continuously through the sample returns exactly one spell.
Total spell P&L equals the panel sum of realized_pnl_base_day + unrealized_pnl_base_day over non-zero rows.

Follow-up

A corporate action changes instrument_id mid-spell. How do you keep the spell whole, and which table establishes that the two ids are the same economic position?
Some rows inside a spell carry recon_status = 'break_quantity'. Include or exclude them, and what does each choice do to the length distribution?
How would you extend this to report the P&L contribution of the first five sessions of each spell separately from the rest?

Measure crowding in a signal you cannot observe others trading

Hard · proxy-design · crowding · bias · risk-reporting

Research suspects the edge is eroding because other participants hold the same positions. Nobody outside the firm reports their book, so the quantity of interest is unobservable. Build the measurement anyway. Propose two or three proxies from data the firm already has in position_daily (borrow_rate_bps, borrow_status, financing_cost_base_day, market_value_base), execution_fill (fill_px, prevailing_bid_px, prevailing_ask_px, liquidity_flag) and parent_order (arrival_mid_px, adv_20d, filled_qty). State the bias in each, name the single number you would put on a monthly risk report, and say what would retire it.

Approach

Write the estimand down before proposing anything: the notional held by others in positions correlated with ours, or operationally, the extra cost and extra drawdown we incur when those holders unwind at the same time we do. Without a stated estimand there is nothing for a bias to be a bias relative to.
Proxy A, borrow scarcity on the short leg: short-notional-weighted borrow_rate_bps and the share of short notional in borrow_status = 'hard_to_borrow'. Bias: it is confounded with lending supply, which moves with index-fund lending programmes, corporate actions and seasonal recalls, so use the spread against a broad borrow benchmark rather than the level.
Proxy B, impact residual: realized implementation shortfall minus the impact model's prediction at matched participation (filled_qty over adv_20d) and matched volatility, since crowded names should cost more than the curve says. Bias: the residual absorbs every misspecification of the curve, including volatility regime and schedule changes, and it is mechanically correlated with the desk's own growth, so it cannot separate others arriving from us getting bigger.
Proxy C, co-movement: correlation between the strategy's daily long-short return and a generic version of the same factor, public or internally reconstructed. Rising correlation alongside falling own-IC is what commoditisation looks like. Bias: the reference series carries its own survivorship and construction choices, and the correlation also rises for benign reasons when factor volatility rises.
Pick one headline and justify the choice. A composite z-score invites false precision by averaging three different biases into one clean-looking number, so prefer a single directional series such as the benchmarked borrow spread, with the other two printed as context. State the retirement condition: a direct holdings source becomes available, or the proxy moves for six months with no matching move in the consequences it claims to predict, namely cost and drawdown.
Write on the report itself what it does not claim. None of these identifies crowding; each is consistent with several explanations. The output is a prompt to review position sizing, not a measurement of a quantity.

Follow-up

The borrow spread widens 60 basis points with no move in the impact residual or in co-movement. What goes in the report?
How would you falsify the crowding hypothesis using an event already in your data, such as a large single-day factor reversal?
If you had one month of a direct holdings feed, what would you check first to calibrate or discard these proxies?

Fix a testing platform that peeks daily across fourteen variants

Medium · peeking · sequential testing · multiple comparisons

A research platform runs fourteen execution-parameter variants against one control, each judged on implementation shortfall. Portfolio managers read a dashboard that recomputes a two-sided t-test every morning for thirty trading days, and anything crossing p < 0.05 is shipped that day. Over the past year, nine of eleven shipped variants failed to reproduce in a follow-up test. Quantify both error sources, then specify the replacement procedure: the stopping rule, the family definition, and what the dashboard is allowed to display.

Approach

Quantify the peeking cost first, because it is the larger of the two. Repeatedly applying a fixed nominal 0.05 threshold as data accrues drives the probability of ever crossing far above 0.05: on the classical repeated-significance figures for a normal mean it is roughly 0.19 by ten equally spaced looks and about 0.28 by thirty, and with an unbounded horizon and a true null it tends to 1.
Quantify the multiplicity cost. Fourteen independent comparisons each at 0.05 give a 1 - 0.95^14 = 0.51 chance of at least one false positive at a single look. Compounded with daily peeking, a dashboard hit at p < 0.05 carries almost no evidential weight, which is exactly what the nine-of-eleven reproduction failure is reporting.
Choose a stopping rule that is valid at the stopping time actually used. If the looks can be scheduled in advance, a group-sequential design with an O'Brien-Fleming spending function preserves the overall level with very little power loss at the planned maximum sample size. If managers will look whenever they like, use an always-valid procedure such as a mixture sequential probability ratio test or a confidence sequence, and accept the larger sample it demands as the price of unrestricted looking.
Define the family before the wave starts. Fourteen variants against one control in one wave is one family. Control the false discovery rate with Benjamini-Hochberg at q = 0.10 rather than the family-wise rate, because a false positive that later fails to reproduce is cheaper here than a missed improvement. State the precondition: the Benjamini-Hochberg guarantee holds under independence or positive regression dependence, and these variants share order flow and are positively dependent, which normally satisfies it. Under arbitrary dependence the Benjamini-Yekutieli correction divides by the sum of 1/i for i = 1..14, about 3.25.
Make the platform enforce the design rather than document it. Display the sequential boundary and the adjusted q-value instead of a raw p-value, withhold the effect estimate until the boundary is crossed or the horizon is reached, and log every look so the number of looks and the membership of the family are recorded facts rather than recollections.

Worked solution (35 min)

Simulate the current procedure under a true null: fourteen arms, thirty daily looks, ship on any p < 0.05. Record the fraction of waves that ship at least one variant.
Reconcile the simulation against its analytic pieces: 1 - 0.95^14 for multiplicity alone at a single look, and the repeated-significance inflation for one variant across thirty looks.
Re-run under the replacement: an O'Brien-Fleming boundary with six scheduled looks plus Benjamini-Hochberg at q = 0.10. Record the null ship rate and the power against a pre-registered 1.5 bps effect.
Back-test the replacement on the eleven variants shipped over the past year and count how many would have crossed its boundary.
Write the stopping rule, the family definition and the look schedule into the platform configuration, so the next wave cannot deviate without an explicit change.

Expected result: Under the null the current procedure ships at least one variant in roughly 99 percent of waves, since 1 - (1 - 0.28)^14 is about 0.99 under independence, and simulation should confirm it. The replacement holds the null ship rate near the stated q. Expect roughly two to four of the eleven past ships to survive the back-test, which is consistent with the two that reproduced.

Verify it yourself

The single-look, fourteen-arm simulated ship rate must reproduce 1 - 0.95^14 = 0.51.
A single arm simulated across thirty daily looks at nominal 0.05 must reproduce roughly 0.28.
Power of the replacement against the pre-registered 1.5 bps effect at the planned maximum sample size must still be at or above 0.80. If it is not, the boundary changed the test's sensitivity and the sample size has to move with it.

Follow-up

An effect estimated at a sequential stop is biased. In which direction, by roughly how much, and how would you report a corrected one?
A variant that appeared in last month's wave and again in this month's wave: one hypothesis or two, and what does your answer imply about how waves are composed?
Would you rather run fourteen variants for thirty days or four variants for one hundred and five days, given the same total order flow?

Return dispersion widened inside one strategy composite

Medium · cohort analysis · dispersion · client reporting

Trailing-12-month net returns for accounts running one strategy used to sit within about 40 bps of each other. This quarter the cross-sectional standard deviation is 210 bps and the AUM-weighted mandate success rate fell. The investment team insists the model is identical across accounts. You have position_daily (business_date, account_id, strategy_id, instrument_id, market_value_base, weight_pct_nav, is_restricted, fx_rate_to_base) and account_mandate (account_id, funded_date, inception_date, status, target_gross_exposure_pct, max_single_name_weight_pct, base_currency, mgmt_fee_bps, effective_from, effective_to). Explain the dispersion with an ordered checklist and say which part is a problem.

Approach

Rank accounts by trailing-12-month net return and inspect the tail before averaging anything. A standard deviation is a distribution statistic and two accounts out of forty can produce it; the number alone does not say whether the cause is systemic or concentrated.
Cut by months since funded_date and by status. An account in 'ramping' holds cash and sits below target_gross_exposure_pct, so it earns a diluted version of the strategy's return. In a rising market that reads as underperformance and in a falling one as outperformance, and both are mechanical rather than informative.
Compute realized gross exposure per account-day from position_daily, as the sum of absolute market_value_base over NAV, and compare it with target_gross_exposure_pct from the account_mandate version valid on that business_date. Join on the effective_from and effective_to range, not on the current row, because mandate terms change mid-life and the current row would apply today's target to last year's book.
Sort the remaining explanations into expected and not expected. Names excluded by is_restricted, base_currency differences moving through fx_rate_to_base, and mgmt_fee_bps differences on a net-of-fee metric are all expected. A weight breaching max_single_name_weight_pct, or an account holding instruments the others do not, is not.
Report dispersion decomposed by cause with dollars attached. The client question is never why sigma is 210 bps; it is why this account returned less than the one on the last page.

Follow-up

Two accounts funded the same day diverge by 90 bps and both reached target exposure inside 20 trading days. Where do you look next?
The AUM-weighted success rate only counts accounts with 36 months of history. How does that interact with what you just found, and does it flatter the number or penalise it?
How would you present cash drag to a client so it reads as a fact about funding rather than an excuse?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Rigour against a deadline

Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.

How do you handle uncertainty when the underlying distribution of a dataset is unknown?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Writing an impact statement that survives a hostile reading

Hard · self-assessment · causal inference · attribution

Write your own annual impact statement. Your work: a market-impact recalibration the desk adopted in March; a signal you researched that a portfolio manager sized and traded; and a rewrite of the nightly position reconciliation that reduced unreconciled rows in position_daily. Realized implementation shortfall fell from 21 bps to 14 bps after March. Market volatility also fell over the same period. State what you added, in basis points where the attribution supports it, and be explicit about where it does not.

Approach

What is probed: whether you can separate correlation from contribution when the correlation favours you, which is the one place almost everyone's standards slip.
Build a counterfactual for the shortfall claim instead of a before-and-after. Shortfall scales with volatility, so a pre and post comparison across a regime change partly measures the market. Use orders that kept the old routing or the old parameters as a control over the same window, matched on participation bucket (order_qty over adv_20d) and side, and report the difference-in-differences rather than the raw 7 bps.
State the part you cannot claim before anyone asks. The signal was sized by the portfolio manager, so its P and L is a joint product. Claim the research decision itself: what you tested, what you rejected, the number of configurations tried, and the standard error you attached. Volunteering the boundary is what makes the claims inside it credible.
Give the reconciliation work a number that is not basis points. Report unreconciled and break rows in position_daily before and after, plus the downstream consequence: marks that fell back to stale_prior_day, and client reports restated. Inventing a basis-point figure for operational work costs you the basis-point figures that are real.
Write a falsifier next to each claim, naming the evidence that would show you added nothing. A reviewer who watches you name your own weakest claim stops auditing the strong ones.

Follow-up

Your control group is 8 percent of order flow. Is the difference-in-differences credible at that size, and what would you need to make it so?
The signal lost money this year. Does it appear in the statement, and in what form?
What did you get wrong this year, and what did it cost?

Explaining why two accounts in one strategy diverged 210 bps

Easy · client communication · attribution · reconciliation

Two separately managed accounts run the identical strategy. Over the trailing twelve months one returned 8.4 percent net and the other 6.3 percent, a gap of 210 bps. The client who owns the lower one has asked in writing why. You have position_daily, account_mandate (SCD2) and the fill history for both accounts. Produce two things: a reconciliation that accounts for the gap down to a residual you state, and a reply of at most 200 words that a non-specialist can act on, without jargon and without blaming the client.

Approach

What is probed: whether you reconcile to the total before you explain anything. An explanation that does not sum to the observed gap is a guess with numbers attached, and the client's next analyst will find the difference.
Reconcile mechanically and in this order, because each step has a clean source: fee difference from mgmt_fee_bps plus any performance fee crystallised above the high-water mark; average cash weight, as one minus the summed weight_pct_nav; restricted names, as is_restricted days multiplied by what those names returned; single-name cap differences across account_mandate SCD2 versions; and the ramp between funded_date and the date gross exposure first reached 90 percent of target.
Carry a residual and state it plainly. If five components explain 170 of 210 bps, the reply says 40 bps unexplained rather than largely explained by. The residual is usually trade timing across the two accounts, and naming it is cheaper than having it discovered.
Order the reply by what the client can do rather than by size of component: what is structural and will persist, such as the fee schedule and their own restricted list; what was one-off and will not repeat, such as the ramp; and what they can change if they choose to.
Remove every term that would need looking up. Your restricted list kept the account out of three names that contributed 62 bps is actionable. Negative selection effect from compliance constraints is not, and the decision to relax a restriction belongs to the client rather than in a recommendation you push.

Follow-up

The residual is 120 bps rather than 40. What goes in the letter, and what do you do before sending it?
The client asks whether their account was deliberately disadvantaged. How do you answer that specific question?
Does the other client need to be told anything, and who decides?