Interview guides · Trexquant Investment

Trexquant Investment Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Trexquant Investment: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

At Trexquant Investment, a Data Scientist operates at the intersection of quantitative finance, machine learning, and high-performance software engineering. As a technology-driven systematic investment firm, Trexquant relies on its data science and quantitative research team to discover alpha—trading signals that predict price movements across global equity and futures markets. Unlike traditional tech companies where data science may focus on consumer app features, a Data Scientist at Trexquant builds predictive models, analyzes massive datasets, and develops algorithmic trading strategies that directly drive investment portfolio performance.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

Separate forecast decay from execution cost
Measure shortfall against arrival, not VWAP
Model impact and borrow before claiming capacity

Overview

The impact of this role is immediate and measurable. You will transform terabytes of structured and alternative market data into mathematical models and statistical features. Whether you are engineering alpha signals, developing NLP strategies to ingest unstructured text, or building automated simulation frameworks, your quantitative output directly feeds Trexquant's systematic production trading engines. The firm's proprietary platform processes thousands of input features, requiring candidates to demonstrate exceptional statistical rigor, clean production-ready Python code, and deep intuition for empirical data patterns.

Candidates joining the Trexquant Investment team enter an environment that balances rigorous academic research with high-frequency quantitative execution. You will work on complex mathematical problems, evaluate market dynamics, and build production pipelines where algorithmic efficiency and code accuracy are paramount. Succeeding in this role requires strong mathematical maturity, resilience under technical scrutiny, and an ability to translate abstract statistical concepts into actionable, high-performing trading algorithms.

The Trexquant Investment interview process

Trexquant Investment candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Coding Challenge

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Technical Interviews

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Behavioral Assessments

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Modelling transaction cost as a constant number of basis points, independent of order size and volatility.

Temporary market impact scales approximately with volatility times the square root of participation, that is, of order quantity divided by average daily volume, so cost per share rises as size rises rather than staying flat. A constant-bps assumption is roughly right for the small orders used to calibrate it and badly wrong for the size the strategy would actually run, which is how a book that backtests well at modest notional loses money at ten times the size. It also makes capacity unmeasurable, because capacity is exactly the notional at which marginal impact equals marginal alpha.

Reporting the best backtest out of many trials as if it were a single pre-registered test.

The maximum of N noisy Sharpe estimates grows roughly like the standard error times sqrt(2 ln N) even when every underlying strategy has zero edge, so with a few hundred variants an in-sample Sharpe near 1 is the expected result of pure noise. Worse, the search is rarely counted honestly: parameter sweeps, universe changes, date-range choices and feature variants all count as trials. Quote the number of configurations tried, deflate the Sharpe for it, and keep a genuinely untouched holdout period. Note also that the asymptotic standard error of a Sharpe estimate is approximately sqrt((1 + SR^2/2)/T) for i.i.d. normal returns, which for three years of daily data is roughly 0.33, so two strategies differing by 0.3 in Sharpe are not distinguishable.

Ending an analysis without a recommendation or next step

Close with what you would do and what would change your mind, stated as a condition you can check later. If the evidence is genuinely inconclusive, recommend the specific next measurement and say what it costs in time or exposure.

Over-explaining the method and under-explaining the implication

Lead with the answer and what you would do about it, then give the approach when asked. Roughly one sentence of method per three of implication is the right ratio for a stakeholder-facing answer; the interviewer already knows what a regression is.

Questions & practice

Walk through a mathematical proof demonstrating the properties of conditional probability and how Bayes' theorem can be applied to iterative pattern matching.

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Sanity-check the answer against a simple bound or a simulated case.
Translate the result into the decision it informs, in one plain sentence.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

How do you design an A/B testing framework to evaluate the performance of two competing execution algorithms operating in live financial markets?

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Permutation test for mean IC with overlapping forward returns

Hard · permutation test · signal evaluation · autocorrelation

panel has as_of_date, instrument_id, zscore_xs and fwd_ret_5d_excess, the five-trading-day forward return in excess of the universe's cap-weighted mean, for 252 dates and roughly 1,500 instruments a date. Compute the daily Spearman rank IC and its mean. Then build a permutation null from scratch and report a two-sided p-value. Compare it with a naive t-test on the 252 daily ICs and with a Newey-West t using four lags. State which null hypothesis your permutation scheme actually tests.

Approach

Compute the daily IC by ranking both columns within as_of_date with method='average' and taking the Pearson correlation of the rank vectors. Decide explicitly how to treat dates where the universe shrinks, because an unweighted mean gives a 40-name date the same weight as a 1,500-name date.
The obvious permutation shuffles zscore_xs within each as_of_date. That is the right exchangeability for the null 'the signal carries no cross-sectional ordering on any given day', and it preserves the daily sample size and the cross-sectional return structure, factor effects included.
But it makes the permuted daily ICs independent across dates, and the real ones are not. A five-day forward return sampled every day shares four days with its neighbour, and the signal itself is persistent, so the observed IC series is strongly autocorrelated. Under an i.i.d. daily return null the variance of the mean of overlapping h-period observations inflates by a factor of h, so the within-day null is too narrow by about sqrt(5) and its p-value is anti-conservative.
Fix it by permuting at the date level: circularly shift the whole signal panel's as_of_date labels by a random offset, keeping each day's cross-section and the signal's serial persistence intact while breaking its alignment with the returns. The 252 distinct shifts give an exact randomization test with a p-value floor of 1/252.
Report all three numbers. The naive t and the Newey-West t with 4 lags (h-1 for h = 5) should differ by a factor near sqrt(5), and the shift-permutation p should agree with the Newey-West figure rather than the naive one. If it does not, one of the two is implemented wrongly and you have a cheap way to find out which.

Worked solution (45 min)

ranks = panel.groupby('as_of_date')[['zscore_xs','fwd_ret_5d_excess']].rank(method='average'); daily IC is the per-date Pearson correlation of the two rank columns.
Within-day scheme: one rng.permutation of the signal rank vector inside each date, 10,000 replications of the mean IC.
Shift scheme: remap as_of_date through a circular offset over the sorted unique dates, re-join to the returns on (shifted_date, instrument_id), and record how much of the panel drops because an instrument is absent on the shifted date, since a large loss biases the null.
naive_t = mean(IC)/(std(IC, ddof=1)/sqrt(252)); Newey-West t with a Bartlett kernel and 4 lags.
Put the three p-values, the two t-statistics, the replication count and the p-value floor in one table.

Expected result: The within-day permutation p-value tracks the naive t and is the smallest of the three. The Newey-West t is smaller than the naive t by a factor near sqrt(5) = 2.24, and the shift-permutation p sits close to it. A mean IC of 0.03 with a daily IC standard deviation near 0.10 gives a naive t around 4.8 and a Newey-West t around 2.1, which is the difference between 'obvious' and 'worth one more year of data'.

Verify it yourself

Switching the forward horizon to one day should collapse the three answers toward each other; if they still differ by a factor of two, something other than the overlap is driving the gap.
The mean of the permuted mean-IC distribution must sit near zero under both schemes; a non-zero centre means the permutation is leaking the alignment it is meant to break.
The shift permutation at offset zero must reproduce the observed statistic exactly.
The daily IC series should show positive autocorrelation out to about four lags and near zero beyond; if it does not, the overlap correction is unnecessary here and you should say so rather than apply it anyway.

Follow-up

With 252 circular shifts and the observed statistic the largest among them, what is the smallest p-value you can report, and is it small enough for the decision being made?
How would you redo this on non-overlapping five-day samples, and how much power do you surrender?
The signal has a 12-day decay half-life. Does that change your shift block length or your Newey-West lag choice?

Explain how you would optimize a SQL query joining a multi-billion row market order book table with a factor dataset, focusing on index strategies, partitioning, and execution plans.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Given a table of daily stock trade transactions, write a SQL query using window functions to identify the top 3 highest volume trading intervals per sector for each calendar month.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Deduplicate a restated signal table into a leak-free panel

Medium · window-functions · dedup · point-in-time

signal_score may hold several rows per (signal_id, instrument_id, as_of_date) because reruns rewrite history; each row carries model_version, knowledge_ts, computed_at, is_backfilled, zscore_xs, decile_rank, coverage_flag and universe_id. Build the panel a backtest may legitimately use on trading date D: for every instrument in universe_id = 'liquid_us_1500', the single most recent score whose knowledge_ts is at or before D 21:00 UTC, preferring a live row over a backfilled one. Return instrument_id, as_of_date, zscore_xs, decile_rank.

Approach

Filter on knowledge_ts, never on as_of_date. as_of_date says what the score describes; knowledge_ts says the earliest instant every input was observable. Put it in the WHERE so it prunes before the window is evaluated.
Rank within the key: ROW_NUMBER() OVER (PARTITION BY signal_id, instrument_id ORDER BY as_of_date DESC, is_backfilled ASC, knowledge_ts DESC, computed_at DESC) and keep rank = 1. In Postgres FALSE sorts before TRUE, so is_backfilled ASC is the live-row preference, written down rather than assumed.
Prefer ROW_NUMBER to a MAX(computed_at) group-then-rejoin: the rejoin duplicates rows whenever two reruns share a computed_at, which is exactly what a batch job produces.
Decide what coverage_flag IN ('stale','imputed') means for this panel and encode it. Dropping those rows silently changes the universe size day to day, which surfaces later as an unexplained jump in measured IC rather than as a missing-data problem.
Express the cutoff as a parameter and the whole thing as a CTE so the backtest loop reuses one query text per date instead of a hand-edited copy.

Worked solution (25 min)

Write the filtered CTE: universe_id = 'liquid_us_1500' and knowledge_ts <= :cutoff.
Add the ROW_NUMBER window with the four-key ordering and select rank = 1.
Assert one row per instrument with a COUNT(*) versus COUNT(DISTINCT instrument_id) probe.
Run the same query with as_of_date <= D substituted for the knowledge_ts filter and count how many rows differ.
Inspect the differing rows: they should be predominantly is_backfilled = TRUE with computed_at after D.

Expected result: Exactly one row per instrument_id that has a qualifying score, with every knowledge_ts at or before the cutoff; size near the universe's nominal 1500 but smaller on days with coverage gaps.

Verify it yourself

COUNT(*) equals COUNT(DISTINCT instrument_id).
MAX(knowledge_ts) in the result is at or before the cutoff.
ABS(zscore_xs) <= 3 for every row, consistent with the documented winsorization.
Excluding is_backfilled = TRUE rows entirely changes only instruments whose sole qualifying version was a backfill.

Follow-up

Write the query that proves the panel is leak-free. What would a violation look like in measured IC, and roughly how large would you expect the inflation to be?
Produce every trading date in one pass instead of one query per date. What does that cost in plan shape?
Two model_version values are live at once during a migration. How does the tie-break change, and who decides?

How do you design a comprehensive product metric suite to monitor both latency and statistical signal decay in a live automated trading environment?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

You are tasked with developing a data pipeline for evaluating a new alternative dataset. How would you design a framework to measure signal quality, feature persistence, and degradation over time?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you approach building a statistical framework to evaluate whether a new trading signal adds true incremental alpha versus merely tilting into existing risk factors?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize research initiatives when managing multiple high-conviction ideas under tight project deadlines?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you test for statistical significance in financial time series when data exhibits severe autocorrelation, heavy tails, and heteroskedasticity?

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
Name the guardrails that would stop a launch even on a positive primary result.
Say whether units interfere with each other, and switch design if they do.

Follow-up

How would you handle interference between treated and control units?
What would you do if you could not randomise at all?

Discuss the trade-offs between backtesting, out-of-sample paper trading, and live micro-allocation A/B testing when validating new alpha research ideas.

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Name the guardrails that would stop a launch even on a positive primary result.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

Size an execution algorithm test on implementation shortfall

Easy · power · mde · execution

Per-order implementation shortfall against arrival price, computed from parent_order (order_qty, filled_qty, arrival_mid_px, avg_fill_px, adv_20d) and execution_fill (commission_amt, exchange_fee_amt, rebate_amt), has averaged 12.4 bps with a standard deviation of 35 bps over the trailing twelve months. The desk releases roughly 1,900 parent orders a month. An execution team proposes a new child-order scheduler and claims a 1.5 bps improvement. Size a two-arm, order-level randomised test at 80 percent power and a two-sided 5 percent level, and say how long it has to run.

Approach

Pin the response variable to the exact metric definition before touching the arithmetic: shortfall in bps versus arrival_mid_px, with commission, exchange fee and rebate included and the unfilled remainder (order_qty minus filled_qty) charged at the terminal mid. The sigma you plug in has to be the standard deviation of that quantity, not of a convenient proxy.
Estimate sigma from the trailing twelve months of the same order population rather than from a remembered number. Shortfall is heavy tailed, so also compute a winsorised sigma and quote the sample size both ways, because the difference tells you how much of your power calculation rests on a handful of orders.
Apply the two-sample difference-in-means formula, n per arm = 2 * sigma^2 * (z_{0.975} + z_{0.80})^2 / delta^2, which under equal allocation reduces to 15.7 * sigma^2 / delta^2.
Decide whether the shipping decision is made on the equal-weighted per-order mean or the notional-weighted mean, because they have different variances. For a weighted mean the effective sample size is (sum of weights)^2 / (sum of squared weights), and notional weights in a book with a long tail of large orders can cut that well below the row count.
Convert to calendar time at realised flow and check the answer against the business horizon. If nine months is unacceptable, the legitimate levers are a larger delta, lower power accepted explicitly, or variance reduction; shortening the test while keeping the same stopping rule is not one of them.

Worked solution (20 min)

Compute per-order shortfall for the trailing twelve months from parent_order joined to execution_fill, charging the unfilled remainder at the terminal mid, and confirm sigma is about 35 bps.
Record the notional-weighted effective sample size (sum of weights)^2 / (sum of squared weights) on the same population, so you know the discount before you commit to an estimator.
Evaluate n per arm = 2 * 35^2 * (1.96 + 0.8416)^2 / 1.5^2.
Divide the two-arm total by 1,900 orders a month to get calendar time.
Recompute at delta = 3.0 bps to demonstrate the quadratic sensitivity to the claimed effect.

Expected result: About 8,550 orders per arm, 17,100 in total, roughly nine months at 1,900 orders a month. Doubling the claimed effect to 3.0 bps cuts the requirement by a factor of four, to about 2,140 per arm and roughly two and a quarter months.

Verify it yourself

(z_{0.975} + z_{0.80}) = 2.8016 and its square is 7.85, so the whole calculation collapses to n = 15.7 * sigma^2 / delta^2 per arm; 15.7 * 1225 / 2.25 should reproduce 8,548.
Halving the detectable effect must multiply the sample size by exactly four; if your spreadsheet does not do that, the formula is wrong.
Simulate: resample two arms from the empirical shortfall distribution with a 1.5 bps shift at n = 8,550 per arm and confirm the rejection rate lands within a point or two of 0.80. A materially lower rate means the normal approximation is straining on the tails.

Follow-up

What happens to the required sample size if the committee decides on the notional-weighted mean instead of the equal-weighted mean?
At 1,900 orders a month, what effect size can you detect in a single month, and would you run that test at all?
If the real goal is 'no worse than the incumbent', how do you set the non-inferiority margin and what changes in the test?

Monthly shortfall jumped while daily medians stayed flat

Easy · tca · seasonality · mix shift · weighting

Notional-weighted implementation shortfall versus arrival rose from 18.4 bps in one month to 27.1 bps in the next. The desk changed no brokers and no default algo. You have parent_order (order_id, instrument_id, side, order_qty, filled_qty, order_type, arrival_ts, arrival_mid_px, avg_fill_px, terminal_ts, adv_20d, algo_name, broker_code) and execution_fill (order_id, exec_ts, fill_qty, fill_px, commission_amt, exchange_fee_amt, rebate_amt). Decide whether trading got worse or the composition of the month changed. Deliverable: an ordered checklist, a one-paragraph verdict, and the decomposition table behind it.

Approach

Recompute shortfall per trading date, notional-weighted within the date, and then recompute the monthly figure weighting each date equally. If the equal-by-date number barely moves while the notional-weighted number jumps, the change is in which dates carried the notional, not in how any date traded.
Rank dates by share of month notional. Quarter-end, index reconstitution and large closing auctions concentrate notional into a handful of sessions, and those sessions are structurally expensive: spreads are wider and participation runs far above the month's median.
Bucket orders by participation, order_qty divided by adv_20d, into fixed buckets and compare within-bucket shortfall across the two months. Temporary impact scales roughly with volatility times the square root of participation, so a shift in the participation distribution moves the blended number with execution quality held constant.
Cut by order_type, separating close_auction and implementation_shortfall orders from vwap and pov, and check whether that mix moved too.
Deliver the verdict as arithmetic: how many of the 8.7 bps are within-bucket and how many are mix, rather than a narrative about the month.

Follow-up

Your equal-by-date average removes the effect. Which weighting belongs in the monthly TCA report shown to portfolio managers, and why is the other one still worth printing?
If the expensive sessions are unavoidable because the strategy rebalances on them, what would you change about the report so the number stays actionable?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Stories where someone disagreed with you

Have two ready. In one, the data was on your side and you had to move someone who outranked you. In the other, the pushback was correct and you changed position. The second is the harder story and it lands better, because it shows you separate being right from being attached to an answer. Name the person's actual objection.

Walk me through a time when an interviewer or senior executive rigorously challenged the authenticity or implementation details of your code. How did you handle the high-pressure interaction?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What would you do differently if you ran that project again?
What did you decide not to do, and why?

Defending a backtest correction that removes an allocated strategy

Medium · stakeholder pushback · backtest integrity · lookahead bias

A colleague's cross-sectional equity signal received capital last month on a backtest showing annualized Sharpe 1.6 over three years. Reproducing it, you find the panel filters signal_score on as_of_date rather than knowledge_ts. Rerunning with the knowledge_ts predicate, holding universe_id, date range, cost model and rebalance schedule fixed, gives Sharpe 0.5 and moves mean daily IC from 0.041 to 0.012. The colleague is senior and presented the original result. You have one meeting with the research lead and the portfolio manager. Deliver a recommendation on whether the allocation stands, and the evidence behind it.

Approach

What is probed: whether a quantitative objection survives social cost. Lead with the defect and the single-variable reproduction, never with a judgement about the colleague. The claim under test is one predicate, not a person's competence.
Rerun both versions from one code path with one line changed, and say so explicitly. Holding universe_id, date range, cost model and rebalance schedule fixed leaves the 1.1 Sharpe gap exactly one candidate cause, which is what makes the result arguable on its merits instead of on whose code is trusted.
Pair the comparison before quoting any error bar. For modest Sharpe ratios the annualized standard error of a Sharpe estimate is approximately 1 over the square root of the number of years, so three years of daily data gives roughly 0.58 and two marginal estimates 1.1 apart are only about two standard errors apart. But the two runs are the same returns except where the leak bites, so test the daily difference series directly: its standard error is far smaller, and the pairing is what turns a marginal result into a decisive one.
Exhibit the mechanism, not just the size. Rank instrument-days by their contribution to the return difference between the two runs and show that the top contributors carry is_backfilled TRUE or coverage_flag 'stale', with knowledge_ts postdating as_of_date by the vendor's restatement lag. A named mechanism is falsifiable; a Sharpe delta alone becomes an argument about your code.
Bring a decision rather than only a finding: the position size the corrected Sharpe supports, and an untouched out-of-sample window that would settle it either way. Being right with no path forward is how a correct objection gets overruled.

Follow-up

Your colleague reruns it and gets 0.9 rather than 0.5. What do you do with the discrepancy before the meeting?
The strategy is up since funding. Does live P and L change your recommendation, and how much of it would?
What would have caught this before the allocation, and why did the existing review not catch it?

Explaining why two accounts in one strategy diverged 210 bps

Easy · client communication · attribution · reconciliation

Two separately managed accounts run the identical strategy. Over the trailing twelve months one returned 8.4 percent net and the other 6.3 percent, a gap of 210 bps. The client who owns the lower one has asked in writing why. You have position_daily, account_mandate (SCD2) and the fill history for both accounts. Produce two things: a reconciliation that accounts for the gap down to a residual you state, and a reply of at most 200 words that a non-specialist can act on, without jargon and without blaming the client.

Approach

What is probed: whether you reconcile to the total before you explain anything. An explanation that does not sum to the observed gap is a guess with numbers attached, and the client's next analyst will find the difference.
Reconcile mechanically and in this order, because each step has a clean source: fee difference from mgmt_fee_bps plus any performance fee crystallised above the high-water mark; average cash weight, as one minus the summed weight_pct_nav; restricted names, as is_restricted days multiplied by what those names returned; single-name cap differences across account_mandate SCD2 versions; and the ramp between funded_date and the date gross exposure first reached 90 percent of target.
Carry a residual and state it plainly. If five components explain 170 of 210 bps, the reply says 40 bps unexplained rather than largely explained by. The residual is usually trade timing across the two accounts, and naming it is cheaper than having it discovered.
Order the reply by what the client can do rather than by size of component: what is structural and will persist, such as the fee schedule and their own restricted list; what was one-off and will not repeat, such as the ramp; and what they can change if they choose to.
Remove every term that would need looking up. Your restricted list kept the account out of three names that contributed 62 bps is actionable. Negative selection effect from compliance constraints is not, and the decision to relax a restriction belongs to the client rather than in a recommendation you push.

Follow-up

The residual is 120 bps rather than 40. What goes in the letter, and what do you do before sending it?
The client asks whether their account was deliberately disadvantaged. How do you answer that specific question?
Does the other client need to be told anything, and who decides?