cswork / libraryOpen practice workspace

Interview guides · GIC

GIC Data Scientist Interview Questions & Guide 2026

Data scientist interview prep for GIC: original practice on experiment design, SQL and metric reasoning, with worked solutions.

The 60-second brief

A Data Scientist at GIC plays a pivotal role in leveraging data to drive decision-making and enhance the investment strategies of the firm. This position is critical, as it involves analyzing vast datasets to uncover insights that can optimize portfolios, identify market trends, and inform strategic decisions. The work you do will directly impact the effectiveness of investment strategies and, consequently, the overall performance of GIC's assets.

If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.

Focus areas

  • Define a qualified stream and defend it
  • Measure catalogue breadth beyond head consumption
  • Split voluntary churn from involuntary payment failure

Overview

In this role, you will collaborate with various teams, including portfolio managers, quantitative analysts, and technology specialists, to develop models and tools that facilitate data-driven investment decisions. You will engage with complex problem spaces such as risk assessment, predictive modeling, and algorithmic trading, making your contributions not only significant but also intellectually stimulating. Expect to work on high-stakes projects that require innovative solutions and a deep understanding of machine learning and statistical analysis.

The GIC interview process

GIC candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Online Coding Assignment

A take-home is graded as an argument, not as a notebook. Somebody reads the submission without you in the room, so every choice has to survive on the page: why the question was framed this way, and what was deliberately left out. The gap between a strong and a weak submission is almost never model quality. It is whether the writeup names the specific question it answers and commits to a recommendation, including what evidence would overturn it. A high-accuracy model attached to no conclusion reads as effort that stopped before the decision.

What to demonstrate

  • Whether the question you answered is stated outright, and whether it is the question the prompt posed rather than an easier neighbour of it
  • Whether the recommendation is specific enough to act on, with the uncertainty attached to it instead of parked in a caveats section at the end
  • Whether analytical choices such as the metric definition, the population filter and the time window are justified in the prose, not merely visible in code

How to prepare

  • Take a dataset you have already worked with, write the one-paragraph conclusion first, then check whether the analysis you were planning actually supports it and cut whatever does not
  • Practise stating a metric in one sentence that fixes the population, the time window and the denominator, then confirm your query computes exactly that sentence and nothing adjacent to it
  • Hand a draft to someone outside the problem and ask them to tell you back what you recommended and why; anything they cannot recover is not on the page yet

Onsite Interview

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

  • Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
  • Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
  • Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

  • Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
  • Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
  • Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
  • Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Comparing consumption week over week across the release calendar and the rights calendar.

A major release, a season drop or a live event produces a spike that dwarfs almost any treatment effect, and the effect is not confined to the new title because it pulls attention from everything else in the same window. Separately, licensed content leaves the catalogue when its window expires, so consumption falls with no product change and the drop is attributed to whatever shipped that week. Both need to be handled by an explicit control: a comparison period chosen for calendar equivalence, a covariate for scheduled releases, or a pre-registered rule for excluding a window, decided before the numbers are seen.

Counting plays without a qualification threshold, or changing the threshold without restating history.

Playback arrives as heartbeats, so a play only exists once you decide what counts, and the common 30-second convention is not a neutral analytics choice: in music it is also the boundary at which a play becomes payable, which makes the warehouse definition a payout definition. The threshold interacts violently with content length, so a catalogue of three-minute tracks and one of forty-minute episodes move in opposite directions when you change it, and a skip-heavy surface can add plays while adding no hours. Any metric mixing pre-threshold and post-threshold counts, or pooling short-form and long-form on a per-stream basis, moves by double digits for reasons that have nothing to do with the product.

Dropping rows with missing values without naming the mechanism

Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.

Naming a model class before naming the deployment constraints

Set out the latency budget, the label delay, the retraining cadence, the interpretability requirement and the number of labelled examples, then pick the model that fits them. A boosted-tree answer to a problem where each decision must be explained to the affected user is a well-executed answer to the wrong question.

Questions & practice

Discuss a machine learning model you have developed and its impact.

Medium · machine learning and modelling

Approach

  1. Say how the offline result would be validated online before it is trusted.
  2. Set a baseline first, so any model has something honest to beat.
  3. Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • Where could label leakage enter this setup?

Write a Python function to implement a basic linear regression model.

Medium · machine learning and modelling

Approach

  1. Say how the offline result would be validated online before it is trusted.
  2. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • How would you choose the decision threshold, and who owns that choice?

Sessionise playback heartbeats into streams under an idle-gap rule

Hard · sessionisation · gaps-and-islands · pandas

heartbeats has profile_id, device_id, content_version_id, heartbeat_ts, playhead_seconds, is_paused and reported_at; clients emit one every 30 seconds while in the foreground. Collapse it into stream rows keyed on (profile_id, device_id, content_version_id) under a 300-second idle gap. Emit started_at, ended_at, max_position_seconds and played_seconds, where played_seconds counts advancing playback only and so excludes pause, seek and rebuffer time. Flag streams whose reported_at trails heartbeat_ts by more than an hour. Then re-run at a 60-second gap and report what moved.

Approach

  1. Sort by the key and heartbeat_ts, take the per-row time delta inside the key with groupby().diff(), mark a new stream where that delta is null or above 300 seconds, and cumulative-sum the flag into a stream ordinal. This is gaps-and-islands, and it is O(n log n) after the sort with no per-group Python loop.
  2. Compute played_seconds from consecutive heartbeats as min(max(playhead_delta, 0), wall_delta), summed inside the stream. The lower clamp discards a backward seek; the upper clamp is what stops a forward seek to the end of a forty-minute episode from crediting forty minutes of playback to one thirty-second interval. Note what this makes the column mean: a rewind and replay inside one stream counts the replayed interval twice, because played_seconds measures time spent advancing the playhead, not distinct coverage of the item.
  3. Accept that the first heartbeat of each stream has no predecessor and contributes nothing, which undercounts by up to one heartbeat interval per stream. Quantify that bias as stream count times 30 seconds against the total instead of leaving it unsaid.
  4. Take ended_at as the last heartbeat_ts, but keep it null when that heartbeat sits within one idle gap of the data cutoff: such a stream may still be running, and giving it an ended_at manufactures a completed stream.
  5. For the offline flag, work on reported_at minus heartbeat_ts rather than on event_date, and report the distinct dates the flagged streams belong to, because that list is the recompute list.
  6. Re-run at 60 seconds and report both the stream count and the change in total played_seconds: the count can only rise, and the total falls by exactly the contributions that now straddle a split.

Worked solution (40 min)

  1. Sort by (profile_id, device_id, content_version_id, heartbeat_ts); compute wall_delta as the grouped heartbeat_ts diff in seconds and pos_delta as the grouped playhead_seconds diff.
  2. new_stream = wall_delta.isna() OR wall_delta above 300; stream_ordinal = new_stream grouped-cumsum; group on the key plus stream_ordinal from here on.
  3. contrib = np.minimum(np.maximum(pos_delta, 0), wall_delta), forced to 0 on the first row of each stream; played_seconds is contrib summed per stream.
  4. Aggregate started_at as min(heartbeat_ts), ended_at as max(heartbeat_ts), max_position_seconds as max(playhead_seconds), and late_offline as max(reported_at minus heartbeat_ts) above 3600 seconds.
  5. Null ended_at where it falls within 300 seconds of the data cutoff.
  6. Re-run with the gap as a parameter at 60 seconds, and diff the two summaries on stream count and total played_seconds.

Expected result: One row per stream whose played_seconds never exceeds ended_at minus started_at in seconds, since each interval contributes at most its wall_delta and the wall_deltas inside a stream telescope to exactly that span. At a 60-second gap the stream count is greater than or equal to the 300-second count, and total played_seconds is lower by exactly the sum of the contributions that now straddle a split.

Verify it yourself

  • Assert played_seconds at or below (ended_at minus started_at) in seconds for every stream; any violation means the clamp against wall_delta was dropped.
  • No stream spans two content_version_ids or two device_ids. Group the heartbeats back by stream ordinal and check nunique equals 1 on both.
  • A hand-built three-heartbeat case with a backward seek (playheads 600, 300, 330 at 30-second spacing) yields played_seconds of 30, not 60 (the wall clock) and not -270 (the raw playhead deltas).
  • The distinct event_date partitions of the late_offline streams are exactly the days that need recomputing, and that set should be small and old rather than spread evenly across the window.
  • Do not assert played_seconds against an item's runtime. The heartbeats input carries no duration_seconds, and the bound does not hold even if you join one: a backward seek followed by a rewatch advances the playhead over the same interval twice inside one stream, and both passes are genuine playback. max_position_seconds is the column that compares against a joined runtime, and then only within the segment-overshoot tolerance of a second or two.

Follow-up

  • A client dies mid-playback and stops reporting. What does its stream look like under this rule, and what does that do to mean stream duration?
  • Which of these numbers determines a per-stream payout, and what does moving the idle gap do to somebody's money?
  • The same profile plays the same track on a phone and a TV at once. Your key separates them. When would you want it not to?

Solve a coding challenge related to data manipulation using Python.

Medium · sql and data manipulation

Approach

  1. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  2. Say which table is the grain you start from, and join outward from it.
  3. State the window function and its partition and ordering out loud before writing it.

Follow-up

  • How would you verify this result without re-running the same query?
  • What breaks if events arrive late or out of order?

How would you handle missing data in a dataset?

Medium · sql and data manipulation

Approach

  1. Check whether any join is one-to-many before aggregating, or the sums inflate.
  2. Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
  3. Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

  • What breaks if events arrive late or out of order?
  • How would you verify this result without re-running the same query?

Weekly engaged accounts from qualified streams on distinct days

Easy · aggregation · active accounts · qualification threshold

fct_stream has one row per playback with account_id, started_at (UTC), played_seconds, is_qualified (true when played_seconds >= 30) and event_date, a date partition key derived from started_at. dim_account has account_id and is_test_account. Write one query returning, for the seven days ending 2026-03-31 inclusive, the count of weekly engaged accounts: distinct non-test accounts with at least one qualified stream on two or more distinct UTC dates inside that window. Return a single row. Count at the account level, not the profile level.

Approach

  1. Filter fct_stream on event_date BETWEEN '2026-03-25' AND '2026-03-31' and is_qualified = true. Filtering on started_at alone leaves the engine no partition to prune, so it reads every day the table holds; event_date is the partition key and is what belongs in the predicate.
  2. Drop test accounts before the date count, with a join or semi-join to dim_account on is_test_account = false. Excluding them afterwards is too late, because a test account that streamed on two days has already been counted as engaged.
  3. GROUP BY account_id and take COUNT(DISTINCT event_date) AS active_days. The distinct is the entire metric: an account with forty qualified streams inside one calendar day has one active day.
  4. Filter HAVING COUNT(DISTINCT event_date) >= 2, then wrap in an outer COUNT(*). The group-by has already made account_id unique, so an outer COUNT(DISTINCT account_id) adds a sort for nothing.
  5. Sanity-bound the answer before reporting it: it must be no larger than the count of distinct non-test accounts with any qualified stream in the window.

Worked solution (20 min)

  1. Compute the ceiling first: COUNT(DISTINCT account_id) over non-test accounts with any qualified stream in the window.
  2. Build a CTE of (account_id, COUNT(DISTINCT event_date) AS active_days) over the filtered rows.
  3. Select COUNT(*) from that CTE where active_days >= 2.
  4. Re-run the final filter at >= 1 and >= 3 to see the shape of the distribution and confirm the metric is not being carried by accounts sitting exactly on the boundary.

Expected result: A single integer. It must satisfy count(>=3) <= count(>=2) <= count(>=1), where count(>=1) is the ceiling computed in step one.

Verify it yourself

  • The three thresholds are monotone: raising the active-days requirement never raises the count.
  • Removing the is_test_account filter raises the count or leaves it unchanged, never lowers it.
  • No account_id appears more than once in the pre-aggregate CTE.

Follow-up

  • If you counted distinct dates per profile and then rolled up to the account, would the number rise or fall, and which unit does the metric actually want?
  • An account plays one 40-second track at 23:58 and another at 00:03. That is two distinct UTC dates and therefore engaged. Is that the intent, and what would you change if not?
  • Offline plays upload in bulk several days late with started_at backdated. Which part of this query silently changes when they land?

What metrics would you consider key when evaluating a new investment opportunity?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

Given a dataset, how do you optimize for runtime efficiency in your code?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • Which segment would you cut first, and what would that rule out?

Discuss how you would analyze the impact of a new market trend on existing investment strategies.

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

How would you approach a situation where the data collected is insufficient for analysis?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • How would you detect that the metric is being gamed rather than genuinely improving?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

  1. Clarify what is being asked and what a complete answer would contain.
  2. State your assumptions explicitly before working the problem.
  3. Work from the decision backwards to the evidence you would need.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Size a ranker test on heavy-tailed weekly listening hours

Easy · power · minimum detectable effect · heavy tails

You are sizing a two-arm, account-randomised test of a new home-row ranker. The primary metric is qualified hours per active account-week from fct_stream (sum(played_seconds)/3600 where is_qualified = true). Over the last four complete ISO weeks the per-account weekly figure has mean 4.6 hours and standard deviation 9.8 hours, median 1.9 hours, and the top 1 percent of accounts hold 14 percent of hours. Eight million accounts are eligible, split evenly, over a 14-day window. Give the minimum detectable effect at 80 percent power and alpha 0.05 two-sided, and say what you would change before trusting it.

Approach

  1. Fix the unit of analysis to the randomisation unit: one row per account holding its 14-day total, not one row per account-week and certainly not one per stream, because two weeks from the same account are positively correlated and do not count as two independent observations.
  2. Apply the two-sample formula MDE = (z_0.975 + z_0.80) * sigma * sqrt(2/n), with z_0.975 + z_0.80 = 1.960 + 0.842 = 2.802 and n the accounts per arm, then divide by the control mean for a relative figure.
  3. Check what the skew actually threatens. At n in the millions the central limit approximation for the sample mean is comfortable, so skew does not break the test; it makes the variance hostage to a few hundred accounts and to whatever released inside the window.
  4. Pre-register a variance control and commit to reporting both readings: hours capped at the 99th percentile, and a two-part decomposition into the share of accounts with any qualified stream times conditional hours among those who had one.
  5. Compare the MDE against the smallest effect worth shipping. If the MDE lands far below that, power is not the binding constraint and the design effort belongs on calendar confounding and on the active-account denominator instead.

Worked solution (20 min)

  1. Collapse fct_stream to one row per account_id over the 14-day window: total qualified played_seconds divided by 3600. Record the count of assigned accounts with zero qualified hours separately, because the active-account denominator is itself treatment-sensitive.
  2. Set n = 4,000,000 per arm and sigma = 9.8, then compute sqrt(2/n) = sqrt(5.0e-7) = 7.071e-4.
  3. MDE = 2.802 * 9.8 * 7.071e-4 = 0.0194 qualified hours per account-week, which is about 70 seconds.
  4. Convert to relative: 0.0194 / 4.6 = 0.42 percent.
  5. Repeat steps 2 to 4 with sigma taken from the 99th-percentile-capped series and put both MDEs in the design doc, with the cap fixed before any data is unblinded.

Expected result: MDE approximately 0.0194 qualified hours per account-week, or 0.42 percent relative, at 80 percent power and alpha 0.05 two-sided. The correct conclusion is that the test is over-powered for any effect the team would act on, so the real design risk is bias from the release calendar and from a shifting active-account denominator, not power.

Verify it yourself

  • Halving n to 2,000,000 per arm must multiply the MDE by exactly sqrt(2) = 1.414, giving 0.0275 hours.
  • The standard deviation of the 14-day total must be less than sqrt(2) times the weekly standard deviation, because weeks within an account are positively correlated; if it is not, the collapse is wrong.
  • An A/A split should put 95 percent of observed differences inside plus or minus 1.96 * sigma * sqrt(2/n) = 0.0136 hours, which is 70 percent of the MDE. A/A differences that routinely exceed the MDE mean the variance estimate or the independence assumption is wrong.

Follow-up

  • Capping at the 99th percentile cuts the standard deviation to 5.1 hours. What does that do to the MDE, and what have you given up if the true effect lives in the heavy accounts?
  • A season drop lands in the second week of the window. Does that change the variance, the bias, or both?
  • The team wants the same MDE in seven days instead of fourteen. What are you spending to get it?

Consumption fell the week a new ranker shipped

Hard · confounding · catalogue-supply · causal-inference

Qualified hours fell 4% in the seven days after a home-row ranker shipped to 100%, and a rollback is being demanded for tomorrow. You have fct_stream (played_seconds, is_qualified, content_version_id, start_source, started_at, play_territory), fct_impression (ranker_version, content_version_id, rendered_at), dim_content_version (content_id, parent_id, rights_expires_at, available_territories, catalogue_added_ts, is_active_version), and no holdout — the rollout went from 5% straight to 100%. Determine whether the ranker caused the fall. Deliverable: a defensible causal read, or an explicit statement of what cannot be concluded and what you would run instead.

Approach

  1. Check supply before demand. Sum the prior period's qualified hours on content_version_ids whose rights_expires_at falls inside the window, whose is_active_version flipped, or whose available_territories lost a territory. If titles carrying several points of hours left the catalogue that Monday, the fall is partly accounted for before the ranker is even discussed.
  2. Separate the mechanically unavailable from the merely deranked. Hours lost on items still available but no longer surfaced belong to the ranker; hours lost on items that cannot legally be served do not, and no ranking change recovers them. Only the first category is evidence for a rollback.
  3. Recover a comparison the rollout destroyed. With no holdout, use the 5% phase as a pre-period and territories unaffected by the expiry as a control, running difference-in-differences with the expiry as the treatment. State the parallel-trends assumption out loud and test it on the four pre-window weeks; abandon the estimator if it fails rather than reporting it with a caveat.
  4. Measure displacement instead of gross hours. For accounts whose usual items left, did total hours fall or did they substitute within the catalogue? Fully substituted removal costs nothing in hours and shows up only in the affected rights holder's payout, which is a different problem with a different owner.
  5. Ask separately what the ranker did to exposure: impression share by content_id under each ranker_version, and top-1% exposure concentration. A ranker that concentrates exposure can depress hours through a mechanism nobody has named yet, and this is what distinguishes that story from the rights story.
  6. Refuse the causal claim the rollout cannot support. Say plainly that a 5%-to-100% ramp across a confounded week yields no unbiased effect estimate, and propose the switchback or re-randomised holdout that would, with its cost stated.

Follow-up

  • Design the switchback: slice length, burn-in long enough to clear carryover on a personalised surface, and how the standard error handles correlation between adjacent slices.
  • If the rollback is ordered regardless of your analysis, what do you measure during the rollback to get a second, independent read?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

  • Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
  • For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
  • Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

  • Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
  • Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
  • Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

  • Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
  • Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
  • Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

  • Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
  • For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
  • Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

  • Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
  • Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
  • Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

  • Repeat the depth protocol on the second-ranked area with the same six-problem structure.
  • Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
  • Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

  • Re-run the six prompts from day one under the same clock and compare both correctness and time.
  • Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
  • Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

The project you argued against

Saying no well is a senior skill and it is rarely rehearsed. Think of a time you told someone their analysis was not worth doing, or that the experiment could not answer their question at the sample size available. Explain what you offered instead. Refusal without an alternative reads as obstruction rather than judgement.

Describe a time when your analysis led to a significant business decision.

Medium · behavioural and stakeholder questions

Approach

  1. Pick a story where you drove the decision, not one where you observed it.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Close with what you would do differently, concretely.

Follow-up

  • What did you decide not to do, and why?
  • How did you know the outcome was caused by your change?

Disagree with a product manager about a completion metric

Medium · disagreement · metric design · influence

A product manager proposes making duration-normalised completion rate the team's primary metric for the quarter, and wants a plain global unweighted version "so it's simple enough to put on a wall." You expect the unweighted version to move several points on catalogue mix alone, with no change in how satisfying anything was. You have one meeting, they own the roadmap, and you will work with them for years. Deliverable: the argument, the evidence you bring, and the fallback you accept if they still want the simple version.

Approach

  1. The probe is whether you can lose the decision without damaging either the metric or the relationship. Bring the failure already reproduced: recompute the unweighted global rate over the last two quarters and point at the weeks it moved several points where the only input that changed was which content people played.
  2. Decompose the variance instead of objecting. Attribute the movement in the unweighted rate to between-cell mix — (content_type, duration decile) — versus within-cell movement. A decomposition is an argument; an assertion that mix matters is a preference.
  3. Concede the real cost honestly: the mix-weighted version is harder to explain and harder to recompute. Bring the mitigation rather than dismissing the objection — weights fixed from a stated reference month, published once, so anyone can reproduce the number without redoing the weighting.
  4. Offer the fallback deliberately and in advance: ship both, with the unweighted rate labelled a diagnostic and the weighted one as the decision metric, and pre-agree the one observation that would mean the simple version has misled the team.
  5. Leave the decision with them, with the consequence written down before it happens. That is what makes a later correction a shared prediction coming true rather than a retrospective argument about who was right.

Follow-up

  • They ship the simple version and it moves four points during a heavy release week. How do you raise it without saying you told them so?
  • Is there a case where the unweighted rate is genuinely the right primary metric?
  • Suppose you are wrong and the mix effect turns out to be small. What would you change about how you argued this?

State the impact of your last year without inflation

Hard · self-assessment · attribution · counterfactual reasoning

You are asked what your work was worth over the last year. Two candidates are on the table. First: a dunning-schedule change you analysed, shipped in March with no holdout, after which the involuntary share of gross churn fell three points. Second: a metric-definition change you drove, which nobody can attribute revenue to. Deliverable: the impact claim you make for each, the counterfactual attached to each, and which one you lead with in a performance review — with the reason.

Approach

  1. The probe is whether you apply the same causal standard to your own work as to a product experiment. Attach the counterfactual before the claim: involuntary share of gross churn responds to card-updater coverage, billing_provider mix, retry schedules and the dunning horizon you read it at, any of which can move three points with the schedule change contributing nothing.
  2. Say which evidence would separate those, then say plainly which you actually have. A staggered rollout by billing_provider, a holdout, or at minimum a flat pre-period series would each support a different strength of claim; with none of them, the honest claim is a contribution to a favourable movement, stated as such.
  3. Check the classification itself before claiming the movement is real: the split between cancelled_voluntary and failed_involuntary must be computed only after the dunning horizon has closed for every period in the month, or retries still in flight are counted as churn and the share reads too involuntary in one direction and corrects in the other.
  4. Price your contribution rather than the outcome. The value of analysis that changed a decision is the decision's delta multiplied by the probability the decision would not have been taken without it, and being explicit about that second factor is most of what honesty means here.
  5. Make the definition-change claim concrete rather than apologetic: name the decisions that would have been taken on the wrong number, the reports it reconciled, the recurring argument it closed, with dates. Then lead with whichever claim survives questioning, not whichever carries the larger number, because an inflated first claim makes the second unbelievable.

Follow-up

  • Your manager writes up the churn improvement and credits it to you. Do you correct it, and to whom?
  • How would you have designed the March rollout so that attribution was possible, at what cost in delay?
  • Name something you worked on last year that had no impact, and say what you learned from that rather than from the wins.