Interview guides · Chevron

Chevron Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Chevron: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

As a Data Scientist at Chevron, you play a pivotal role in harnessing data to drive strategic decision-making and enhance operational efficiency. Your work will directly impact various aspects of the business, from optimizing production processes to improving safety protocols and enhancing customer experiences. By leveraging advanced analytics, predictive modeling, and machine learning techniques, you will contribute to Chevron's mission of delivering sustainable energy solutions while maximizing value for stakeholders.

A large share of questions open as "how would you measure X", where the real work is choosing the metric, fixing its denominator, and defining the population it applies to. Any computation comes last and is frequently not required at all.

Focus areas

Measure catalogue breadth beyond head consumption
Define a qualified stream and defend it
Correct discovery slates for position bias

Overview

This role is critical as it not only involves analyzing complex datasets but also requires translating insights into actionable strategies. You will collaborate with cross-functional teams, including engineering, product development, and operations, to implement data-driven solutions that tackle real-world challenges. The scale and complexity of the data you will handle, combined with the strategic influence of your insights, make this position both interesting and impactful.

Expect to engage with diverse projects that span various domains, such as exploration and production, refining, and marketing. Your contributions will help Chevron navigate the evolving energy landscape and maintain its competitive edge.

The Chevron interview process

Chevron candidates report 2 rounds · ≈ 2-4 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Interview Rounds

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Where candidates lose points

Testing hours, revenue or completion with a difference in means on a heavy-tailed distribution.

Listening and viewing hours per account are strongly right-skewed and content popularity is close to power-law, so the variance of a sample mean is dominated by a few accounts and the central limit approximation converges slowly at realistic sample sizes. A t-test on mean hours can flip sign when one heavy account's week changes, and an experiment can appear significant because a single title released into one arm's window. Capping at a pre-registered percentile, or decomposing into a rate (did they stream at all) and a conditional intensity, controls the variance, at the stated cost that capping biases toward zero exactly when the true effect lives in the tail.

Reading recommendation quality from click-through or play-through on logged slates.

Slate position drives interaction independently of relevance, and the ranker that produced the log also decided what got exposure, so logged data is confounded by the policy you are trying to evaluate. Inverse-propensity estimators fix this only when the logging propensities are known and every item had a non-zero chance of being shown; without exploration slots that positivity condition fails silently and the estimate is confidently wrong. The visible symptom is a feedback loop: offline metrics improve, consumption concentrates further into the head of the catalogue, and the online test does not reproduce the offline gain.

Reporting a mean for a heavy-tailed metric without saying what it hides

For spend, session length or items per order, a small fraction of units carries most of the total, so the mean has a wide standard error and one account can move it. Fix the handling before you see the result: cap or winsorise at a pre-declared percentile, and report the median or the share above a threshold next to the mean. Capping changes the estimand, so say which question the capped number answers, and check how much of any difference comes from the top 0.1 percent of units.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Questions & practice

Write a function to calculate the mean and standard deviation of a list of numbers.

Medium · statistics and probability

Approach

Say what the estimate is of, and over what population it generalises.
Write down the assumption the method needs before you use the method.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

Explain the time complexity of your favorite sorting algorithm.

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Five invariant checks over a playback fact table

Easy · data-quality · pandas · validation

fct_stream arrives with stream_id, started_at, ended_at (nullable), played_seconds, max_position_seconds, completion_ratio, is_qualified, event_date, and duration_seconds joined from the content dimension (null for live events). Write one function returning a tidy frame with a row per rule: rule name, eligible rows, violations, violation share, one example stream_id. The five rules are duplicate stream_id; played_seconds exceeding wall-clock ended_at minus started_at; max_position_seconds exceeding duration_seconds; completion_ratio disagreeing with max_position_seconds divided by duration_seconds; and event_date not equal to the UTC date of started_at.

Approach

Give every rule its own eligible-row mask before its violation mask. Rules 2 to 4 are undefined where ended_at or duration_seconds is null, and a comparison against NaN evaluates False, so a mostly-null column otherwise reports a clean bill of health.
Compare the recomputed completion_ratio with a tolerance (absolute difference above 1e-6), never with ==. It is stored as a float division and exact equality fails on rows that are correct.
Allow a small tolerance on rules 2 and 3 too: a final segment can carry the playhead a second or two past duration_seconds, and heartbeat timestamps come off client clocks. State the tolerance you chose rather than burying it.
Return violations and eligible rows as separate columns so the share has a stated denominator, and carry one example stream_id per rule so the output is actionable rather than a number.
Read rule 5 as a signal rather than corruption: event_date drifting from the UTC date of started_at is the signature of offline playback uploaded after the partition closed, and its distinct dates are the recompute list.

Worked solution (20 min)

Write a helper taking (name, eligible_mask, violation_mask) and returning one result row, which forces every rule to declare its eligible population.
Rule 1: duplicated('stream_id', keep=False), eligible is all rows. Rule 5: event_date not equal to started_at.dt.date, eligible is all rows.
Rules 2 to 4: eligible masks are ended_at.notna() and duration_seconds.notna(); violation masks are played_seconds above wall_seconds plus 2, max_position_seconds above duration_seconds plus 2, and the absolute completion_ratio difference above 1e-6.
Take one example stream_id per rule with .loc[mask].head(1), guarding the empty case so a clean rule does not raise.
Concatenate the five rows and sort by violation share descending.

Expected result: A five-row frame with columns rule, eligible_rows, violations, violation_share and example_stream_id, where eligible_rows for rules 2 to 4 is strictly below the table length whenever ended_at or duration_seconds contains nulls.

Verify it yourself

Injecting one row whose played_seconds is ten times its wall clock raises exactly rule 2's count by one and leaves the other four unchanged.
Setting duration_seconds to null on that same row moves it out of the eligible counts for rules 3 and 4, rather than into their violation counts.
violation_share is violations divided by that rule's eligible_rows, not by len(df); the two differ for rules 2 to 4.
Rule 1's violation count is even when duplicates arrive in pairs, because keep=False flags both copies.

Follow-up

Rule 5 fires on 0.4 percent of rows, all from one app_version, all with reported_at days after started_at. Is that a bug, and what do you do about the daily numbers already published?
How would you turn these into a blocking check in the pipeline without failing the load every time one client version misbehaves?
Which of the five would you expect to fire on live events specifically, and how do you keep them out of the denominator?

Given a dataset, how would you optimize a query to improve retrieval speed?

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

Can you demonstrate a basic SQL query to retrieve specific data from a table?

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Daily qualified hours on local dates with trailing seven-day sums

Hard · timezones · window functions · running aggregates · late-arriving data

fct_stream has account_id, started_at (UTC), event_date (date partition key derived from started_at), played_seconds, is_qualified and play_territory (ISO-3166-1 alpha-2, resolved at play time). A lookup tz_offset(territory, utc_offset_minutes) gives each territory a fixed offset. Product wants daily qualified hours for March 2026 on each territory's local calendar date, not UTC, plus a trailing seven-day sum per territory. Offline plays upload days late with started_at backdated. Return territory, local_date, qualified_hours and trailing_7d, and state which of event_date and started_at you filter on and why.

Approach

Derive local_date = (started_at + utc_offset_minutes * INTERVAL '1 minute')::date, joining the lookup on play_territory rather than the account's signup_country. A play belongs to the clock where it happened; signup_country is a property of the account's origin and is wrong for anyone who travels or who signed up under a bundle sold in another market. Say out loud which one the requester meant.
Scan wide, then narrow. To cover local March everywhere you must read UTC 2026-02-28 through 2026-04-01, because a territory at UTC+13 begins its 1 March at 11:00 UTC on 28 February, and one at UTC-11 is still in 31 March at 10:00 UTC on 1 April. Put the widened range on event_date so the partition prune still happens, then filter local_date to March in the outer query.
State the precondition on the offset. A single fixed offset per territory is only correct while no daylight-saving transition falls in the window, and several territories shift during March. Either resolve with a real zone (started_at AT TIME ZONE an IANA name) or accept a bounded error: at most one hour of plays per affected territory moves across one date boundary, on one day of the month.
Aggregate to (territory, local_date), then take the trailing window. Prefer SUM(...) OVER (PARTITION BY territory ORDER BY local_date RANGE BETWEEN INTERVAL '6 days' PRECEDING AND CURRENT ROW), which is calendar-correct even when a territory has a day with no qualified streams. ROWS BETWEEN 6 PRECEDING is correct only on a dense date spine — on a gapped series it silently reaches back more than seven calendar days.
Say what the number means for the recent edge. event_date follows the backdated started_at, so a late offline upload lands in a day you already published and that day's figure grows after the fact. Either restate closed days on every run and publish a revision, or carry an ingestion-time column and report the figure as-of a stated load.

Worked solution (45 min)

Build a dense (territory, local_date) spine covering 2026-02-23 through 2026-03-31, so every March row has six full preceding days available to its window.
Compute local_date over a UTC scan widened to event_date BETWEEN '2026-02-28' AND '2026-04-01' (widened further for the spine's February tail) and aggregate SUM(played_seconds)/3600 where is_qualified.
LEFT JOIN the aggregate onto the spine, COALESCE hours to 0, apply the RANGE window, then restrict output to local_date in March.
Re-run the identical query a week later and diff the early-March days to measure how much the late offline uploads moved a closed number.

Expected result: 31 rows per territory covering local_date 2026-03-01 through 2026-03-31, with qualified_hours >= 0 and trailing_7d equal to the sum of that row and the six preceding spine rows — full windows throughout, because the spine starts in February.

Verify it yourself

For a territory at UTC+00 the local-date total and the UTC event_date total for March are identical; for every other territory they differ by exactly the boundary hours, which is the evidence the conversion did something.
trailing_7d on any row equals the manual sum of qualified_hours over that row and the six preceding spine rows, including rows that are zero.
Dropping the spine's February tail changes trailing_7d for the first six March days and nothing else, confirming the frame is reaching back seven calendar days rather than seven present rows.

Follow-up

A territory changes its UTC offset mid-month. What happens to your March total, and what do you print for the local day that is 23 or 25 hours long?
Would you key on play_territory or the account's signup_country? Give a concrete case where they differ and the choice changes the answer.
What do you publish for the most recent day while offline uploads are still arriving, and how do you communicate that it will move?

What drives your interest in data science, and how do you stay current with industry trends?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you prioritize your work when faced with multiple deadlines?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

If tasked with improving the efficiency of a drilling operation, what data would you analyze?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you approach a situation where data trends indicate a potential safety issue?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Describe how you would design an experiment to test a new feature in a product.

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

How would you structure a data pipeline for real-time analytics?

Medium

Approach

State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.
Clarify what is being asked and what a complete answer would contain.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Value a catalogue removal with synthetic control

Hard · synthetic control · difference-in-differences · causal inference · rights windows

A non-exclusive licence covering 1,400 content_version rows expires on a fixed date in six territories and is renewed in the other 48. Nothing else about the product changes that week. Using fct_stream, dim_content_version.available_territories and fct_subscription_period, estimate the removal's effect on qualified hours per active account-week and on 90-day voluntary churn. No randomisation is available and the six territories were chosen by the counterparty. Specify the estimator, the control group, the assumption you need and how you would test it.

Approach

Set the panel: one row per territory-week of qualified hours per active account-week, at least 24 pre-expiry weeks, with the six affected territories treated and the 48 renewed ones as the donor pool.
Reject plain difference-in-differences as the primary estimator. The six territories were selected by a counterparty, not at random, and territory-level hours differ by an order of magnitude, so an unweighted parallel-trends claim is not credible. Fit it as a secondary event study with leads and lags instead.
Fit a synthetic control per treated territory, with donor weights chosen to match the pre-period outcome path plus the predictors that matter here: plan-tier mix and the pre-period share of hours coming from the affected rights_holder_id. Aggregate the six gaps afterwards rather than pooling territories first.
Clean the donor pool before fitting. Exclude any donor where the same licence or another material rights window changes inside the event window; a contaminated donor is the failure mode that quietly halves the estimate.
Do inference by placebo-in-space: refit the synthetic control treating each donor as if it were treated, rank the true post-to-pre RMSPE ratio within that distribution, and report the exact p-value, whose floor with 48 donors is 1/49 = 0.020.
Split the answer into mechanical and behavioural. The removed versions' pre-period share of qualified hours is the gross loss, the synthetic-control gap is the net, and one minus net over gross is the substitution rate. Read churn only on monthly-billing accounts and declare the annual cohort censored, since an annual account has no renewal decision inside a 90-day window.

Worked solution (45 min)

Build the territory-week panel from fct_stream using play_territory as the unit, restricted to is_qualified = true, spanning 24 weeks before and 12 weeks after the expiry date, with the active-account denominator computed identically in every territory.
Compute gross exposure per treated territory: the share of pre-period qualified hours on content_version rows whose available_territories included that territory and whose rights_expires_at is the expiry date.
Fit the synthetic control per treated territory on the pre-period and read the post-period gap; also fit a two-way fixed-effects event study with leads from week minus 12 to minus 1 and lags to plus 12.
Run 48 placebo-in-space fits and rank the treated post-to-pre RMSPE ratios against that distribution.
Repeat the whole procedure with a placebo expiry date 12 weeks earlier, where the estimated effect must be zero.
Build the churn read separately on monthly-billing accounts with period_end_ts inside the 90-day window and renewal_outcome = 'cancelled_voluntary', computed only after the dunning horizon has closed.

Expected result: Per-territory synthetic-control gaps in qualified hours per active account-week, aggregated across the six treated territories, with placebo-in-space p-values whose floor is 1/49 = 0.020, and an event-study DiD as the secondary specification whose pre-expiry leads are jointly indistinguishable from zero. The reported pair is the net hours change and the substitution rate against the gross loss, with voluntary churn read on monthly-billing accounts only and the annual cohort declared censored.

Verify it yourself

Pre-period RMSPE must be small relative to the post-period gap. A post-to-pre RMSPE ratio below about 2 is not a result, whatever the point estimate looks like.
Donor weights should not concentrate on a single territory; a synthetic control that is 90 percent one donor is a matched-pair comparison with extra machinery.
The event-study leads from week minus 12 to minus 1 must be jointly insignificant. A visible pre-trend invalidates the DiD specification regardless of what the synthetic control reports.
The placebo expiry 12 weeks earlier must produce a gap sitting inside the placebo distribution; a significant placebo means the fit is chasing seasonality.

Follow-up

Two of the six territories account for 80 percent of the affected hours. Does aggregating the six gaps into one number still mean anything?
The pre-period fit is poor for one territory. Do you drop it, report it, or widen the donor pool, and what does each choice do to the placebo inference?
Management wants a renewal decision for next year. Which number do you hand them, and what does it assume about a different catalogue and a different competitive set?

Weekly listening hours fell; check the calendar first

Easy · seasonality · data-completeness · metric-hygiene

A dashboard reports qualified hours per active account-week — sum(played_seconds)/3600 over fct_stream rows with is_qualified, divided by distinct (account_id, ISO week) pairs having a qualified stream — down 9% against the prior week. It is being read on a Wednesday and the current week is still open. Two public holidays fell inside the prior week. Using fct_stream (started_at, event_date, played_seconds, is_qualified, account_id, play_territory), establish whether anything real changed. Deliverable: a corrected comparison plus a written rule for how this chart is read from now on.

Approach

Stop comparing an open week to a closed one. Either truncate the prior week at the same weekday the current week has reached, or exclude the open week entirely. A Wednesday read covers three of seven days and the four missing days include both weekend days.
Establish the day-of-week profile before judging any week: qualified hours by ISO weekday across the last eight weeks. A truncated week is not a scaled-down full week, it is a systematically weekday-heavy one, and the ratio distorts in a specific direction — the numerator loses the two highest-hour days while the denominator only loses accounts who stream exclusively at weekends.
Test partition completeness separately from seasonality. Recompute a day that closed two weeks ago and compare it against the value stored at the time. Offline playback uploads in bulk when the device reconnects, so a recent event_date keeps accruing for days; and if the pipeline stamps event_date at ingest rather than from started_at, those hours land on the wrong day entirely, which is a different defect with a different fix.
Separate the holiday from a trend. Cut the prior week by play_territory and split territories that observe the holidays from those that do not, then recompute the week-over-week change on the non-observing set. A holiday that lifts consumption in two large markets raises the prior week's baseline, so the current week reads as a fall with no behaviour changing anywhere.
Restate the comparison as complete-week against complete-week with the territory cut shown, and write the reading rule down: the current week is suppressed until it closes plus a stated late-arrival lag, and holiday weeks are annotated on the chart rather than explained in a thread each time.

Follow-up

The holiday raised hours but also raised the active-account denominator. Which way does that push the ratio, and is the sign consistent with what you observed?
What evidence sets the late-arrival lag, rather than picking a round number of days?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Describe a time when your analysis led to a significant business impact.

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Disagree with a product manager about a completion metric

Medium · disagreement · metric design · influence

A product manager proposes making duration-normalised completion rate the team's primary metric for the quarter, and wants a plain global unweighted version "so it's simple enough to put on a wall." You expect the unweighted version to move several points on catalogue mix alone, with no change in how satisfying anything was. You have one meeting, they own the roadmap, and you will work with them for years. Deliverable: the argument, the evidence you bring, and the fallback you accept if they still want the simple version.

Approach

The probe is whether you can lose the decision without damaging either the metric or the relationship. Bring the failure already reproduced: recompute the unweighted global rate over the last two quarters and point at the weeks it moved several points where the only input that changed was which content people played.
Decompose the variance instead of objecting. Attribute the movement in the unweighted rate to between-cell mix — (content_type, duration decile) — versus within-cell movement. A decomposition is an argument; an assertion that mix matters is a preference.
Concede the real cost honestly: the mix-weighted version is harder to explain and harder to recompute. Bring the mitigation rather than dismissing the objection — weights fixed from a stated reference month, published once, so anyone can reproduce the number without redoing the weighting.
Offer the fallback deliberately and in advance: ship both, with the unweighted rate labelled a diagnostic and the weighted one as the decision metric, and pre-agree the one observation that would mean the simple version has misled the team.
Leave the decision with them, with the consequence written down before it happens. That is what makes a later correction a shared prediction coming true rather than a retrospective argument about who was right.

Follow-up

They ship the simple version and it moves four points during a heavy release week. How do you raise it without saying you told them so?
Is there a case where the unweighted rate is genuinely the right primary metric?
Suppose you are wrong and the mix effect turns out to be small. What would you change about how you argued this?

Scope a one-line request about home row performance

Easy · scoping · stakeholder communication · metric definition

A director messages you: "Is the new home row working?" Nothing else. A new ranker_version has been serving a fraction of profiles for eleven days. You have fct_impression (surface, slate_position, ranker_version, is_exploration_slot, logging_propensity, experiment_assignment_id, was_clicked) and fct_stream (impression_id, start_source, is_qualified, played_seconds). You get a fifteen-minute call before they go into a rollout meeting. Deliverable: the three questions you ask before writing any SQL, the single primary metric you commit to with its guardrail, and the questions you tell them this data cannot answer.

Approach

The probe is whether you convert a vague request into a decision before producing a number. Ask what happens at each answer — rollback, widen, iterate — because a question whose answer changes nothing is a report request and should be scoped as one.
Pin the unit of analysis out loud. fct_impression is at (profile, slate, slot) grain and experiment_assignment_id is per assignment, so the comparison must be aggregated to the assignment unit first; comparing impression-level rates lets a change in slate length move the metric on its own.
Commit to one primary metric from the tree — qualified hours per active account-week for assigned accounts — and name the guardrail pair explicitly: share of qualified streams with start_source = 'autoplay_continuation', and median completion_ratio within content_type. A ranker can lift qualified stream counts by queueing short items that clear the 30-second threshold, and the guardrail is the only thing that catches it.
State the refusals with structural reasons, not time reasons: eleven days gives no matured cohort, so month-6 retention and net revenue per active account-month are unanswerable; and logging_propensity is populated only where is_exploration_slot = true, so the positivity condition for an off-policy estimate fails outside those slots.
Write the scope back in one paragraph — decision, metric, guardrail, the date the read becomes valid — and get it agreed in the thread before querying, so the number that arrives is the number that was asked for.

Follow-up

They reply "just give me click-through by slate position." What do you say, and what would that number actually tell them?
The eleven days include a weekend and a large release landing on day six. Does that change the metric you commit to, or only the read date?
What would have to be true for you to be willing to answer the retention question from this experiment?