Interview guides · Amplify

Amplify Data Scientist Interview Questions & Guide 2026

Causal inference drills for Amplify data scientist prep: diff-in-diff, matching, instruments, and the limits of observed data.

The 60-second brief

As a Data Scientist at Amplify, you are stepping into a highly impactful role at a pioneering K–12 education company that serves over 15 million students across all 50 states. Your work directly influences how the company scales its operations, distributes its curriculum, and plans for future growth. Rather than working in an isolated research environment, you will be deeply embedded in core business functions like Sales Analytics and Supply Chain Analytics.

How much statistics you need depends on the flavour of the seat. Experiment-facing work wants you deep enough to notice that repeated looks at accumulating data inflate the false positive rate of a fixed-sample test; modelling-facing work wants estimation and honest uncertainty intervals.

Focus areas

Align cohorts to term weeks, not calendar weeks
Separate mastery signals from raw usage exposure
Read item difficulty and discrimination before trusting accuracy

Overview

This role is critical because it bridges the gap between complex data and strategic business decisions. You will be responsible for building state-of-the-art forecasting models that predict demand, optimize inventory, and drive revenue growth across a massive educational product portfolio encompassing tens of thousands of ISBNs. The scale and complexity of the data require a rigorous, analytical mindset combined with a strong sense of business acumen.

You can expect to work with a highly cross-functional scrum team of Analytics Engineers, Data Analysts, and business stakeholders. Whether you are developing an LSTM model to predict long-term sales trends or using driver decomposition to explain a sudden shift in supply chain demand, your insights will empower real-time decision-making. At, you are not just building models; you are championing a data-driven culture that ultimately supports educators and inspires students to think deeply and creatively.

The Amplify interview process

Amplify candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Hiring Manager Interview

Much of this round runs on your own history, but the manager is not collecting a project list. They are working out what it is like when something goes wrong on your watch: how late the bad news tends to arrive, and whether a number you hand over has been checked by anyone including you. That is why the strongest material is a project where you can describe the part that did not work and what it cost. A result you cannot take full responsibility for, however clean, gives them nothing to trust you with afterwards.

What to demonstrate

Whether you volunteer the limits of a result you are proud of, or wait to be pushed onto them
How errors surfaced in your past work, and whether you or somebody else found them
Whether the scope you claim matches the level of detail you can still produce about it
What you did the first time a stakeholder acted on something of yours that turned out to be wrong

How to prepare

Rebuild one headline figure from memory down to the join and the filter, so a question about the denominator does not stall the conversation
For each project you raise, write the sentence you would say to someone who had already acted on a number that later turned out wrong
Mark which parts of a project were yours and which belonged to other people, and state that boundary yourself before anyone asks

Technical Screen

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

System Design Interview

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Onsite/Virtual Panel

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Learner-level standard errors on class-level interventions.

Anything an instructor controls, and anything deployed by school, is assigned at the section or school level, and outcomes within a section are correlated through the shared instructor, schedule, and device fleet. Treating the learner as the unit of independence understates variance by the design effect 1 + (m - 1) * rho for roughly equal cluster sizes. At a typical section size of 25 and rho of 0.15 that is a factor near 4.6 on variance, which routinely converts a null into a 'significant' result.

Consent and age rules silently truncate the data, not just the joins.

Where age_gated is TRUE, behavioural logging and cross-system joins are restricted, so those learners are missing from the very tables used to compute engagement. An analysis that simply inner-joins will produce a population skewed to older learners and self-serve accounts while appearing complete. Check the age_gated share of every population you report on, and state it.

Building features from data that postdates the prediction time

Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.

Answering a product-sense question with a list of features

Answer with a decision and the measurement that would settle it: the hypothesis, the primary metric, the guardrails, and the result that would make you not ship. A feature brainstorm cannot be wrong, which is exactly why it earns no points.

Questions & practice

What metrics do you use to evaluate a demand forecasting model, and why?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Say how the offline result would be validated online before it is trusted.
Set a baseline first, so any model has something honest to beat.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Explain how you would use Prophet to model holiday effects on educational product sales.

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Sessionise a raw heartbeat stream into activity rows

Hard · sessionisation · event streams · groupby · time series

You get a DataFrame heartbeats with columns learner_id, content_item_id, content_version_no, and ts (UTC timestamps, unsorted). Produce one row per learner per session per content item with started_at, ended_at, wall_seconds, and active_seconds. A session closes after 30 minutes with no heartbeat from that learner. active_seconds sums consecutive gaps inside a session, capping each gap at 120 seconds. Attribute each gap to the content item of the earlier heartbeat. State your rule for a gap of exactly 1800 seconds and for a session with a single heartbeat.

Approach

Sort by learner_id then ts, and compute the gap with a per-learner diff so the first heartbeat of each learner has no gap. A global diff lets one learner's last heartbeat set the next learner's first gap.
Flag a session break where gap > 1800 seconds, then cumsum that boolean within the learner to get a session index. Say whether exactly 1800 opens a new session; either choice is defensible but it must be stated because it moves the session count.
Each gap covers the interval before the current heartbeat, so attribute min(gap, 120) to the content_item_id of the previous row via shift, not the current row.
Group by (learner_id, session_index, content_item_id): sum the attributed capped gaps for active_seconds, take min and max ts for started_at and ended_at, and derive wall_seconds from those two.
Handle the degenerate group explicitly. A single heartbeat gives active_seconds 0 and wall_seconds 0. Dropping those rows quietly removes real opens and biases any completion or abandonment rate computed later.
Validate that within every session the summed active_seconds never exceeds the session's wall_seconds, which catches attribution and cap errors in one assertion.

Worked solution (35 min)

Sort by (learner_id, ts) and compute gap = ts.diff().dt.total_seconds(), then null the gap on the first row of each learner using a learner-change mask.
new_session = gap.isna() | (gap > 1800); session_index = new_session.groupby(learner_id).cumsum().
contribution = gap.clip(upper=120).fillna(0), attributed to the previous row's content_item_id using shift within the learner and session.
Aggregate by (learner_id, session_index, content_item_id) with sum for active_seconds and min and max of ts for the timestamps.
Assert per session that sum(active_seconds) <= (max(ts) - min(ts)) in seconds, and print the count of single-heartbeat groups.

Expected result: A frame keyed by (learner_id, session_index, content_item_id) where active_seconds equals the sum of min(gap, 120) over consecutive heartbeats with the first heartbeat contributing zero, wall_seconds equals ended_at minus started_at, and sum(active_seconds) <= wall_seconds holds for every session.

Verify it yourself

Heartbeats at 0, 60, and 120 seconds on one item give one session with wall_seconds 120 and active_seconds 120.
Heartbeats at 0 and 3600 seconds give two sessions, each with wall_seconds 0 and active_seconds 0.
Inserting a 300-second gap inside a session adds exactly 120 to active_seconds, not 300, while wall_seconds rises by the full 300.

Follow-up

The client emits a heartbeat every 60 seconds on desktop and every 15 seconds on a managed device fleet. What does the 120-second cap do to the comparison between those two populations?
Your reconstructed active_seconds disagrees with the stored column by 4 percent overall. How do you localise the disagreement rather than argue about the total?
A learner has two tabs open on different items at once. What does your output claim, and what should it claim?

How do you optimize a SQL query that is joining two massive tables and running too slowly?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Say which table is the grain you start from, and join outward from it.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Walk me through how you use Pandas to handle missing values and outliers in a time series dataset.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Given a table of daily inventory levels, write a query to identify periods where an item was out of stock for more than 3 consecutive days.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
State the window function and its partition and ordering out loud before writing it.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Daily scored submissions under rubric scoring lag

Easy · date boundaries · null semantics · reporting lag

fct_assessment_response(response_id, learner_id, content_item_id, content_version_no, submitted_at, is_correct, score_points, max_points, scoring_mode, scored_at) stores both timestamps in UTC. scored_at is NULL until a rubric item is graded and lags submitted_at by days when scoring_mode = 'rubric_human'. Produce two dense daily series over the last 90 days: submissions keyed on submission date, and completed scorings keyed on scored date. Then state the reporting lag after which each series stops moving, computed from the data rather than asserted, and say what biases your lag estimate.

Approach

Key the two series on different columns on purpose: date(submitted_at) for volume, date(scored_at) for graded throughput, with scored_at IS NOT NULL on the second. They answer different questions and are not two versions of one number.
Define scored as scored_at IS NOT NULL rather than is_correct IS NOT NULL. Check first whether rubric rows carry score_points with is_correct left NULL after grading; if they do, the is_correct test undercounts by exactly the rubric share.
Estimate lag empirically with percentiles of scored_at - submitted_at grouped by scoring_mode. This is computed only over rows already scored, so it is survivorship-biased downward. Report alongside it the share of submissions in each age bucket still unscored.
Generate a dense date spine and LEFT JOIN both series onto it so a zero-volume day is a 0, not an absent row that a chart will interpolate across.
Name the timezone problem instead of guessing: these tables carry a locale on the learner but no timezone, so a local-calendar day series is not computable from this schema. Say that rather than pretending date(submitted_at) is an institution's day.

Worked solution (15 min)

Build the date spine for the 90-day window, then the two aggregates, then join.
Run a distribution of is_correct for rows with scored_at IS NOT NULL to decide which column defines scored.
Compute median and p95 of scored_at - submitted_at per scoring_mode, and the unscored share by 1, 3, 7 and 14 day age buckets.
Write one sentence stating the reporting lag you are imposing and what it costs.

Expected result: Two gap-free 90-row daily series, submitted_count and scored_count, plus a lag table per scoring_mode with median lag, p95 lag, and the still-unscored share by age bucket.

Verify it yourself

SUM(submitted_count) over the window equals COUNT(*) of responses whose submitted_at falls in the window.
For a window ending today, SUM(scored_count) is at or below SUM(submitted_count), and the gap is concentrated in the final days rather than spread evenly.
Rows with scoring_mode = 'auto' should show a lag near zero. A large auto lag means scored_at is written by a batch job, and the whole lag model needs rebuilding around that job's schedule.

Follow-up

The last five days of the submitted series will be revised upward as late rows land. How do you present a series whose recent points are known to be incomplete?
Rubric grading is done by people with weekly schedules. What does that do to a daily scored series, and would an ISO week grain be more honest?
If you had to pick a single cut-off date after which the data is considered final, how would you derive it and what percentage of eventual scorings would you be discarding?

How do you prioritize tasks when you are receiving urgent requests from both the Sales and Supply Chain teams?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Metrics that survive an adaptive item selector

Medium · adaptive selection · irt · measurement design

Content selection targets a 70% success probability per learner. First-attempt accuracy reads 69.8% before a model change and 70.3% after. Using dim_content_item (irt_a, irt_b, calibration_n, version_no, status) and fct_assessment_response (attempt_no, is_correct, objective_id), propose a measurement design that can detect a real learning change under this selector. Specify the fixed-form probe set, its size and calibration requirement, the statistic you report, and say why the adaptive accuracy series carries no information about learning.

Approach

Show why the series is pinned. Under a 2PL item, P(correct) = 1 / (1 + exp(-a(theta - b))). A selector targeting 0.70 solves for b given theta: a(theta - b) = ln(0.7/0.3) = 0.847, so b = theta - 0.847/a. As theta rises the selector raises b with it and measured accuracy stays at the target.
Demote accuracy accordingly: it measures how well the selector hit its own target, so keep it as a selector-calibration guardrail and never as an outcome.
Build a fixed-form probe: 10 to 15 items with frozen (content_item_id, version_no), status='published', calibration_n at least 200, spanning irt_b roughly -1.5 to +1.5, served on a fixed schedule identically to every arm and never chosen adaptively.
Report a theta estimate or mean probe score. For 2PL, test information is I(theta) = sum of a_i^2 P_i Q_i, so 12 items with a = 1.0 measured near their peak give I = 3 and SE = 1/sqrt(3) = 0.58 logits per learner. Individual estimates are noisy, but a 400-learner group mean has SE near 0.029 logits, which is where the comparison lives.
Protect the probe: exclude its items from the adaptive pool so practice cannot leak into the measurement, and freeze versions, because editing an item changes the meaning of every historical response to it.
Add the difficulty audit as a second guardrail: mean irt_b of the items backing mastery decisions, which exposes a selector that simply got easier.

Worked solution (30 min)

Derive the selector identity b = theta - ln(0.7/0.3)/a = theta - 0.847/a, and evaluate it at a = 1.2, where the served item sits 0.71 logits below the learner's theta.
Specify the probe: 12 frozen item versions, calibration_n >= 200 each, irt_b spread across -1.5 to +1.5, administered every four weeks to all arms.
Compute probe precision: I = sum a_i^2 P_i Q_i = 12 x 1.0 x 0.25 = 3, SE = 0.58 logits per learner, SE of a 400-learner mean = 0.58/20 = 0.029 logits.
Write the reporting rule: probe mean as the outcome, adaptive accuracy and mean irt_b of mastery-backing items as guardrails.

Expected result: A 10 to 15 item fixed-form probe on frozen versions with calibration_n >= 200, excluded from the adaptive pool, reporting a group mean theta or probe score with SE derived from test information; adaptive first-attempt accuracy demoted to a selector-calibration guardrail alongside mean irt_b, with 69.8% versus 70.3% declared uninformative about learning.

Verify it yourself

b = theta - ln(0.7/0.3)/a = theta - 0.847/a, and at a = 1.2 the offset is 0.71 logits.
Probe items are frozen at one version_no and removed from adaptive selection.
Per-learner SE from 12 items at a = 1.0 is about 0.58 logits, and you did not present individual theta as precise.

Follow-up

How many probe items would you need before reporting an individual learner's theta rather than a group mean?
What happens to the probe if the selector begins serving items that share a template with probe items?
The probe costs learner time every cycle. How do you justify that cost to a curriculum owner?

Pace adherence at term-week six rose eleven points

Medium · definition-change · survivorship · pacing

Pace adherence at term-week 6 rose from 48 percent to 59 percent between two consecutive terms, and the dashboard query changed owners last month. Tables: fct_enrollment (enrollment_id, learner_id, course_id, units_completed, units_total, status, scheduled_start_date, due_date, enrolled_at) and dim_term (term_id, term_start, term_length_weeks). Nothing shipped to pacing in that period. Decide whether learners are pacing better, and reconcile the old and new numbers to the point.

Approach

Diff the definitions before touching the data. Recover the prior query and compare it line by line against the current one on four axes: the denominator population, the status filter, how term-week 6 is derived, and what happens to enrollments with NULL scheduled_start_date.
Recompute both definitions on the same term and the same enrollment set. If the difference does not account for the full eleven points there is a second cause, and knowing that before you start hunting saves the hunt.
Test the survivorship mechanism directly: count enrollments with status in ('dropped','expired') at the end of term-week 6 and compute their units_completed over units_total. If that ratio sits well below the active median, removing them from the denominator lifts the rate mechanically and the lift is arithmetic, not behaviour.
Verify the numerator independently, because units_completed is maintained incrementally rather than derived on read. Recount it from lesson activity for a sample of a couple of hundred enrollments and compare against the stored column to rule out a backfill as a second mover.
Confirm term-week 6 maps to the intended dates in both terms through dim_term rather than calendar week, since an edited term_start silently shifts the evaluation window and can move the number on its own.

Follow-up

The old definition penalises a course for enrollments that dropped in week two. Is that the behaviour you actually want from a pacing metric, and who is the metric for?
How do you treat enrollments with NULL scheduled_start_date, and what does each choice do to the reported number?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

Where your data came from

Data people depend on systems owned by other teams, and much of the job is negotiating for instrumentation, access, or a fix to a broken pipeline. Prepare an example of getting something changed upstream that you did not control. Describe what you asked for, what you traded, and how you worked while you waited.

How do you handle a dataset where the time series is non-stationary?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What would you do differently if you ran that project again?
What did you decide not to do, and why?

Describe a situation where your data analysis contradicted the intuition of business leaders. How did you handle it?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that project again?
What did you decide not to do, and why?

Explain a risk score to a non-technical sales executive

Easy · communicating uncertainty · renewal risk · stakeholder communication

Your renewal-risk score joins fct_subscription_period to seat activation from dim_learner.first_activity_at and to pace adherence from fct_enrollment. For one institution with 900 seats_provisioned and 31% activation at term-week 6, it outputs a 0.62 probability of non-renewal. A sales VP asks whether you are losing the account or not. You have two minutes and no slides. Deliverable: the spoken answer, plus the one line you would put in the weekly account review so the number is not later quoted as a certainty. Probed: whether you can carry uncertainty and still give a decision.

Approach

Answer the question that was asked before qualifying it. 'Treat this as at-risk and act this week' is the answer; the probability is the support for it.
Translate the score into a frequency on a reference class the VP already trusts, such as how many of the last fifty accounts scored in the 0.55-0.70 band did not renew, instead of explaining calibration in the abstract.
Name the driver that is still movable before period_end: 31% seat activation at term-week 6 can be changed this term, whereas last term's usage cannot, so that is where the conversation goes.
State what would move the score and by when, so the number arrives attached to an action rather than as a verdict.
For the written line, use a band and a review date rather than a bare decimal, because a single number in a document will be re-quoted without its interval.

Follow-up

The VP asks for every account scored above 0.50. What do you tell them about where that threshold came from?
How would you check whether 0.62 is calibrated at all, and on what sample?