Interview guides · Course Hero

Course Hero Data Scientist Interview Questions & Guide 2026

Course Hero data scientist prep on data pitfalls: selection bias, target leakage, and checking an outlier's provenance before you drop it.

The 60-second brief

A Data Scientist at Course Hero plays a pivotal role in harnessing data to drive insights that enhance the learning experiences of millions of users. This position is crucial in shaping the company’s educational products, which rely heavily on data-driven decision-making. By analyzing user behavior, optimizing content delivery, and improving product features, you will directly influence the effectiveness of Course Hero’s services.

Learn the economics of the product category before the loop. Marketplaces, subscription products and ad-supported products turn on different core quantities (match rate and liquidity, retention and churn, fill rate and yield) and fail in different characteristic ways.

Focus areas

Reconstruct active time from raw heartbeat streams
Model seat activation as the renewal leading indicator
Align cohorts to term weeks, not calendar weeks

Overview

In this role, you will engage with diverse datasets and leverage advanced analytical techniques to solve complex business problems. As a member of a cross-functional team, you will collaborate closely with product managers, engineers, and educators to develop models that not only enhance user engagement but also improve educational outcomes. The impact of your work will resonate throughout the organization, making this position both critical and rewarding.

The Course Hero interview process

Course Hero candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Phone Screen

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

Technical Interviews

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Onsite Assessment

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Pre/post gain studies that select on low pretest scores manufacture improvement.

Any measure with reliability below 1 produces regression to the mean, so a group chosen for scoring in the bottom quartile will score higher on retest with no intervention at all. The apparent gain scales with measurement error, which for a short quiz is large. The fix is a control group selected by the identical rule, or a design that models the pretest as a covariate rather than as a selection filter.

Learner-level standard errors on class-level interventions.

Anything an instructor controls, and anything deployed by school, is assigned at the section or school level, and outcomes within a section are correlated through the shared instructor, schedule, and device fleet. Treating the learner as the unit of independence understates variance by the design effect 1 + (m - 1) * rho for roughly equal cluster sizes. At a typical section size of 25 and rho of 0.15 that is a factor near 4.6 on variance, which routinely converts a null into a 'significant' result.

Ignoring interference between units in a marketplace experiment

Ask whether one unit's treatment can change another unit's outcome through shared inventory, a matching pool, a social graph or a common budget. Where it can, randomise at a level that contains the spillover, such as region or time slice, and say explicitly what that costs you in statistical power.

SQL that silently fans out on a one-to-many join

State the grain of each table and the grain you want in the result before writing the join. Pre-aggregate the many side to the join key, or use EXISTS or a window function, and verify with a row count against COUNT(DISTINCT id) rather than trusting that the numbers look plausible.

Questions & practice

Write a function that calculates the mean and standard deviation of a given list of numbers.

Medium · statistics and probability

Approach

Quantify uncertainty explicitly rather than reporting a point estimate alone.
Say what the estimate is of, and over what population it generalises.
Write down the assumption the method needs before you use the method.

Follow-up

Which assumption here is most likely to be violated in practice?
How would you explain this result to someone who does not know statistics?

How would you implement a decision tree from scratch?

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

What is overfitting, and how can you prevent it?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

Sessionise a raw heartbeat stream into activity rows

Hard · sessionisation · event streams · groupby · time series

You get a DataFrame heartbeats with columns learner_id, content_item_id, content_version_no, and ts (UTC timestamps, unsorted). Produce one row per learner per session per content item with started_at, ended_at, wall_seconds, and active_seconds. A session closes after 30 minutes with no heartbeat from that learner. active_seconds sums consecutive gaps inside a session, capping each gap at 120 seconds. Attribute each gap to the content item of the earlier heartbeat. State your rule for a gap of exactly 1800 seconds and for a session with a single heartbeat.

Approach

Sort by learner_id then ts, and compute the gap with a per-learner diff so the first heartbeat of each learner has no gap. A global diff lets one learner's last heartbeat set the next learner's first gap.
Flag a session break where gap > 1800 seconds, then cumsum that boolean within the learner to get a session index. Say whether exactly 1800 opens a new session; either choice is defensible but it must be stated because it moves the session count.
Each gap covers the interval before the current heartbeat, so attribute min(gap, 120) to the content_item_id of the previous row via shift, not the current row.
Group by (learner_id, session_index, content_item_id): sum the attributed capped gaps for active_seconds, take min and max ts for started_at and ended_at, and derive wall_seconds from those two.
Handle the degenerate group explicitly. A single heartbeat gives active_seconds 0 and wall_seconds 0. Dropping those rows quietly removes real opens and biases any completion or abandonment rate computed later.
Validate that within every session the summed active_seconds never exceeds the session's wall_seconds, which catches attribution and cap errors in one assertion.

Worked solution (35 min)

Sort by (learner_id, ts) and compute gap = ts.diff().dt.total_seconds(), then null the gap on the first row of each learner using a learner-change mask.
new_session = gap.isna() | (gap > 1800); session_index = new_session.groupby(learner_id).cumsum().
contribution = gap.clip(upper=120).fillna(0), attributed to the previous row's content_item_id using shift within the learner and session.
Aggregate by (learner_id, session_index, content_item_id) with sum for active_seconds and min and max of ts for the timestamps.
Assert per session that sum(active_seconds) <= (max(ts) - min(ts)) in seconds, and print the count of single-heartbeat groups.

Expected result: A frame keyed by (learner_id, session_index, content_item_id) where active_seconds equals the sum of min(gap, 120) over consecutive heartbeats with the first heartbeat contributing zero, wall_seconds equals ended_at minus started_at, and sum(active_seconds) <= wall_seconds holds for every session.

Verify it yourself

Heartbeats at 0, 60, and 120 seconds on one item give one session with wall_seconds 120 and active_seconds 120.
Heartbeats at 0 and 3600 seconds give two sessions, each with wall_seconds 0 and active_seconds 0.
Inserting a 300-second gap inside a session adds exactly 120 to active_seconds, not 300, while wall_seconds rises by the full 300.

Follow-up

The client emits a heartbeat every 60 seconds on desktop and every 15 seconds on a managed device fleet. What does the 120-second cap do to the comparison between those two populations?
Your reconstructed active_seconds disagrees with the stored column by 4 percent overall. How do you localise the disagreement rather than argue about the total?
A learner has two tabs open on different items at once. What does your output claim, and what should it claim?

Explain your approach to optimizing a query in SQL.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Seat activation rate by institution within thirty days

Easy · activation · null handling · aggregation

You have dim_learner(learner_id, org_id, created_at, first_activity_at, current_status) and fct_subscription_period(subscription_period_id, account_id, org_id, seats_purchased, seats_provisioned, period_start, period_end, status), plus dim_term(term_id, org_id, term_start_date). For each org and term, return the share of provisioned seats whose learner's first activity falls inside the 30 days beginning at term_start_date, meaning first_activity_at >= term_start_date AND first_activity_at < term_start_date + 30 days. The denominator is seats_provisioned on the subscription period covering term_start_date, not a count of learner rows. first_activity_at is NULL for a seat that was created but never used, and dim_learner stores one first-ever activity per learner, so this measures new-seat activation inside a rollout window and not per-term usage. Report orgs where seats_provisioned exceeds seats_purchased rather than clipping them.

Approach

Pick the covering subscription period with term_start_date BETWEEN period_start AND period_end. If overlapping renewal rows both match, choose a deterministic tie-break such as the latest period_start and say so, because the denominator is otherwise ambiguous.
Write the window closed at both ends: COUNT(DISTINCT learner_id) where dim_learner.org_id matches and first_activity_at >= term_start_date AND first_activity_at < term_start_date + INTERVAL '30 days'. The lower bound is the load-bearing half. With only the upper bound, every learner who activated in any earlier term still satisfies the predicate, so each term inherits all of its predecessors' activations, the rate climbs past 1 on orgs that renew, and the movement has nothing to do with the rollout being measured.
The NULL is the signal for a dead seat. A NULL first_activity_at fails both bounds on its own, so no explicit IS NOT NULL is needed, but the predicate must live in the numerator aggregate and never in a WHERE clause that also feeds the denominator: an unused seat still occupies a provisioned seat.
Take the denominator from seats_provisioned on the period. Counting dim_learner rows instead removes never-activated seats from both sides of the fraction and pushes the rate toward 1.
Mind the types. term_start_date is a DATE and first_activity_at is a timestamp, so the window opens at midnight of term_start_date in whatever zone the timestamps carry and closes at midnight 30 days later. State the zone you are assuming rather than letting the cast pick one.
LEFT JOIN the learner aggregate onto the period so an org with zero activations returns 0 rather than disappearing from the result set, and emit an over_provisioned flag where seats_provisioned > seats_purchased instead of capping the denominator. Over-provisioning is a contract fact and hiding it makes activation look worse than the contract says.

Worked solution (20 min)

Write the period selection CTE first and count how many (org_id, term_id) pairs match more than one period. Fix the tie-break before going further.
Write the activation aggregate as its own CTE keyed on (org_id, term_id) with both bounds on first_activity_at, so the window is visibly separate from the denominator.
Self-join dim_term on org_id and count pairs of terms whose term_start_date values sit within 30 days of each other. That count is exactly how far SUM(activated_seats) can exceed the distinct learner count, and on most calendars it is zero.
LEFT JOIN aggregate onto period, compute activated_seats::numeric / NULLIF(seats_provisioned, 0).
Add the over_provisioned flag and a count of orgs where activation_rate exceeds 1.

Expected result: One row per (org_id, term_id) carrying activated_seats, seats_provisioned, activation_rate, and an over_provisioned boolean. Orgs with no activated learners appear with activated_seats = 0 and a rate of 0, not as missing rows. Learners whose first activity falls outside every term window contribute to no numerator while still occupying a provisioned seat.

Verify it yourself

SUM(activated_seats) across the result equals the number of (learner_id, term_id) pairs satisfying the two-sided predicate. It equals COUNT(DISTINCT learner_id) only when no org has two terms starting within 30 days of each other, so run the dim_term self-join before quoting the simpler equality.
No learner contributes to two rows of the same org unless that org's windows overlap. Drop the lower bound and this check fails on every org with more than one term, which is the fastest way to catch the unbounded predicate.
activation_rate is NULL only where seats_provisioned = 0.
Any activation_rate above 1 means seats_provisioned understates the learners dim_learner assigns to that org; count those orgs and report them rather than clipping the value.

Follow-up

dim_learner stores one first-ever activity, so an org with a stable roster shows activation falling every term as returning learners are already spent. What would you add to the model to measure per-term activation, and which of the two does an administrator actually want?
seats_provisioned can also trail the number of accounts on the license. What does your rate do then, and what do you report instead of a number above 1?
An administrator wants this weekly during the first month of term. What does the metric look like on day 3, and how do you present a partially matured cohort?

How do you prioritize tasks when faced with multiple deadlines?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

Given a dataset, explain how you would identify outliers and their potential impact on the analysis.

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

Discuss a method you would use to segment users based on their behavior.

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you approach predicting user engagement for a new feature?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

Say what you would check first and why it is the highest-information step.
Clarify what is being asked and what a complete answer would contain.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Estimate a district-wide rollout effect without any randomised holdout

Hard · difference-in-differences · synthetic control · parallel trends

One institutional account with 210 sections switched on auto-assigned remediation for every section on a single date mid-term. No holdout was permitted. You have 46 comparable accounts that did not switch it on, eight pre-period term-weeks, and dim_term to align calendars that differ across accounts. Estimate the effect on pace adherence at term-week 6 and on verified mastery rate. State your identifying assumption, your inference method given a single treated cluster, and the smallest p-value this design can produce.

Approach

Align every account on term-week through dim_term before differencing anything. Accounts run different academic calendars, so a calendar-date difference-in-differences compares the treated account's term-week 3 to a comparison account's exam week and reports the calendar gap as the effect.
Freeze the learner panel before treatment. Keep enrollments that existed before the switch date so the estimate is not moved by roster_sync creating or deactivating accounts in response to the rollout; composition change after treatment is post-treatment selection and will bias the estimate in an unknown direction.
Test parallel pre-trends as an event study with eight leads over term-weeks -8 to -1, not as a single pre-mean comparison. A single pre-mean can match while the trends diverge, which is exactly the failure mode the assumption is about.
With one treated unit, prefer synthetic control to a two-group difference-in-differences: choose non-negative donor weights summing to one that match the pre-period outcome path plus grade_band mix, seat activation rate and device_type mix, then read the post-period gap. Report the pre-period RMSPE so the reader can judge whether the fit earns the estimate.
Do inference by permutation, not by cluster-robust standard errors, which are not usable with a single treated cluster. Refit the synthetic control treating each of the 46 donors as if treated, rank the real unit's post-to-pre RMSPE ratio among them, and report that permutation p-value; its floor is 1 / (46 + 1) = 0.021.
Run placebo-in-time at a fake switch date four term-weeks early and a leave-one-donor-out sweep before writing any number into a deck.

Worked solution (40 min)

Build the term-week index for all 47 accounts from dim_term and reshape both outcome panels onto term-week.
Restrict to enrollments existing before the switch date and record how many post-period learners that excludes.
Fit the event study with eight leads and the post lags, and inspect the lead coefficients individually and jointly.
Fit the synthetic control on term-weeks -8 to -1, reporting donor weights and pre-period RMSPE.
Run 46 placebo-in-space refits and rank the treated unit's post-to-pre RMSPE ratio among them.
Run the placebo-in-time refit at a fake switch date four term-weeks early and confirm it produces no gap.

Expected result: A term-week-aligned synthetic control estimate of the post-switch gap in pace adherence at term-week 6 and in verified mastery rate, reported with donor weights, pre-period RMSPE, an event study showing flat leads, and a permutation p-value that cannot fall below 1 / 47 = 0.021 regardless of how large the gap is.

Verify it yourself

Donor weights must be non-negative and sum to one. A negative weight means you fitted an unconstrained regression, not a synthetic control.
The eight lead coefficients should be individually and jointly indistinguishable from zero, or parallel trends is not supported and the estimate should not be reported as causal.
The placebo-in-time refit must produce a gap near zero. A gap there means the design is picking up something other than the switch.

Follow-up

Pre-period RMSPE is high and the synthetic path never tracks the treated account. What do you report, and what do you refuse to report?
The account also renegotiated its contract that month. How does that change what your estimate is an estimate of?
If you could have asked for an encouragement design instead, what would the instrument be, what would it estimate, and what assumption would replace parallel trends?

First-attempt accuracy rose four points after a content release

Medium · psychometrics · mix-shift · adaptive-selection

First-attempt accuracy rose from 61 percent to 65 percent over the 28 days following a content release. You have fct_assessment_response (learner_id, content_item_id, content_version_no, objective_id, attempt_no, is_correct, submitted_at) and dim_content_item (content_item_id, version_no, irt_b, irt_a, calibration_n, status, published_at), plus a flag for whether each item was served by the adaptive selector. Determine how much of the four points, if any, is a change in learner performance. Deliver a corrected number with its restriction stated.

Approach

Restrict to attempt_no = 1 and to (content_item_id, version_no) pairs that appear in both windows unchanged, then recompute the delta on that matched set. Editing an item changes what its historical responses mean, which is why the metric definition carries the version restriction rather than joining on content_item_id alone.
Decompose the aggregate move rather than eyeballing it: with per-item accuracy a and response share w, write the delta as sum of w_pre * (a_post - a_pre) for the within-item term plus sum of (w_post - w_pre) * a_pre for the mix term, assigning the interaction explicitly so the two terms sum to the raw delta.
Compare the response-weighted mean irt_b across the two windows, restricted to items with calibration_n at or above a stated floor. Items republished at a new version_no have irt_b NULL until they recalibrate, so they vanish from a difficulty-weighted mean and must be counted as their own bucket.
Separate adaptive from fixed-form traffic and report them apart. A selector targeting a fixed success probability holds observed accuracy near that target by construction, so an adaptive accuracy series moves when the selector gets more conservative, not when learners improve.
Publish only the fixed-form, version-stable number as the performance claim, and state the share of total responses it covers so the reader knows how much of the product it speaks for.

Follow-up

Design a fixed-form measurement set that runs alongside the adaptive experience purely to produce a comparable accuracy series. What does it cost in learner time, and how often would you refresh it?
How many responses do you want behind an item's irt_b before you will use that item in a difficulty-weighted comparison?
An author rewrites one distractor in a multiple-choice item. Should version_no increment, and what breaks downstream if it does not?

Suggested practice checklist

Roughly 90 minutes a night on weekdays with one longer weekend block. The plan deliberately cuts scope rather than compressing everything, on the assumption that finishing one thing a night beats half-starting four.

Day 1 — Fix the scope and set a baseline

Read the role description and write the three things the loop will almost certainly test, then write an explicit not-doing list for everything else and keep it visible all week.
Take one 20-minute SQL prompt and one 10-minute metric question cold, and write the single sentence that says what blocked each attempt, since that sentence is what decides which two topics get the most evenings.
Set the week's one rule: one problem finished to completion every night, including the night you only have 40 minutes.

Deliverable: A one-page scope with an explicit not-doing list and two cold attempts, each carrying one sentence on what blocked it.

Day 2 — One query pattern, written three times

Choose the single pattern most likely to appear (a cohort retention grid, or a funnel counted by user) and write it three times from a blank file rather than editing the previous attempt.
On the third attempt, write the grain of every CTE as a comment before writing its body.
Stop at 90 minutes even if the third version is imperfect, and write the one thing you would fix with another hour.

Deliverable: Three independent versions of the same query plus a note on what changed between them.

Day 3 — Only the statistics you will be asked to defend

Write, in under 200 words, how you would decide whether a difference between two groups is real: the test, its assumptions, and what you would switch to when an assumption fails.
Compute a 95 percent confidence interval for a difference in proportions by hand on realistic numbers, then write in one sentence what changes if the two samples are paired rather than independent.
Write your answer to "what does a p-value mean", check it against a definition, and delete the version that describes it as the probability the hypothesis is true.

Deliverable: A 200-word written answer and one hand-computed interval you can reproduce under pressure.

Day 4 — One case, and the assumptions holding it up

Answer one product case aloud in 20 minutes with a recording running, then listen back with a pen and mark every claim you asserted without saying what it rested on: an assumed user behaviour, an assumed data source, an assumed baseline rate, an assumed grain.
Pick the three assumptions the recommendation actually depends on, write how you would check each one against data, and say which one being wrong would flip the recommendation rather than merely weaken it.
Write the four-step structure you used onto a card small enough to hold in working memory when you are nervous.

Deliverable: One recording, three load-bearing assumptions each with a written check, and a four-step structure card.

Day 5 — Your own work, timed

Write a 90-second version and a four-minute version of your main project, and time both out loud rather than reading them.
Prepare answers to the two follow-ups that always come: what you would do differently, and how you knew it worked.
Put one number in the first sentence and be able to say exactly where that number came from and what it excludes.

Deliverable: Two timed narratives with one defensible number in the opening line.

Day 6 — The one full rehearsal, in a longer weekend block

Run a 60-minute mock covering query work, a case and a behavioural question in a single sitting with no breaks, because sustained attention is the thing evenings have not trained.
Immediately afterwards, and before hearing any feedback, write the three moments you lost the thread.
Spend the rest of the block only on those three moments, and on nothing you merely feel shaky about.

Deliverable: Mock notes naming three failure moments with a specific fix written under each.

Day 7 — Taper

Write the 20-minute warm-up you will actually do on the morning of the interview: one query you can already write from a blank file, one metric you can define out loud, and nothing you have never seen before.
Re-read only your own notes from this week, and open no new material.
Write down the logistics: the tool you will be asked to work in, whether lookups are allowed, and the sentence you will use when you do not know something.

Deliverable: A one-page card holding the case structure, the project numbers, and the logistics.

Being wrong in public

A number you shipped turned out to be wrong, and someone had already acted on it. That is one of the most useful stories a data person can carry. What is being scored is how fast you noticed, who you told first, and what you changed in the process so the same class of error could not repeat quietly.

Give an example of how you communicated complex data findings to a non-technical audience.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Quantify the outcome, including what you would not claim credit for.
Close with what you would do differently, concretely.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Defend a null result against an already-announced launch

Medium · null results · experiment design · stakeholder communication

A section-randomised test of a new practice-sequencing feature ran across 62 class sections in 11 orgs. Primary metric: verified mastery events per active learner. With section-clustered standard errors the effect is +1.8% (95% CI -4.1% to +7.9%). Leadership has already told two district administrators that the feature improves outcomes. You have ten minutes in a review. Deliverable: what you say, what you put on one slide, and what you propose next. Probed: whether you can hold a null under commercial pressure without either caving or lecturing the room on statistics.

Approach

Lead with the decision, not the p-value: state the interval in the unit the audience already uses (mastery events per active learner per week) and say plainly what the study could and could not have detected.
Separate the interval's half-width from the minimum detectable effect before you put either on a slide. The stated 95% CI of -4.1 to +7.9 has a half-width of 6.0 points, so the clustered standard error is 6.0 / 1.96 = 3.06 points; the MDE at 80% power and two-sided alpha = 0.05 is (1.96 + 0.8416) * 3.06 = about 8.6 points, roughly 1.43 times the half-width. Quote 8.6, and show that it follows from 62 sections at the section size and intraclass correlation you assumed via the design effect 1 + (m - 1) * rho, so that 'no effect found' is visibly separated from 'too small to see'.
Check the pre-registered guardrails before conceding anything: median active_seconds per mastery event and the calibrated irt_b distribution of the items backing mastery decisions. If both are flat, the null is about learning, not about a broken pipeline.
Handle the promise made to the two districts as its own item with a named owner, rather than leaving the correction implicit in a statistics discussion.
Close with the cheapest decision-grade next step: extend to enough additional already-instrumented sections to reach the effect size leadership actually cares about, with the stopping rule fixed before the extension starts.

Follow-up

If the interval had been -0.5% to +8.0%, would you support launch, and what exactly changes in your reasoning?
Leadership asks you to re-run with learner-level standard errors since it is the same data. What do you say?
What would you have set up at design time so that this meeting was easier?

State the impact of your last project without inflating it

Hard · impact measurement · attribution · causal reasoning

Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.

Approach

Name the metric by its definition rather than its label. 'Verified mastery rate' means nothing without its numerator, denominator, window, and the reporting lag the delayed retention check forces.
Separate the three claims usually merged into one: the metric moved, your work moved it, and the movement was worth what it cost.
State the counterfactual explicitly and say what produced it: a holdout, matched sections, or a pre-period trend. If it was a before-and-after with no control, say that in those words.
List the co-occurring changes with dates: a term boundary, a roster sync at scale, a pricing change, another team shipping into the same surface. In this domain the academic calendar alone routinely dwarfs a product effect.
Convert the shortfall into a specific design artefact: the holdout you would have reserved, the instrumentation you would have shipped before the change, the analysis plan you would have written first.

Follow-up

What fraction of the observed movement would you defend under cross-examination, and on what evidence?
Your project shipped in term-week one. How does that change what you can claim?
If the effect is genuinely not separable, was the project worth doing?