Interview guides · Khan Academy

Khan Academy Data Scientist Interview Questions & Guide 2026

Data scientist prep for Khan Academy: practice defining a education technology metric, picking its guardrails and defending the tradeoff.

The 60-second brief

As a Data Scientist at Khan Academy, you play a foundational role in fulfilling the organization's mission to provide a free, world-class education for anyone, anywhere. Operating at the intersection of product development, learning science, and data engineering, you empower cross-functional teams to make evidence-based decisions that directly impact over 181 million registered learners globally. Your day-to-day work involves transforming massive streams of educational and behavioral data into actionable insights, driving product strategies that optimize student engagement and improve learning outcomes.

Ask early whether the loop includes an asynchronous take-home or a timed live case, because the two are graded on different things. A take-home is read as an artifact: the question you decided to answer, what you did about missing or malformed records, and a conclusion stated plainly enough for someone to act on. A reviewer who cannot rerun your notebook discounts the result whatever score is printed in it. Hold to the stated time box and write down what you would have done with more of it, since the follow-up round is usually a live defence of the same work.

Focus areas

Cluster experiments at class or school level
Separate mastery signals from raw usage exposure
Read item difficulty and discrimination before trusting accuracy

Overview

The scope of this role extends across multiple high-impact product areas, including the core learner experience, content recommendation engines, and teacher-parent tools. You will partner closely with product managers, engineers, and learning scientists to design theories of action, instrument robust event tracking, and quantify the real-world impact of new features. Because Khan Academy operates as a mission-driven nonprofit, your work carries a unique weight: every metric drop you diagnose and every successful experiment you analyze directly translates to better educational access and equity for students in historically under-resourced communities.

Succeeding in this role requires a rare blend of rigorous technical capability and deep empathy for learners and educators. You must be comfortable navigating messy, large-scale behavioral datasets while maintaining a clear view of the pedagogical goals behind the numbers. If you are a curious, highly collaborative practitioner who wants to apply advanced statistical modeling and experimentation frameworks to redefine the future of education, this role offers an unmatched platform for professional impact.

The Khan Academy interview process

Khan Academy candidates report 5 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Evaluation

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

SQL and Product Assessment

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Cross-Functional Interviews

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Final Rounds

Where a loop includes a partner from outside the data team, that conversation usually carries the same weight as the technical ones and gets the least preparation. The person opposite you will not follow a derivation and does not need to. They are working out whether having you involved would make their decisions better or slower. The failure mode is not being too technical. It is answering a question about a decision with a description of your method, leaving the translation to them. What they carry into the debrief is the sentence you handed them, not the analysis underneath it.

What to demonstrate

Whether a statistical result arrives as something the partner could act on, with the one caveat that would change their decision kept and the rest left out
Whether you can state what you need from their side, in their terms: instrumentation that does not exist yet, a definition they own, or a holdout they have to agree to
Whether uncertainty is given as a range someone can plan against, rather than as hedging that invites them to ignore the result
Whether you ask what decision is actually on the table before explaining anything

How to prepare

Take a result you know well and write the version for someone who stops reading after one sentence, then the three-minute version, and check the short one is not the long one with the qualifications stripped out
For a past project, list everything you asked a non-technical partner for and how you phrased it, then rewrite each ask so it names what goes unmeasured without it
Practise saying where a result does not apply, out loud, in one sentence that a partner could repeat accurately to someone else

Where candidates lose points

Consent and age rules silently truncate the data, not just the joins.

Where age_gated is TRUE, behavioural logging and cross-system joins are restricted, so those learners are missing from the very tables used to compute engagement. An analysis that simply inner-joins will produce a population skewed to older learners and self-serve accounts while appearing complete. Check the age_gated share of every population you report on, and state it.

Pre/post gain studies that select on low pretest scores manufacture improvement.

Any measure with reliability below 1 produces regression to the mean, so a group chosen for scoring in the bottom quartile will score higher on retest with no intervention at all. The apparent gain scales with measurement error, which for a short quiz is large. The fix is a control group selected by the identical rule, or a design that models the pretest as a covariate rather than as a selection filter.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Accepting a metric definition without asking about the denominator

Pin down the denominator, the eligibility filter and the time window before computing anything: conversion rate per session, per user, per eligible user and per new user are four different numbers with different behaviour. Restate the definition in one sentence and get agreement before you analyse.

Questions & practice

How do you check for confounding variables when analyzing observational student learning data?

Medium · statistics and probability

Approach

Write down the assumption the method needs before you use the method.
Sanity-check the answer against a simple bound or a simulated case.
Translate the result into the decision it informs, in one plain sentence.

Follow-up

How would you explain this result to someone who does not know statistics?
Which assumption here is most likely to be violated in practice?

Describe how you would apply regression discontinuity design to evaluate the impact of a grade-advancement threshold on student retention.

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Check what information would not exist at prediction time, and exclude it.

Follow-up

What would you monitor after launch to know the model is still valid?
Where could label leakage enter this setup?

Sessionise a raw heartbeat stream into activity rows

Hard · sessionisation · event streams · groupby · time series

You get a DataFrame heartbeats with columns learner_id, content_item_id, content_version_no, and ts (UTC timestamps, unsorted). Produce one row per learner per session per content item with started_at, ended_at, wall_seconds, and active_seconds. A session closes after 30 minutes with no heartbeat from that learner. active_seconds sums consecutive gaps inside a session, capping each gap at 120 seconds. Attribute each gap to the content item of the earlier heartbeat. State your rule for a gap of exactly 1800 seconds and for a session with a single heartbeat.

Approach

Sort by learner_id then ts, and compute the gap with a per-learner diff so the first heartbeat of each learner has no gap. A global diff lets one learner's last heartbeat set the next learner's first gap.
Flag a session break where gap > 1800 seconds, then cumsum that boolean within the learner to get a session index. Say whether exactly 1800 opens a new session; either choice is defensible but it must be stated because it moves the session count.
Each gap covers the interval before the current heartbeat, so attribute min(gap, 120) to the content_item_id of the previous row via shift, not the current row.
Group by (learner_id, session_index, content_item_id): sum the attributed capped gaps for active_seconds, take min and max ts for started_at and ended_at, and derive wall_seconds from those two.
Handle the degenerate group explicitly. A single heartbeat gives active_seconds 0 and wall_seconds 0. Dropping those rows quietly removes real opens and biases any completion or abandonment rate computed later.
Validate that within every session the summed active_seconds never exceeds the session's wall_seconds, which catches attribution and cap errors in one assertion.

Worked solution (35 min)

Sort by (learner_id, ts) and compute gap = ts.diff().dt.total_seconds(), then null the gap on the first row of each learner using a learner-change mask.
new_session = gap.isna() | (gap > 1800); session_index = new_session.groupby(learner_id).cumsum().
contribution = gap.clip(upper=120).fillna(0), attributed to the previous row's content_item_id using shift within the learner and session.
Aggregate by (learner_id, session_index, content_item_id) with sum for active_seconds and min and max of ts for the timestamps.
Assert per session that sum(active_seconds) <= (max(ts) - min(ts)) in seconds, and print the count of single-heartbeat groups.

Expected result: A frame keyed by (learner_id, session_index, content_item_id) where active_seconds equals the sum of min(gap, 120) over consecutive heartbeats with the first heartbeat contributing zero, wall_seconds equals ended_at minus started_at, and sum(active_seconds) <= wall_seconds holds for every session.

Verify it yourself

Heartbeats at 0, 60, and 120 seconds on one item give one session with wall_seconds 120 and active_seconds 120.
Heartbeats at 0 and 3600 seconds give two sessions, each with wall_seconds 0 and active_seconds 0.
Inserting a 300-second gap inside a session adds exactly 120 to active_seconds, not 300, while wall_seconds rises by the full 300.

Follow-up

The client emits a heartbeat every 60 seconds on desktop and every 15 seconds on a managed device fleet. What does the 120-second cap do to the comparison between those two populations?
Your reconstructed active_seconds disagrees with the stored column by 4 percent overall. How do you localise the disagreement rather than argue about the total?
A learner has two tabs open on different items at once. What does your output claim, and what should it claim?

Write a SQL query using window functions to calculate the 7-day rolling average of active learners per grade level.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

How would you use a self-join or window function to identify consecutive days of platform usage for students?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
What breaks if events arrive late or out of order?

Consecutive instructional-week streaks and term-over-term return

Hard · gaps and islands · window functions · retention

Using fct_lesson_activity(learner_id, enrollment_id, activity_date, active_seconds) and a calendar dim_term_week(term_id, term_week_no, week_start_date, week_end_date, is_instructional), where holiday weeks carry is_instructional = FALSE, compute for each learner and term the longest run of consecutive instructional weeks containing at least one activity day. A non-instructional week must not break a run, and activity inside one neither extends nor starts a run. Then compute term-over-term return rate into term T+1 with a floor of five activity days in term T. Return learner_id, term_id, activity_days counted over the whole term including non-instructional weeks, longest_streak_weeks and the return flag.

Approach

Collapse activity to (learner_id, term_id, term_week_no) with COUNT(DISTINCT activity_date). fct_lesson_activity is one row per item per session, so a learner with twelve rows on one afternoon is one activity day and counting rows inflates everything downstream.
Re-index before differencing. Apply DENSE_RANK() OVER (PARTITION BY term_id ORDER BY term_week_no) across instructional weeks only, producing a gap-free sequence where a holiday week has simply been removed. Differencing raw term_week_no instead splits a run at every holiday, which is exactly the artefact the task forbids.
Accept what dropping those weeks costs. Activity in a non-instructional week is removed from the streak calculation entirely, so a learner whose only activity lands in a holiday week has activity_days >= 1 and longest_streak_weeks = 0. That is what a streak over instructional weeks means, but it is why the two columns can disagree and why the checks must expect a zero streak alongside non-zero activity rather than assert a floor of 1.
Apply the island trick on the re-indexed sequence: instructional_index - ROW_NUMBER() OVER (PARTITION BY learner_id, term_id ORDER BY instructional_index) is constant within a run. Group on that constant, count rows per group, take the max per learner and term.
Build activity_days from the unfiltered term aggregate, then LEFT JOIN the streak result onto it and COALESCE longest_streak_weeks to 0. An inner join here deletes the holiday-only learners from the output and from the return-rate denominator, which is the same bug as the streak-floor assertion wearing different clothes.
Set the activity floor on COUNT(DISTINCT activity_date) >= 5 across term T, applied to the return-rate denominator only. The floor exists to remove provisioned-but-unused accounts, so leaking it into the numerator silently redefines the metric as active in both terms. Note that a learner can clear the floor on holiday-week activity alone and enter the denominator with a zero streak.
Define the return flag as EXISTS any activity row in term T+1 for that learner, pairing terms by an explicit term ordering rather than date arithmetic, since term lengths differ. Learners whose org has no term T+1 loaded must be excluded and counted, not treated as non-returners.

Worked solution (40 min)

Build the learner-week activity CTE with COUNT(DISTINCT activity_date) and confirm one row per (learner_id, term_id, term_week_no).
Build the instructional re-index from dim_term_week and join activity onto it, dropping non-instructional weeks entirely.
Apply the index-minus-row-number island grouping and take MAX(run_length) per learner and term.
Compute activity_days per learner and term over the whole term, LEFT JOIN the streak onto it with COALESCE to 0, and count the learners who come out with activity_days > 0 and longest_streak_weeks = 0.
Apply the five-day floor to the denominator and attach the term T+1 existence flag.
Build a two-row fixture by hand and verify the holiday behaviour before trusting the full run.

Expected result: One row per (learner_id, term_id) with activity_days, longest_streak_weeks and returned_next_term, where longest_streak_weeks is 0 for a learner whose activity landed only in non-instructional weeks, plus a rollup giving the return rate over learners meeting the five-day floor and a count of learners excluded for having no term T+1 loaded.

Verify it yourself

longest_streak_weeks never exceeds the count of instructional weeks in that term, and is at least 1 for every learner with at least one activity date inside an instructional week.
Learners with activity_days > 0 and longest_streak_weeks = 0 exist and all of their activity dates fall inside weeks with is_instructional = FALSE. If any such learner has an activity date in an instructional week, the LEFT JOIN or the island grouping is dropping rows; if the count of these learners is large, suspect the is_instructional flags in dim_term_week before the query.
A learner active in weeks 3 and 5 where week 4 is non-instructional yields a streak of 2; the same learner where week 4 is instructional yields 1. Build that fixture and run it.
Return rate with a one-day floor is below return rate with a five-day floor. If it comes out higher, the floor has leaked into the term T+1 return condition.

Follow-up

This streak rewards a learner doing one minute a week over one doing four hours in a single week. Which of those is the product claim, and what would you pair the streak with to catch the difference?
Roster sync deactivates accounts in bulk at term end. How does that interact with the five-day floor and with the return denominator?
Show how return rate moves as the floor goes from one day to five, and argue which floor is the honest one to publish.

If we wanted to expand our reach in under-resourced school districts, what engagement metrics should our product team prioritize?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Define a set of north-star and proxy metrics for a newly introduced gamified learning streak feature.

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

A teacher dashboard feature has been launched to increase classroom engagement. What primary and secondary metrics would you track to determine its efficacy?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you evaluate whether a self-paced video module is actually driving deeper conceptual understanding versus passive viewing?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Explain the concept of statistical significance and how you would explain Type I and Type II errors to a non-technical product manager.

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

This category focuses on your capability to design, execute, and interpret controlled experiments while avoiding common statistical traps.

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.
Name the guardrails that would stop a launch even on a positive primary result.

Follow-up

What would you do if you could not randomise at all?
What would you conclude if the result is positive but the test is underpowered?

Decide what an administrator usage report counts

Hard · goodhart · reporting · guardrails

You must publish a five-line usage report that institutional administrators read before renewal. Once it is published, instructors will assign work to move whatever it counts. Available: fct_lesson_activity (active_seconds, is_assigned, completion_status), fct_assessment_response (max_points, scoring_mode, is_correct, response_seconds, hint_count), fct_enrollment (units_completed, units_total) and dim_learner (first_activity_at, age_gated). Choose the five lines and their source columns, name one metric you refuse to publish and the behaviour it would cause, and state the pairing that keeps each line honest.

Approach

Start from behaviour rather than from data availability. Publishing a number to the buyer converts it into an assignment, so the test for each line is whether its gamed version is still something you want happening.
Choose lines that survive that test: seat activation with the never-activated count shown rather than hidden; learners with at least one scored submission in the last four weeks; pace adherence at the current term-week; verified mastery events with the reporting lag printed; and objectives with no coverage yet, which is a gap list, is directly actionable, and cannot be inflated.
Name the source columns per line so the report is reproducible: first_activity_at against seats_provisioned, fct_assessment_response.submitted_at with scoring_mode, units_completed over units_total against the term dimension, the mastery ledger with its delayed check, and the objective coverage join.
Refuse active minutes per learner. Publishing it causes seat-time assignments, which is the clearest case of teaching to the number in this domain and the weakest claim the product can defend as learning. Keep it internally as a guardrail where it is useful and harmless.
Pair every countable line with an integrity partner: pace adherence with assessment integrity (response_seconds above the item floor, hint_count below cap), submissions with mean max_points so splitting one task into five cannot move the line, activation with the raw never-activated count.
State the population on the report itself, including the age_gated share and any excluded enrollment sources, because those learners are missing from the behavioural tables and the report will otherwise look complete while describing a partial population.

Worked solution (30 min)

For each candidate line, write the one sentence describing how an instructor would move it with no learning, and drop the lines whose gamed version is undesirable.
Write the five surviving lines with their source columns and windows.
Attach an integrity or substance partner to each countable line, and write the single sentence justifying the refusal of active minutes per learner.
Add the population footer: age_gated share, excluded enrollment sources, and the reporting lag on verified mastery.

Expected result: Five lines (seat activation with the never-activated count, four-week learners with a scored submission, pace adherence at term-week, verified mastery events with its lag, uncovered objectives), each with named source columns and an integrity partner; active minutes per learner refused with the seat-time assignment behaviour it would cause; population and lag stated on the report itself.

Verify it yourself

Every count line has a partner that defeats the obvious way to inflate it.
Source columns are named, not just metric names.
The report states the population it covers, including the age_gated share and any excluded enrollment sources.

Follow-up

An administrator says your report shows fewer active learners than their own dashboard. What are the three likeliest causes?
A district asks for a per-teacher leaderboard built from these lines. What do you do?
If you could publish only four lines, which one goes, and what breaks?

Pace adherence at term-week six rose eleven points

Medium · definition-change · survivorship · pacing

Pace adherence at term-week 6 rose from 48 percent to 59 percent between two consecutive terms, and the dashboard query changed owners last month. Tables: fct_enrollment (enrollment_id, learner_id, course_id, units_completed, units_total, status, scheduled_start_date, due_date, enrolled_at) and dim_term (term_id, term_start, term_length_weeks). Nothing shipped to pacing in that period. Decide whether learners are pacing better, and reconcile the old and new numbers to the point.

Approach

Diff the definitions before touching the data. Recover the prior query and compare it line by line against the current one on four axes: the denominator population, the status filter, how term-week 6 is derived, and what happens to enrollments with NULL scheduled_start_date.
Recompute both definitions on the same term and the same enrollment set. If the difference does not account for the full eleven points there is a second cause, and knowing that before you start hunting saves the hunt.
Test the survivorship mechanism directly: count enrollments with status in ('dropped','expired') at the end of term-week 6 and compute their units_completed over units_total. If that ratio sits well below the active median, removing them from the denominator lifts the rate mechanically and the lift is arithmetic, not behaviour.
Verify the numerator independently, because units_completed is maintained incrementally rather than derived on read. Recount it from lesson activity for a sample of a couple of hundred enrollments and compare against the stored column to rule out a backfill as a second mover.
Confirm term-week 6 maps to the intended dates in both terms through dim_term rather than calendar week, since an edited term_start silently shifts the evaluation window and can move the number on its own.

Follow-up

The old definition penalises a course for enrollments that dropped in week two. Is that the behaviour you actually want from a pacing metric, and who is the metric for?
How do you treat enrollments with NULL scheduled_start_date, and what does each choice do to the reported number?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Explaining a method to someone who will not follow the maths

Half of this section is about translation. Be ready to describe how you explained a result to someone who did not want the method, only the implication, and what you did when the simplified version started being repeated in a way that overstated it. Correcting your own simplification is a strong beat.

How do you handle sample ratio mismatch (SRM) when you discover it in an ongoing educational experiment?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Pick a story where you drove the decision, not one where you observed it.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Explain a risk score to a non-technical sales executive

Easy · communicating uncertainty · renewal risk · stakeholder communication

Your renewal-risk score joins fct_subscription_period to seat activation from dim_learner.first_activity_at and to pace adherence from fct_enrollment. For one institution with 900 seats_provisioned and 31% activation at term-week 6, it outputs a 0.62 probability of non-renewal. A sales VP asks whether you are losing the account or not. You have two minutes and no slides. Deliverable: the spoken answer, plus the one line you would put in the weekly account review so the number is not later quoted as a certainty. Probed: whether you can carry uncertainty and still give a decision.

Approach

Answer the question that was asked before qualifying it. 'Treat this as at-risk and act this week' is the answer; the probability is the support for it.
Translate the score into a frequency on a reference class the VP already trusts, such as how many of the last fifty accounts scored in the 0.55-0.70 band did not renew, instead of explaining calibration in the abstract.
Name the driver that is still movable before period_end: 31% seat activation at term-week 6 can be changed this term, whereas last term's usage cannot, so that is where the conversation goes.
State what would move the score and by when, so the number arrives attached to an action rather than as a verdict.
For the written line, use a band and a review date rather than a bare decimal, because a single number in a document will be re-quoted without its interval.

Follow-up

The VP asks for every account scored above 0.50. What do you tell them about where that threshold came from?
How would you check whether 0.62 is calibrated at all, and on what sample?

State the impact of your last project without inflating it

Hard · impact measurement · attribution · causal reasoning

Pick one project from the past year and account for its impact as if the listener could audit every number. Cover the metric you claim to have moved and its exact definition, how much of the movement is attributable to your work, what else was changing at the same time, what the counterfactual was and where it came from, and what you would have needed to measure at the start to make the claim clean. If the effect is not separable, say so and say what you would do differently. Probed: attribution discipline applied to your own work.

Approach

Name the metric by its definition rather than its label. 'Verified mastery rate' means nothing without its numerator, denominator, window, and the reporting lag the delayed retention check forces.
Separate the three claims usually merged into one: the metric moved, your work moved it, and the movement was worth what it cost.
State the counterfactual explicitly and say what produced it: a holdout, matched sections, or a pre-period trend. If it was a before-and-after with no control, say that in those words.
List the co-occurring changes with dates: a term boundary, a roster sync at scale, a pricing change, another team shipping into the same surface. In this domain the academic calendar alone routinely dwarfs a product effect.
Convert the shortfall into a specific design artefact: the holdout you would have reserved, the instrumentation you would have shipped before the change, the analysis plan you would have written first.

Follow-up

What fraction of the observed movement would you defend under cross-examination, and on what evidence?
Your project shipped in term-week one. How does that change what you can claim?
If the effect is genuinely not separable, was the project worth doing?