Interview guides · Faculty.ai

Faculty.ai Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Faculty.ai: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

The Data Scientist role at Faculty.ai is central to the company’s mission of helping organizations realize the potential of artificial intelligence. You will not be working in a silo; instead, you will collaborate with cross-functional teams to solve complex, high-stakes problems for a diverse range of clients. This role requires a blend of rigorous technical expertise and the ability to translate abstract business challenges into actionable data-driven solutions.

Seniority shifts the scope more than the words in the title do. Earlier-career loops mostly check that you execute a well-posed analysis correctly; senior loops check that you can decide which question is worth answering and defend what you chose not to do.

Focus areas

Turn a vague request into a measurable question
Define numerator, denominator and window precisely
Pick a randomisation unit that respects interference

Overview

Success in this position requires more than just coding proficiency. You will be expected to demonstrate a deep understanding of the end-to-end machine learning lifecycle—from initial data exploration and hypothesis generation to model deployment and long-term performance monitoring. Whether you are building bespoke AI products or providing strategic consulting, you will be a key contributor to the intellectual and technical rigor that Faculty.ai is known for.

The Faculty.ai interview process

Faculty.ai candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Screen

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Technical Assessments

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Commercial and Behavioral Interviews

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Case Studies

Underneath the business framing, this round is usually asking whether you can turn a fuzzy goal into a quantity that could be computed from data such a business would plausibly hold. That means a metric with a stated numerator, denominator, eligibility rule and time window, plus an honest account of the conditions under which it would mislead you. Answers come apart when a candidate names a familiar metric and never defines it, because every follow-up then lands on an ambiguity that was left open and the candidate has to invent the definition under pressure.

What to demonstrate

Whether a named metric arrives with its denominator, eligibility rule and window attached rather than assumed
Whether the measure follows from the mechanism you proposed, or is a recognisable metric retrofitted to it afterwards
Whether you name a guardrail that would reveal the gain came from somewhere you did not want it to come from
Whether you can say what data the plan requires and what you would settle for if that logging were never implemented

How to prepare

Take five metrics you reach for by reflex and write each as one sentence containing numerator, denominator, eligibility rule and time window. The ones you cannot finish are the ones that will fail under follow-up.
For a product you use daily, write the measurement plan you would propose for a change to it: primary metric, one guardrail, the unit of analysis, and the table the numbers would come from.
Practise the substitution question. For three metrics you like, write what you would measure instead if the event you depend on were not being logged.

Where candidates lose points

Watching an experiment daily and stopping when it crosses significance

A fixed-sample test controls type I error at one pre-declared look. Checking repeatedly and stopping at the first p < 0.05 inflates the false positive rate to roughly 0.15 to 0.20 for ten looks, and it rises further with more frequent checks, because the p-value takes a random walk that will eventually dip below the threshold under the null. The usual defences are a fixed horizon declared before launch, group-sequential boundaries such as O'Brien-Fleming that spend alpha across a planned number of looks, or always-valid confidence sequences that are correct under continuous monitoring. Compounding it, the effect size reported conditional on having crossed the threshold is biased away from zero, and the bias is larger the lower the power was, so an underpowered test that 'won' typically overstates the lift it found.

Slicing a flat experiment until a segment reaches significance

Testing one metric across twenty segments at a nominal 5% level produces a significant result about two thirds of the time when nothing is happening anywhere, and the segment that surfaces is by construction the one with the most favourable noise. The reported effect in that slice is then badly overstated, because selection on significance conditions the estimate on being large. What makes it dangerous rather than merely wrong is that a post-hoc segment always has a plausible story attached, so it survives the meeting. The controls are declaring the small number of segments of interest before launch, correcting across the ones tested, and treating anything discovered afterwards as a hypothesis that needs its own adequately-powered test rather than a finding.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Reading a dozen metrics with no multiplicity control

Nominate one primary metric before launch and treat the rest as guardrails or exploratory, with Bonferroni or Benjamini-Hochberg applied when you intend to make claims from them. Twenty independent tests at 0.05 under the null produce at least one false positive about 64 percent of the time.

Questions & practice

How would you determine if two discrete random variables are independent given their joint distribution?

Medium · statistics and probability

Approach

Sanity-check the answer against a simple bound or a simulated case.
Quantify uncertainty explicitly rather than reporting a point estimate alone.
Say what the estimate is of, and over what population it generalises.

Follow-up

How would you explain this result to someone who does not know statistics?
What sample size would you need to detect an effect half this size?

Describe the trade-offs between various evaluation metrics for a classification model.

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Rebuild per-visitor ordering without groupby convenience methods

Easy · pandas · vectorisation · window logic

You have a DataFrame of 2 million fct_event rows with visitor_id, occurred_at_utc and event_id, unsorted and containing duplicate timestamps within a visitor. Produce three new columns: event_rank, the 1-based position of the event within its visitor ordered by occurred_at_utc; seconds_since_prev, the gap to that visitor's previous event, NULL for the first; and is_first_for_visitor. You may use sort_values, shift, cumsum, numpy and boolean masking. You may not use groupby.transform, groupby.apply, groupby.cumcount, groupby.rank or merge_asof. Break timestamp ties on event_id.

Approach

Sort once by ['visitor_id', 'occurred_at_utc', 'event_id'] and reset the index. The whole exercise reduces to row arithmetic on a sorted frame, and the tiebreak on event_id is what makes the result reproducible across runs.
Mark visitor boundaries with is_first = df['visitor_id'].ne(df['visitor_id'].shift()). This is the single fact every other column derives from.
Compute seconds_since_prev as the diff of the timestamp column, then overwrite it with NaT/NaN wherever is_first is True. The shift crosses the boundary between visitors and will otherwise hand the first row of each visitor the last event of the previous one.
Build event_rank from a running counter that resets at boundaries: take a global cumulative position (np.arange(len(df))) and subtract, per row, the global position at which that visitor started. Get the start position by forward-filling the positions where is_first is True, which is a cumsum-free reset and is O(n).
Verify against the forbidden method once, as a test rather than as the implementation, and confirm the two agree on every row.

Worked solution (20 min)

Sort on the three-key tuple and reset_index(drop=True).
Compute is_first via .ne(.shift()), which is True for row 0 because the shifted value is NaN.
pos = np.arange(len(df)); start = pd.Series(np.where(is_first, pos, np.nan)).ffill(); event_rank = (pos - start + 1).astype(int).
gap = df['occurred_at_utc'].diff().dt.total_seconds(); gap[is_first] = np.nan.
Assert event_rank equals df.groupby('visitor_id').cumcount() + 1 on the sorted frame.

Expected result: Three columns on the sorted frame: event_rank starting at 1 for every visitor and increasing by 1 with no gaps, seconds_since_prev null exactly where is_first_for_visitor is True, and is_first_for_visitor summing to df['visitor_id'].nunique().

Verify it yourself

is_first_for_visitor.sum() equals the number of distinct visitor_id values.
seconds_since_prev.isna().sum() equals that same number.
event_rank.max() per visitor equals that visitor's row count, and event_rank is non-negative everywhere.
A visitor with two events at the identical timestamp gets ranks 1 and 2 with a gap of 0.0, ordered by event_id.

Follow-up

The frame does not fit in memory. How does your approach change if you can only process one visitor-partitioned chunk at a time?
occurred_at_utc is client-supplied and sometimes runs backwards within a visitor. Does your seconds_since_prev go negative, and should it?
How would you extend this to reset the counter at every change of surface as well as visitor?

Given a large dataset, how would you optimize a query that involves multiple self-joins?

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Can you explain how to handle null values during a join operation in SQL?

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.

Follow-up

What breaks if events arrive late or out of order?
How would you verify this result without re-running the same query?

How do you use SQL window functions to calculate running totals or rankings within a dataset?

Medium · sql and data manipulation

Approach

Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Find reactivation gaps in account paid-period history

Hard · gaps-and-islands · window-functions · subscriptions

fct_subscription_period holds account_id, subscription_id, period_start_utc, period_end_utc, period_status and change_reason. A mid-period plan or seat change closes one row and opens another, so a single continuous paid tenure is often many rows, and an account may hold two overlapping subscriptions. Collapse rows with period_status in ('active','past_due') into continuous tenures per account, treating gaps of three days or less as continuous. Return account_id, tenure_start, tenure_end, and for every tenure after the first, the gap in days that preceded it.

Approach

Filter to paid rows only: period_status IN ('active','past_due'). Trialing periods are not tenure, and including them turns every trial that never converted into a one-period tenure followed by a fake churn.
Order by period_start_utc and take a running maximum of all prior ends: MAX(period_end_utc) OVER (PARTITION BY account_id ORDER BY period_start_utc, period_end_utc, subscription_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING). Those three columns are the only stable ordering this schema exposes, so check first that they are unique within an account; if rows tie on all three, the island numbering is order-dependent between runs and you need a real row key before the result is reproducible.
LAG on its own is wrong here because with overlapping or nested periods the immediately preceding row by start date is not the one that ends latest, so the running maximum is the part that cannot be shortcut.
Flag a new island when prior_max_end IS NULL OR period_start_utc > prior_max_end + interval '3 days', then number islands with a running SUM of the flag over the same ordering and an explicit ROWS frame.
Group to (account_id, island) taking MIN(period_start_utc) and MAX(period_end_utc), then LAG(tenure_end) OVER (PARTITION BY account_id ORDER BY tenure_start) to compute the preceding gap in days for every tenure after the first.
Sanity-check with change_reason, which is the only lineage this schema carries: list its distinct values first, then confirm that rows recording a plan or seat change sit inside a tenure rather than opening one, and that every tenure after the first opens on a row whose reason records a restart rather than an ordinary renewal. Do not reconcile against a churn timestamp on dim_account, which this schema does not define; and where such a column does exist, a cancellation timestamp records when the request was made and routinely sits weeks before the period it ends.

Worked solution (35 min)

Find an account with a known mid-period upgrade and dump its period rows to use as the trace case.
Check that (period_start_utc, period_end_utc, subscription_id) is unique per account, since the whole ordering rests on it.
Write the paid-rows CTE and the running MAX with the explicit frame.
Add the island flag and the running SUM, then verify the trace account yields one island.
Group to tenures and add the LAG-based gap in days.
List the distinct change_reason values, then count accounts with more than one tenure and compare against the count of accounts carrying a restart-flavoured reason anywhere in their history.

Expected result: One row per (account_id, tenure). An account with a continuous history returns a single row with a NULL preceding gap. Every non-first tenure has gap_days strictly greater than 3 by construction.

Verify it yourself

No two tenures for the same account overlap or touch within the grace window.
MIN(tenure_start) per account equals MIN(period_start_utc) per account in the filtered input.
The account with a known mid-period upgrade produces one tenure, not two.
(period_start_utc, period_end_utc, subscription_id) has no duplicates within an account; if it does, say how many accounts are affected before reading the output.
The share of accounts with more than one tenure is small; if it is most accounts, the status filter or the grace window is wrong.

Follow-up

Why three days of grace? What do 0 and 30 days each do to the count of accounts classed as reactivated?
An account runs two concurrent subscriptions for different teams. One tenure or two, and what does the revenue reader expect?
How would you turn these tenures into a monthly gross logo churn series without double-counting an account that churned and returned in the same month?

How do you define success for a new product, and which product metric design framework would you apply?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

If you notice a sudden drop in a key product metric, what is your systematic approach to diagnosing the root cause?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

How do you design an A/B test to evaluate a new feature rollout, and how do you ensure statistical significance?

Medium · a/b testing and experimentation

Approach

Decide the analysis before seeing data, including how long it runs and when you look.
Say whether units interfere with each other, and switch design if they do.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

What are the common experimentation pitfalls that could lead to biased results or false positives?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

A surrogate for twelve-month value inside a three-week test

Hard · surrogate metrics · experimentation · long-term value

A monetisation change — an earlier and harder paywall — will be tested for three weeks. The decision quantity is twelve-month cumulative net revenue per acquired account, which cannot be observed in three weeks. Available: fct_event, fct_session, fct_subscription_period (mrr_cents_constant_fx, period_status, change_reason, is_first_paid_period), dim_account, and fct_experiment_exposure (unit_type, unit_id, variant, first_exposed_at_utc, is_in_analysis_population). Construct a surrogate index readable at three weeks, state the assumption that makes it valid, name the mechanism that breaks it, and give the pre-registered rule for refusing to decide. Deliverable: the index, the assumption, and the refusal rule.

Approach

State the surrogacy condition before building anything: a surrogate is valid only if the treatment's entire effect on twelve-month revenue runs through it. Then say where it fails here, because a paywall moves short-run revenue directly and long-run revenue through churn and through who becomes a payer at all.
Fit the index on history rather than on intuition: regress twelve-month cumulative net revenue per account on features observable by day 21 — first-paid flag, activation days, week-3 core-action days, seats billed — using cohorts old enough to have a twelve-month outcome, and hold out a later cohort.
Report the out-of-sample fit as the headline rather than the point estimate: the held-out R-squared and the calibration of predicted against actual deciles are what license any use of the index, and a decile plot catches the mis-calibration an R-squared hides.
Name the bias direction explicitly: the relationship was estimated under the old paywall, so under a harder paywall the marginal payer is a different person and the index over-predicts their value, biasing the treatment arm optimistic.
Surround the index with the direct three-week readouts it cannot contain — revenue per exposed unit, cancel-within-first-period rate, and the free-side signup and activation counts — and pre-register a numeric refusal rule, because the failure mode of a surrogate is being used confidently in exactly the case it was not fit for.

Worked solution (40 min)

Assemble the training set: accounts with first_paid_at_utc at least twelve months old, their realised twelve-month cumulative net revenue, and the day-21 features.
Fit on pre-change cohorts, evaluate on a held-out later cohort, and report both out-of-sample R-squared and predicted-versus-actual decile calibration.
Write the surrogacy assumption as a falsifiable sentence, then write the specific mechanism in this treatment that violates it.
Specify the direct arm readouts on fct_experiment_exposure with is_in_analysis_population = TRUE: revenue per exposed unit, cancel-within-first-period rate, and free-side signup and activation counts.
Write the refusal rule with numbers attached, covering both the fit threshold and the free-side divergence case.

Expected result: A fitted surrogate with a reported out-of-sample R-squared and calibration, a written surrogacy assumption plus the mechanism that violates it, the resulting bias direction, and a numeric refusal rule agreed before launch.

Verify it yourself

Standard errors are clustered on fct_experiment_exposure.unit_id, the identifier of the randomisation unit. unit_type is the kind label for that unit — a single value such as 'user' across the whole table — so clustering on it collapses the design to one or two clusters and carries no information about who was randomised together. A per-event or per-session standard error on a user-randomised test is the opposite error: too small, and it will manufacture significance.
The analysis population is the exposed set with is_in_analysis_population = TRUE, and a sample-ratio check against the intended split passes before any outcome is read.
The index applied to the control arm should reproduce the historical relationship. If control is already mis-calibrated, the treatment estimate is not interpretable at all.

Follow-up

What evidence would make you trust a surrogate for this particular change rather than a different one?
The index reads plus 8 percent while three-week cancel-within-period is up. Which do you act on, and what do you tell the decision-maker?
How would you size this test, on what unit, and what does that do to the three-week horizon?

Tell a pipeline outage from a collapse in usage

Medium · pipeline-failure · instrumentation · event-volume

Weekly active accounts completing a core action fell 9%, and almost the entire fall sits in accounts whose events carry surface = 'ios'. App crash rates and store reviews are unchanged. You have fct_event with occurred_at_utc, received_at_utc, event_name, is_core_action, app_version and surface, plus fct_session and the ingestion job run log. Establish within the hour whether iOS engagement fell or iOS events stopped arriving, name the evidence that distinguishes them, and say what you would publish on the dashboard in the meantime.

Approach

Compare the event-name composition inside surface = 'ios' against the prior four weeks as shares, not counts. A behaviour collapse scales most event names together; a dropped event definition or a broken downstream filter hits specific event_name values while page_view and session-opening events hold steady. That shape difference is the fastest discriminator available.
Profile the received_at_utc minus occurred_at_utc distribution per day for surface = 'ios'. A stalled-then-backfilling pipeline shows a fat upper tail and a recovering p99; a silently dropped stream shows an unchanged lag distribution over a smaller volume. The two failure modes have different remedies and different histories.
Cut by app_version. A logging SDK change arrives with one build and ramps with its adoption curve; an infrastructure fault arrives across every build within the same hour. Checking this costs one group-by and rules out half the hypothesis space.
Cross-check against a signal that does not travel the suspect path: server-emitted events with session_id NULL, and subscription or billing activity for the same accounts. If those accounts are still transacting, the users did not leave.
Publish an ex-iOS total with an explicit annotated break rather than a blended total. A blended number during a known ingestion gap is wrong in a direction you can already name, and republishing it daily spreads the artefact into every downstream report.

Follow-up

Suppose the events do eventually backfill. What is your policy for restating the published weekly numbers, and who needs to be told?
What monitor would have caught this before a human noticed the weekly metric, and what would it alert on?
If is_core_action is maintained in the tracking plan, what governance would stop a change to that list from silently moving a north-star metric?

Suggested practice checklist

For someone who has spent the last year in notebooks, dashboards or modelling work and has not written raw SQL under time pressure. The first four days rebuild query fluency against a fixture you control and can verify by hand; the last three attach that fluency to the rest of the loop.

Day 1 — Build a fixture you can check answers against

Create a local Postgres or SQLite database with four tables (users, sessions, events, orders) holding roughly 200 rows you generated yourself, so you know the contents well enough to predict every result.
Deliberately seed the cases that break queries: a user with no sessions, a session with no events, two orders sharing a timestamp, a NULL in one join key, and one duplicated user row.
Before writing any SQL, hand-compute five answers on paper (how many users placed at least one order, median orders per ordering user, and three others) and save them as the ground truth for the week.

Deliverable: A one-command seed script plus a text file of five hand-computed answers to grade every later query against.

Day 2 — Joins, filters and NULL semantics

Answer "which users have no orders" three ways (LEFT JOIN with IS NULL, NOT EXISTS, NOT IN) and confirm that the NOT IN version returns zero rows once the subquery contains a NULL, because the comparison is never TRUE.
Reproduce the LEFT JOIN that silently collapses to an inner join by putting a right-table predicate in WHERE, then fix it by moving the predicate into the ON clause, and record both row counts.
Create a fan-out bug on purpose by joining orders to order_items and summing the order total, then correct it with a pre-aggregated subquery and explain in one line which table changed the grain.

Deliverable: One annotated .sql file holding the three join traps, each with the wrong result and the corrected result side by side.

Day 3 — Window functions and frames

Write three window queries against the fixture: a running order total per user, the rank of each order within its user by value, and the day gap to that user's previous order, then check each against the day-one ground truth.
Run ROW_NUMBER, RANK and DENSE_RANK over a column containing ties, print all three side by side, and write one sentence on when each is the correct choice.
Switch one query from the default frame (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, which is what you get when ORDER BY is present and no frame is written) to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and explain why the output differs only when the ORDER BY column has duplicates.

Deliverable: Three verified window queries plus a short note explaining the RANGE versus ROWS difference in your own words.

Day 4 — The four analytical query patterns

Write a monthly retention grid: first order month per user, then months-since-first as the column, and verify that month zero equals the cohort size exactly.
Sessionize the events table under a 30-minute inactivity rule using LAG plus a cumulative sum over a new-session flag.
Build a four-step funnel that counts distinct users rather than events at each step, and state the rule you applied to a user who reaches step three without ever logging step two.

Deliverable: One file with the retention, sessionization and funnel patterns, each carrying a one-line note on the assumption it bakes in.

Day 5 — Write SQL the way you will have to write it live

Set a 12-minute timer and solve three medium prompts in a plain editor with no execution and no autocomplete, then run them and tally syntax errors separately from logic errors.
Narrate one solution aloud while writing it, stating the grain of each intermediate result (one row per user, one row per user-day) before you type its body.
Rewrite your slowest solution as a CTE chain where every CTE name states its grain, and time yourself re-solving it from blank.

Deliverable: A recording of one narrated solution plus an error tally that separates syntax from logic.

Day 6 — One day for everything that is not SQL

Write the preconditions of the two-sample t-test from memory, then check them: independent observations, and a difference in means whose sampling distribution is approximately normal, which at large sample sizes follows from the central limit theorem rather than from normality of the raw values.
Write the difference between an odds ratio from logistic regression and a relative risk, and state the condition under which the two are close (low outcome prevalence).
Prepare a 90-second answer to "how would you know this model is any good" that names the metric, the baseline you would beat, and the cost of the errors you care about.

Deliverable: One page of notes covering test preconditions, the odds-ratio caveat and the model-quality answer.

Day 7 — Full loop rehearsal

Run a 45-minute mock with someone willing to interrupt: 20 minutes of SQL, 15 minutes defining a metric, 10 minutes on a past project.
Re-solve from blank the two queries you were slowest on this week and compare the times against day five.
Write a five-line answer to "walk me through a project" that puts a number in the first sentence and names the decision the work changed.

Deliverable: Mock feedback notes plus a timed project narrative you can deliver without reading it.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

Tell me about a time you disagreed with a team member’s technical decision. How did you resolve it?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Defend a flat experiment readout against a post-hoc segment

Medium · experiments · segmentation · pushback

A feature you evaluated is flat on seven-day activation: +0.05pp with a 95% interval of [-0.47pp, +0.57pp], from 61,000 exposed users per arm in fct_experiment_exposure joined to dim_user and fct_event. Baseline activation is 32%. The launch team asks you to drop every surface except mobile_web, where the point estimate is +1.1pp, and re-run. You have ten minutes in their planning meeting. Deliver a spoken position: what you will and will not do, and the decision you recommend.

Approach

Recognise what is being probed: whether you hold a statistical position under social pressure without becoming either rigid or apologetic. A generic answer says the segment is not significant; a strong one separates the request into a question that is answerable (is the mobile_web number real?) and one that is not (can we ship on it?), and answers both.
Price the multiplicity out loud. The slice was chosen after seeing the results, so its estimate is selected on favourable noise and is biased away from zero. With k independent looks at a nominal 5% level, the chance of at least one false positive is 1 - 0.95^k: 26% at six segments, 64% at twenty. Quote the k you actually inspected, not the k you reported.
Use the arithmetic already in front of you. On the point estimates, a +1.1pp mobile_web effect combined with a pooled +0.05pp implies the remaining surfaces average negative in proportion to mobile_web's share of exposures. State that as a testable implication of their story rather than as a rebuttal of it.
Ask the one question that settles the category: was mobile_web named in the analysis plan before launch? If it was, it is a planned comparison and gets a corrected reading. If it was not, it is a hypothesis, and the honest move is to size the test that would confirm it.
Convert the refusal into a cost. Size a mobile_web-only confirmatory test at the claimed effect, state the weeks of mobile_web traffic it needs, and close with the recommendation: do not ship this as a lift, and note that the interval already rules out anything at or above +0.6pp, which is itself a useful input to the roadmap.

Follow-up

The confirmatory test you sized needs nine weeks of mobile_web traffic and the team has three. What do you recommend instead?
Suppose mobile_web was pre-registered. How does your reading change, and what correction do you apply?
Your interval excludes +0.6pp. Is that the same as saying the feature does nothing?

Quantify your own impact without claiming the topline you touched

Hard · self-assessment · attribution · communication

You are writing the impact section of your own review. Over the year you ran four experiments, one of which shipped and three of which were flat; you corrected the definition of gross monthly revenue churn so that cancellation is recognised at period_end_utc; and you built a self-serve funnel dashboard. Weekly active accounts rose 14% over the same period. Your reviewer knows the data well. Write the three impact claims you would defend, stating for each what you contributed, what evidence supports it, and what portion of the outcome you are not claiming.

Approach

Recognise what is being probed: whether you apply to your own work the causal standard you would apply to somebody else's roadmap claim. Nearly everyone who would reject 'accounts that do Y retain better' will write 'I drove a 14% increase' without noticing it is the same error with a friendlier subject.
Sort the work by the kind of evidence it can carry. The shipped experiment is the only item with a randomised estimate, so it is the only one where an effect size is defensible, and you claim the interval rather than the point estimate.
Claim the three flat experiments as decisions prevented and price them. Features not built, or built differently, on evidence, with the engineering weeks reallocated as the number somebody else can verify. A defensible null is a delivered decision and should be written as one.
Claim the definition fix as correctness, not as improvement. The old figure was overstated by a specific percentage and appeared in a specific set of recurring documents; the impact is the change it produced in the forecast built on top of it, not a change in churn itself.
Claim the dashboard on usage and displacement: distinct weekly users of it, and the ad-hoc request count for six months before against six months after. If the request log does not exist, record the claim as unverified rather than estimating it upward.
Disclaim the 14% explicitly and once. State that it cannot be separated from seasonality, other teams' launches and a pricing change, and bound your own contribution from above using the shipped experiment's interval converted into headline units.

Follow-up

Your shipped experiment's interval was +0.2pp to +1.4pp on activation. How much of the 14% can that account for, and how do you say so without undercutting yourself?
A peer in the same cycle claims the full 14%. What, if anything, do you do about it?
If you could only keep two of your three claims, which do you drop, and why that one?