Interview guides · Cogito

Cogito Data Scientist Interview Questions & Guide 2026

Take-home style practice for data scientist candidates: a messy dataset, a fixed time budget, and an analysis you write up and defend.

The 60-second brief

As a Data Scientist at Cogito, you will play a pivotal role in harnessing data to drive meaningful insights and innovations. This position is crucial for developing and enhancing products that leverage advanced analytics and machine learning to better understand user interactions and improve decision-making processes. Your work will directly impact how users experience Cogito's offerings, shaping the future of data-driven solutions in the realm of communication and engagement.

Scope your preparation by the data you would actually touch, because the title will not tell you. A seat that lives in event logs and weekly readouts rewards fluency in aggregation and metric definitions; a seat that owns a model in production rewards fluency in train/serve skew, retraining cadence and drift monitoring. The fastest way to find out which one you are interviewing for is to ask what the team shipped last quarter and what it gets paged about.

Focus areas

Measure catalogue breadth beyond head consumption
Correct discovery slates for position bias
Separate release-calendar spikes from underlying trend

Overview

In this role, you will engage with cross-functional teams to address complex problems, utilizing your expertise to inform product development and strategy. The challenges you tackle will vary in scale and complexity, providing a dynamic work environment that encourages creativity and collaboration. You will contribute to exciting initiatives that not only enhance Cogito's product portfolio but also drive strategic business objectives, making your impact felt across the organization.

The Cogito interview process

Cogito candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Technical Assessment

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Behavioral Interview

Behavioural answers from data candidates get audited in a way that answers from other roles do not. When you say a model lifted retention, the next question is the denominator, the window, and how you knew the lift was not seasonal. So attach the measurement to each claim while you tell it: what the metric was before, over what period, and against what comparison. Numbers with no baseline read as rounded-up memory, and one unsupported figure tends to make the rest of the story sound rehearsed.

What to demonstrate

Whether each impact number arrives with a baseline, a window and a comparison, or as a bare percentage
Whether you can name the method that attributed the effect to your work (an experiment, a staged rollout, a seasonal control) or concede the link was correlational
Whether the magnitudes stay internally consistent when the interviewer multiplies them against the scale you described earlier

How to prepare

For each story, write the impact line as metric, value before, value after, window, and how attribution was established. Any line missing two of those five is a follow-up you will answer badly.
Re-derive one headline number from the source table rather than the deck that reported it. Resume numbers drift upward across retellings.
Decide in advance which figures you cannot share, and prepare the ratio or relative change you can give instead, so a confidentiality limit does not read as evasion.

Onsite Interviews

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Testing hours, revenue or completion with a difference in means on a heavy-tailed distribution.

Listening and viewing hours per account are strongly right-skewed and content popularity is close to power-law, so the variance of a sample mean is dominated by a few accounts and the central limit approximation converges slowly at realistic sample sizes. A t-test on mean hours can flip sign when one heavy account's week changes, and an experiment can appear significant because a single title released into one arm's window. Capping at a pre-registered percentile, or decomposing into a rate (did they stream at all) and a conditional intensity, controls the variance, at the stated cost that capping biases toward zero exactly when the true effect lives in the tail.

Treating the account as the person, or the profile as the person.

A household account carries several people, profiles frequently are not switched, and shared-screen, car and speaker playback often lands on a default profile with no user behind it. Personalisation trained on a profile therefore learns a mixture, retention regressions attribute one member's behaviour to another, and a per-account taste statistic describes a household composite. The practical consequence is that apparent personalisation wins can be device or context effects, so any identity-level claim needs a stated unit and an acknowledgement of what that unit actually aggregates.

Analysing at a different unit than the one randomised

Say out loud what was randomised (user, device, account, cluster) and make the analysis unit match, or account for the clustering with cluster-robust standard errors, the delta method, or aggregation up to the randomised unit. Randomising users and then running a test over sessions understates variance and inflates the false-positive rate.

Building features from data that postdates the prediction time

Check every feature against the timestamp at which the model would actually score, and drop anything computed from a window that includes or follows the label event. For a forecasting use case, split train and test by time rather than at random, and split by entity when the same entity recurs.

Questions & practice

Can you discuss a time you implemented a machine learning model in production?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

Permutation test for hours per account across two ranker arms

Medium · permutation-test · heavy-tails · experiment-analysis

arm_hours holds one row per account: account_id, arm in {control, treatment}, qualified_hours over a seven-day window. Roughly 40 thousand accounts per arm, about 38 percent of them at zero hours, and the non-zero tail is long. Without calling a library test function, write a permutation test on the difference in mean hours with 10,000 relabellings. Then run it as two parts: the difference in the share of accounts with any hours, and the difference in mean hours among accounts with hours. Report all three and say which belongs in the readout.

Approach

Shuffle the labels, not the data. Draw a permutation of the arm indicator over accounts, which is the unit that was randomised, and hold the hours vector fixed.
Make each replication O(n): precompute the grand sum and the arm sizes, so a shuffled difference is the treated subset sum over n_t minus (grand sum minus that subset sum) over n_c. Ten thousand replications then take seconds instead of a minute.
Use the two-sided p-value (1 + count of permuted absolute differences at or above the observed) divided by (B + 1). The plus one is not cosmetic: it makes the p-value valid rather than optimistic, and it means the smallest reportable value here is 1/10001, not zero.
For the two-part version, run the same machinery on the 0/1 indicator for the rate, then on the non-zero subset for the conditional mean, and say plainly that conditioning on a post-treatment outcome breaks the randomisation, so the conditional arm is descriptive rather than causal.
Report the rate test and the overall mean test as the result, with the conditional mean as colour, and give the effect size in hours beside each p-value, because at 80 thousand accounts almost anything is detectable.

Worked solution (30 min)

Pull hours into a float array and arm into a boolean array; record n_t, n_c, the grand sum and the observed difference of means.
Loop B = 10,000 times, taking np.random.permutation(n)[:n_t] as the treated index set and accumulating hours[idx].sum().
Convert each shuffled treated sum into a difference of means, compare its absolute value against the observed, count, and apply the plus-one form.
Repeat with hours replaced by (hours > 0).astype(float) for the rate test.
Subset to hours above zero, re-run for the conditional mean, and assemble a three-row result carrying effect, p-value and the unit each is defined on.

Expected result: Three rows: the mean-hours difference with a permutation p-value bounded below by 1/10001; the streaming-rate difference in percentage points with its own p-value; and the conditional mean difference among non-zero accounts, labelled descriptive because it conditions on a post-treatment outcome.

Verify it yourself

The permutation distribution of the mean difference is centred on zero within its own Monte Carlo error. A centre away from zero means the labels were not actually being reshuffled between replications.
The permutation p-value on the rate agrees to two decimal places with a two-proportion z-test, which is a fair cross-check because a 0/1 indicator is not heavy-tailed.
Doubling B leaves the p-value stable to within roughly 1 over sqrt(B). A p-value that moves a lot between runs is not yet resolved.
Multiplying every hours value in both arms by 60 leaves all three p-values identical under the same seed and scales only the effect sizes, since a permutation test is invariant to a common positive rescaling.

Follow-up

The permutation p-value on the mean is 0.03 and the rate test is flat. What is the most likely explanation, and does it change the decision?
How would CUPED on pre-period hours change your power here, and what would disqualify a covariate?
Accounts are households. Does that affect the validity of this test, or only its interpretation?

Duration-decile-weighted completion rate with fixed reference weights

Hard · metric-definition · standardisation · pandas

Implement this metric. Numerator: qualified streams with completion_ratio at or above 0.9. Denominator: qualified streams with a non-null duration_seconds, so live events are out. Compute the rate inside each (content_type, duration decile) cell, then aggregate with catalogue-mix weights fixed from a reference month. You get streams and content for the last eight weeks plus ref_streams for the reference month. Return the weighted index and the unweighted global rate for each of the eight weeks, and the share of reference weight your cells actually covered.

Approach

Fix the decile boundaries from the reference month, within content_type, over that month's qualified streams. Not over the catalogue, and not per week: the weights and the cells have to be defined on the same population or the weighted sum is adding rates over cells the weights do not describe.
Store the boundaries explicitly and bin every week against them with pd.cut, with open-ended outer edges, so a duration longer than anything in the reference month still lands in the top cell instead of becoming NaN and quietly leaving the denominator.
Weights are the reference month's share of qualified streams per (content_type, decile) cell, summing to one across all cells. Apply them to each week's cell rates and report the covered weight separately, because a week missing a cell entirely gives a renormalised index, and renormalising silently is how the series gains a step change nobody can explain.
Keep the unweighted rate beside it. The pair is the deliverable: the weighted line is the answer, and the gap between the two is the size of the mix effect you removed, which is the first thing anyone reading it will ask about.
Sanity-test the whole construction by feeding the reference month back in as the current week; the weighted and unweighted rates must then be identical to floating-point error.

Follow-up

The weighted index is flat and the unweighted rate fell four points. What shipped?
When would you refresh the reference month, and what do you owe the series when you do?
Podcast episodes and film have very different completion shapes. Would you ever report one number across them at all?

Write a SQL query to find the top N users by engagement metrics.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Write a function to calculate the Fibonacci sequence.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Compute rates by summing numerator and denominator separately, never by averaging rates.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Pick one canonical version per work without losing remaster plays

Medium · deduplication · window functions · dimension grain

dim_content_version has content_version_id, content_id (stable across versions of the same work), title, content_type, duration_seconds, catalogue_added_ts, is_active_version and updated_at. A remaster, re-edit, dub or re-encode is a new row sharing content_id, and more than one row per content_id can carry is_active_version = true. Produce exactly one canonical row per content_id, preferring is_active_version = true, then the latest catalogue_added_ts, then the highest content_version_id as a deterministic tie-break. Then join fct_stream (content_version_id, played_seconds, is_qualified, started_at) and report qualified hours per content_id for last month.

Approach

ROW_NUMBER() OVER (PARTITION BY content_id ORDER BY is_active_version DESC, catalogue_added_ts DESC NULLS LAST, content_version_id DESC), keeping rn = 1. The third sort key exists only so two runs return the same row; without it the winner among ties is whatever the engine happened to produce, and the result diffs against yesterday for no reason.
Spell NULLS LAST out. In PostgreSQL, DESC defaults to NULLS FIRST, so a version with no catalogue_added_ts would win the canonical slot ahead of every dated one. Other engines order NULLs differently, which is exactly why the clause belongs in the query rather than in your head.
Aggregate the streams first, at content_id grain, by joining fct_stream to dim_content_version on content_version_id and grouping on content_id. Every version's plays belong to the work.
Only then attach the canonical attributes, with a join from the aggregate to the rn = 1 CTE on content_id. Inner-joining fct_stream to the canonical row instead discards every play of every non-canonical version, which is most of the catalogue's remasters and all of its dubs.
Report the count of content_ids carrying more than one is_active_version row as a data-quality line beside the result, since that condition is what made the dedup necessary and its size decides whether this is a query fix or a pipeline fix.

Worked solution (25 min)

Size the problem before solving it: count content_ids with more than one row, and separately those with more than one is_active_version = true row.
Write the ROW_NUMBER CTE with explicit NULLS LAST, keep rn = 1, and assert COUNT(*) equals COUNT(DISTINCT content_id) in the dimension.
Aggregate fct_stream to content_id through the version dimension, filtering is_qualified and last month, as SUM(played_seconds)/3600.
Join the canonical attributes on and compare the grand total against the same aggregate computed at version grain.

Expected result: One row per content_id with at least one qualified stream last month, carrying title and duration from the canonical version and qualified_hours summed over all versions of the work. The grand total equals the ungrouped total over the same stream filter.

Verify it yourself

COUNT(*) from the canonical CTE equals COUNT(DISTINCT content_id) in dim_content_version.
Rerunning the CTE returns an identical set of content_version_ids, confirming the tie-break made it deterministic.
Total qualified_hours matches the version-grain total to the second; any shortfall is the dedup-then-join bug.

Follow-up

Which grain should hours be reported at, content_id or content_version_id, and who is the consumer of each?
Rights expire per version, so one version's window can be closed while another is live. What does your canonical pick then mean for an availability report built on the same CTE?

How do you prioritize your tasks when working on multiple projects?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you ensure the integrity of your data during analysis?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

What steps would you take to identify the root cause of a sudden drop in user engagement?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you approach building a recommendation system for a new product?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

Walk me through your process for designing an experiment to test a new feature.

Medium · a/b testing and experimentation

Approach

Name the randomisation unit first; it decides the variance and what the test can detect.
Say whether units interfere with each other, and switch design if they do.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

Explain the differences between supervised and unsupervised learning.

Medium

Approach

Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.
Work from the decision backwards to the evidence you would need.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Value the home slate when the counterfactual is unobservable

Medium · metric design · proxy bias · position bias · recommendation evaluation

Define how you would measure what the algorithmic home slate is worth. You have fct_impression (impression_id, slate_id, slate_position, surface, viewport_visible_ms, ranker_version, is_exploration_slot, logging_propensity, was_clicked) and fct_stream (impression_id, start_source, started_at, played_seconds, is_qualified). Deliverable: name the quantity you actually care about, say plainly whether it is observable, then give the proxy you would report, its direction of bias with the reasoning, and what would have to be true for the proxy to be unbiased.

Approach

Name the estimand precisely and then concede it: incremental qualified hours caused by the slate, defined as hours consumed with the slate minus hours the same accounts would have consumed without it in the same window. The second term never appears in any table, so no query answers this and the honest first sentence says so.
Give the naive proxy and its bias direction: qualified streams with start_source = 'algorithmic_slate' joined to fct_impression on impression_id within a fixed attribution window, which is biased upward because it counts every play the account would have reached through search, resume or a direct link, and the upward bias is largest exactly for head content the account already intended to play.
Give the better proxy and state its precondition rather than assuming it: a propensity-weighted play rate computed only on rows with is_exploration_slot = true, where logging_propensity is populated, which is unbiased for the logging policy only if propensities are known, every item had non-zero probability of being shown, and no unlogged feature drove both exposure and play. Outside the exploration slots positivity fails silently, so the estimate does not extend to items the logger never demotes.
Bound the naive proxy instead of abandoning it: compare slate-attributed qualified hours against the same accounts' non-slate hours in a matched pre-period, which gives an upper and a crude lower figure and turns a single unfalsifiable number into a range with a stated interpretation.
State what each method cannot say: an offline ranking metric says nothing about downstream retention, interleaving picks a winner between rankers without producing an hours or revenue figure, and only a slate-off arm answers the value-of-the-slate question, which is usually not shippable, so the answer that goes in the memo is the exploration-slot estimate plus the naive range with the bias direction written next to it.
Pin the attribution mechanics so the proxy is at least reproducible: a fixed window from rendered_at to started_at, a documented tie-break when the same content_version_id appears in two slates, and a requirement that viewport_visible_ms is above zero before an impression counts as exposure at all.

Worked solution (30 min)

Write the estimand as a difference of two terms and mark which term exists in the warehouse and which does not, in one line each.
Write the naive attributed-hours metric fully: join fct_stream to fct_impression on impression_id, restrict to is_qualified and start_source = 'algorithmic_slate', fixed window, viewport_visible_ms > 0, and state the bias direction as upward with the reason in the same sentence.
Write the propensity-weighted estimator on is_exploration_slot rows and list its three preconditions explicitly: known propensities, positivity, and no unlogged confounder of exposure and play.
Construct the bounding comparison: matched pre-period non-slate hours for the same accounts, and state what the resulting range does and does not license you to claim.
Write the two or three sentences that would go in the memo, with the bias direction attached to the number rather than in a footnote.

Expected result: The estimand is incremental qualified hours attributable to the slate, which is not observable because the no-slate term does not exist. Reported proxy: propensity-weighted play rate on exploration slots, unbiased only under known propensities, positivity and no unlogged confounding, presented alongside slate-attributed qualified hours labelled as an upper bound, with the matched pre-period comparison supplying the range.

Verify it yourself

The answer contains an explicit sentence saying the quantity of interest is not measurable from logged data, before any proxy is offered.
Each proxy carries a bias direction, and the direction is justified by a mechanism rather than asserted.
The three preconditions for the propensity-weighted estimate are named, and positivity is identified as the one that fails silently outside exploration slots.
The attribution rule fixes a window, a tie-break and an exposure condition, so two analysts computing it get the same number.

Follow-up

How large would you make the exploration slot fraction, on which surface, and how would you bound its cost in engagement before proposing it?
Slate-attributed hours rise 8 percent after a ranker change while total qualified hours are flat. What is the most likely explanation and which metric distinguishes it?
Your propensity-weighted estimate has an enormous variance because a few rows carry small logging_propensity. What do you do, and what does the fix cost you in bias?

Trial conversion fell for cohorts that have not matured

Hard · cohort-maturity · mix-shift · acquisition-quality

Trial-to-paid conversion — cohort accounts with a period_index = 1 row in fct_subscription_period whose payment_status is 'paid' or 'retried_paid', over cohort size — is reported weekly by trial_start_ts cohort and has fallen from 38% to 29% across five weeks. Trial length is 14 days. You have dim_account (trial_start_ts, signup_channel, signup_platform, signup_country, is_test_account, first_paid_ts) and fct_subscription_period (account_id, period_index, payment_status, dunning_attempts, period_start_ts). Establish whether conversion fell and for whom. Deliverable: an age-aligned series plus a composition-adjusted estimate of the real change.

Approach

Align cohort age first. No period_index = 1 row can exist before the trial ends, so cumulative conversion is identically zero through day 13 and the whole curve lives between day 14 and the close of the dunning horizon — roughly 14 plus 21 days. The three most recent cohorts sit inside that window and cannot read anything but low. Plot each cohort's cumulative conversion against days since trial_start_ts and read every cohort at one common age.
Quantify how much maturity alone explains. Take the conversion-by-age curve from fully matured cohorts, read off the fraction of final conversion it has reached at each age, and multiply that fraction by a mature cohort's final rate to predict what a young cohort should be showing today. Whatever the prediction reproduces is not a regression and should never have been plotted as one.
Decompose the residual across signup_channel, signup_platform and signup_country. Paid social and carrier bundles convert at very different rates from organic, so a weight shift between them moves the blended figure with every channel's own rate flat.
Make the composition adjustment explicit rather than rhetorical: reweight recent cohorts to the channel-by-country mix of a fixed reference cohort and recompute, then publish the reweighted series beside the raw one and name which channels moved and by how much.
Only then look for a within-channel product cause, and rule out the instrumentation confounders first: is_test_account leakage, attribution opt-out inflating the organic bucket at a platform-specific rate, and app-store payment_status rows arriving later than card-direct ones.
Deliver a number with an interval rather than a direction: the age-aligned, composition-adjusted change, with the cohorts too young to read named explicitly as excluded.

Follow-up

Attribution opt-out shifts accounts into the unattributed bucket at a platform-specific rate. How does that corrupt your reweighting, and what do you do about it?
If the residual concentrates in one paid channel, what do you need from marketing before calling it acquisition quality rather than a product regression?
How many mature cohorts do you need before the composition-adjusted change clears the noise floor?

Suggested practice checklist

For someone who has spent the last year in notebooks, dashboards or modelling work and has not written raw SQL under time pressure. The first four days rebuild query fluency against a fixture you control and can verify by hand; the last three attach that fluency to the rest of the loop.

Day 1 — Build a fixture you can check answers against

Create a local Postgres or SQLite database with four tables (users, sessions, events, orders) holding roughly 200 rows you generated yourself, so you know the contents well enough to predict every result.
Deliberately seed the cases that break queries: a user with no sessions, a session with no events, two orders sharing a timestamp, a NULL in one join key, and one duplicated user row.
Before writing any SQL, hand-compute five answers on paper (how many users placed at least one order, median orders per ordering user, and three others) and save them as the ground truth for the week.

Deliverable: A one-command seed script plus a text file of five hand-computed answers to grade every later query against.

Day 2 — Joins, filters and NULL semantics

Answer "which users have no orders" three ways (LEFT JOIN with IS NULL, NOT EXISTS, NOT IN) and confirm that the NOT IN version returns zero rows once the subquery contains a NULL, because the comparison is never TRUE.
Reproduce the LEFT JOIN that silently collapses to an inner join by putting a right-table predicate in WHERE, then fix it by moving the predicate into the ON clause, and record both row counts.
Create a fan-out bug on purpose by joining orders to order_items and summing the order total, then correct it with a pre-aggregated subquery and explain in one line which table changed the grain.

Deliverable: One annotated .sql file holding the three join traps, each with the wrong result and the corrected result side by side.

Day 3 — Window functions and frames

Write three window queries against the fixture: a running order total per user, the rank of each order within its user by value, and the day gap to that user's previous order, then check each against the day-one ground truth.
Run ROW_NUMBER, RANK and DENSE_RANK over a column containing ties, print all three side by side, and write one sentence on when each is the correct choice.
Switch one query from the default frame (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, which is what you get when ORDER BY is present and no frame is written) to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and explain why the output differs only when the ORDER BY column has duplicates.

Deliverable: Three verified window queries plus a short note explaining the RANGE versus ROWS difference in your own words.

Day 4 — The four analytical query patterns

Write a monthly retention grid: first order month per user, then months-since-first as the column, and verify that month zero equals the cohort size exactly.
Sessionize the events table under a 30-minute inactivity rule using LAG plus a cumulative sum over a new-session flag.
Build a four-step funnel that counts distinct users rather than events at each step, and state the rule you applied to a user who reaches step three without ever logging step two.

Deliverable: One file with the retention, sessionization and funnel patterns, each carrying a one-line note on the assumption it bakes in.

Day 5 — Write SQL the way you will have to write it live

Set a 12-minute timer and solve three medium prompts in a plain editor with no execution and no autocomplete, then run them and tally syntax errors separately from logic errors.
Narrate one solution aloud while writing it, stating the grain of each intermediate result (one row per user, one row per user-day) before you type its body.
Rewrite your slowest solution as a CTE chain where every CTE name states its grain, and time yourself re-solving it from blank.

Deliverable: A recording of one narrated solution plus an error tally that separates syntax from logic.

Day 6 — One day for everything that is not SQL

Write the preconditions of the two-sample t-test from memory, then check them: independent observations, and a difference in means whose sampling distribution is approximately normal, which at large sample sizes follows from the central limit theorem rather than from normality of the raw values.
Write the difference between an odds ratio from logistic regression and a relative risk, and state the condition under which the two are close (low outcome prevalence).
Prepare a 90-second answer to "how would you know this model is any good" that names the metric, the baseline you would beat, and the cost of the errors you care about.

Deliverable: One page of notes covering test preconditions, the odds-ratio caveat and the model-quality answer.

Day 7 — Full loop rehearsal

Run a 45-minute mock with someone willing to interrupt: 20 minutes of SQL, 15 minutes defining a metric, 10 minutes on a past project.
Re-solve from blank the two queries you were slowest on this week and compare the times against day five.
Write a five-line answer to "walk me through a project" that puts a number in the first sentence and names the decision the work changed.

Deliverable: Mock feedback notes plus a timed project narrative you can deliver without reading it.

Your part, not the team's

Most data work is done by groups, so an interviewer has to work out which piece was yours. An answer that runs on 'we' for several minutes gets interrupted with a question about what you personally did, and by then the answer sounds defensive even when it is true. Mark your own contribution as you go, and name the parts that belonged to someone else instead of leaving them ambiguous. Keep a few specifics back as well, like the name of the metric or who actually objected, so a probe can be answered with something you had not already said.

Describe a challenging project and how you managed the difficulties.

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
Name the disagreement or constraint, and how you resolved it with evidence.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

Scope a one-line request about home row performance

Easy · scoping · stakeholder communication · metric definition

A director messages you: "Is the new home row working?" Nothing else. A new ranker_version has been serving a fraction of profiles for eleven days. You have fct_impression (surface, slate_position, ranker_version, is_exploration_slot, logging_propensity, experiment_assignment_id, was_clicked) and fct_stream (impression_id, start_source, is_qualified, played_seconds). You get a fifteen-minute call before they go into a rollout meeting. Deliverable: the three questions you ask before writing any SQL, the single primary metric you commit to with its guardrail, and the questions you tell them this data cannot answer.

Approach

The probe is whether you convert a vague request into a decision before producing a number. Ask what happens at each answer — rollback, widen, iterate — because a question whose answer changes nothing is a report request and should be scoped as one.
Pin the unit of analysis out loud. fct_impression is at (profile, slate, slot) grain and experiment_assignment_id is per assignment, so the comparison must be aggregated to the assignment unit first; comparing impression-level rates lets a change in slate length move the metric on its own.
Commit to one primary metric from the tree — qualified hours per active account-week for assigned accounts — and name the guardrail pair explicitly: share of qualified streams with start_source = 'autoplay_continuation', and median completion_ratio within content_type. A ranker can lift qualified stream counts by queueing short items that clear the 30-second threshold, and the guardrail is the only thing that catches it.
State the refusals with structural reasons, not time reasons: eleven days gives no matured cohort, so month-6 retention and net revenue per active account-month are unanswerable; and logging_propensity is populated only where is_exploration_slot = true, so the positivity condition for an off-policy estimate fails outside those slots.
Write the scope back in one paragraph — decision, metric, guardrail, the date the read becomes valid — and get it agreed in the thread before querying, so the number that arrives is the number that was asked for.

Follow-up

They reply "just give me click-through by slate position." What do you say, and what would that number actually tell them?
The eleven days include a weekend and a large release landing on day six. Does that change the metric you commit to, or only the read date?
What would have to be true for you to be willing to answer the retention question from this experiment?

Defend a flat result on a flagship discovery launch

Medium · defending findings · statistical power · experiment readout

A new ranker ran to 50 percent of accounts for four weeks. Your read: qualified hours per active account-week is +0.4 percent with a 95 percent interval of [-0.9 percent, +1.7 percent]. Meanwhile the share of qualified streams with start_source = 'algorithmic_slate' rose six points and the share with start_source = 'search' fell five. The team reads the slate shift as the win and wants to ship. The launch review is Friday. Deliverable: a five-minute verbal position and the one table you put on the screen.

Approach

The probe is whether you can hold a position under social pressure without overclaiming in the other direction. Separate the two claims explicitly: the start-source shift is well measured and real; the hours effect the launch was justified on is not distinguishable from zero at this sample size. Those are different statements and only one is contested.
Do the power arithmetic before the meeting, using the observed per-account variance on a right-skewed hours metric. Arrive able to say "at this n we could not have detected less than X percent", which is a fact, instead of "it didn't work", which is an opinion the room can simply disagree with.
Show the substitution as arithmetic, not interpretation: decompose treatment hours by start_source and show slate hours rising by approximately what search hours fell. A start-source mix shift with no change in total is exactly what displacement looks like, and the decomposition sums to the total so there is nothing to argue about.
Refuse the symmetric overclaim. The interval's upper bound is commercially meaningful, so "the ranker does nothing" is not supportable either; say that out loud, because it is what makes the rest of your position credible.
Offer the falsifiable next step with a price: a longer read powered for the hours effect, or CUPED on a pre-period consumption covariate measured before assignment, with the required account-weeks stated. Put the disagreement on the pre-registered metric, not on the team's judgement.

Follow-up

The PM argues start-source mix is a leading indicator of retention. How would you test that claim, and how long would it take?
Suppose the interval had been [+0.1 percent, +1.9 percent]. Does your position change, and by how much?
The decision goes against your read and the ranker ships. What do you do on Monday?