Interview guides · Curinos

Curinos Data Scientist Interview Questions & Guide 2026

A two-week prep plan for Curinos data scientist candidates: write your answer before reading the solution, revisit misses three days later.

The 60-second brief

As a Data Scientist at Curinos, you play a pivotal role in transforming data into actionable insights that drive strategic decisions and enhance business performance. This position is crucial for developing advanced analytical solutions that impact various product offerings and improve user experiences. Your work will directly influence how Curinos navigates the competitive landscape of financial services, enabling informed decision-making that aligns with the company's long-term goals.

When randomisation is off the table, the skill being checked is naming an identification strategy together with the assumption it rests on: parallel trends for difference-in-differences, relevance and exclusion for an instrument, overlap and conditional ignorability for matching. Say the assumption out loud and say how you would try to break it.

Focus areas

Split voluntary churn from involuntary payment failure
Correct discovery slates for position bias
Diagnose rebuffering by device, network and POP

Overview

In this role, you'll engage with complex datasets, leveraging statistical analysis and machine learning to uncover trends and patterns that inform product development and customer engagement strategies. You will collaborate with cross-functional teams, including product managers and engineers, to implement data-driven solutions that enhance product features and optimize operational efficiencies. The opportunity to work on diverse and challenging problems, coupled with the strategic importance of data in driving business success, makes this role both exciting and critical to Curinos.

The Curinos interview process

Curinos candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Assessments

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Final Interviews

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Comparing consumption week over week across the release calendar and the rights calendar.

A major release, a season drop or a live event produces a spike that dwarfs almost any treatment effect, and the effect is not confined to the new title because it pulls attention from everything else in the same window. Separately, licensed content leaves the catalogue when its window expires, so consumption falls with no product change and the drop is attributed to whatever shipped that week. Both need to be handled by an explicit control: a comparison period chosen for calendar equivalence, a covariate for scheduled releases, or a pre-registered rule for excluding a window, decided before the numbers are seen.

Counting plays without a qualification threshold, or changing the threshold without restating history.

Playback arrives as heartbeats, so a play only exists once you decide what counts, and the common 30-second convention is not a neutral analytics choice: in music it is also the boundary at which a play becomes payable, which makes the warehouse definition a payout definition. The threshold interacts violently with content length, so a catalogue of three-minute tracks and one of forty-minute episodes move in opposite directions when you change it, and a skip-heavy surface can add plays while adding no hours. Any metric mixing pre-threshold and post-threshold counts, or pooling short-form and long-form on a per-stream basis, moves by double digits for reasons that have nothing to do with the product.

Dropping rows with missing values without naming the mechanism

Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.

Reading experiment results before checking the arm split

Compare observed arm counts against the intended allocation ratio, not an assumed even split, and set the alarm far below the conventional 0.05: at 0.05 roughly one healthy experiment in twenty trips it, which is why sample-ratio checks usually run at p < 0.001 or stricter. The test's power scales with sample size, so it misses a real diversion on a small experiment and fires on an imbalance too small to move the estimate on a very large one. A flag means go find the assignment or logging fault before reading any outcome, not report a mismatch.

Questions & practice

Discuss the concept of overfitting and how to prevent it.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.
Say how the offline result would be validated online before it is trusted.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Walk us through your process for developing a predictive model.

Medium · machine learning and modelling

Approach

Say how the offline result would be validated online before it is trusted.
Check what information would not exist at prediction time, and exclude it.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Five invariant checks over a playback fact table

Easy · data-quality · pandas · validation

fct_stream arrives with stream_id, started_at, ended_at (nullable), played_seconds, max_position_seconds, completion_ratio, is_qualified, event_date, and duration_seconds joined from the content dimension (null for live events). Write one function returning a tidy frame with a row per rule: rule name, eligible rows, violations, violation share, one example stream_id. The five rules are duplicate stream_id; played_seconds exceeding wall-clock ended_at minus started_at; max_position_seconds exceeding duration_seconds; completion_ratio disagreeing with max_position_seconds divided by duration_seconds; and event_date not equal to the UTC date of started_at.

Approach

Give every rule its own eligible-row mask before its violation mask. Rules 2 to 4 are undefined where ended_at or duration_seconds is null, and a comparison against NaN evaluates False, so a mostly-null column otherwise reports a clean bill of health.
Compare the recomputed completion_ratio with a tolerance (absolute difference above 1e-6), never with ==. It is stored as a float division and exact equality fails on rows that are correct.
Allow a small tolerance on rules 2 and 3 too: a final segment can carry the playhead a second or two past duration_seconds, and heartbeat timestamps come off client clocks. State the tolerance you chose rather than burying it.
Return violations and eligible rows as separate columns so the share has a stated denominator, and carry one example stream_id per rule so the output is actionable rather than a number.
Read rule 5 as a signal rather than corruption: event_date drifting from the UTC date of started_at is the signature of offline playback uploaded after the partition closed, and its distinct dates are the recompute list.

Worked solution (20 min)

Write a helper taking (name, eligible_mask, violation_mask) and returning one result row, which forces every rule to declare its eligible population.
Rule 1: duplicated('stream_id', keep=False), eligible is all rows. Rule 5: event_date not equal to started_at.dt.date, eligible is all rows.
Rules 2 to 4: eligible masks are ended_at.notna() and duration_seconds.notna(); violation masks are played_seconds above wall_seconds plus 2, max_position_seconds above duration_seconds plus 2, and the absolute completion_ratio difference above 1e-6.
Take one example stream_id per rule with .loc[mask].head(1), guarding the empty case so a clean rule does not raise.
Concatenate the five rows and sort by violation share descending.

Expected result: A five-row frame with columns rule, eligible_rows, violations, violation_share and example_stream_id, where eligible_rows for rules 2 to 4 is strictly below the table length whenever ended_at or duration_seconds contains nulls.

Verify it yourself

Injecting one row whose played_seconds is ten times its wall clock raises exactly rule 2's count by one and leaves the other four unchanged.
Setting duration_seconds to null on that same row moves it out of the eligible counts for rules 3 and 4, rather than into their violation counts.
violation_share is violations divided by that rule's eligible_rows, not by len(df); the two differ for rules 2 to 4.
Rule 1's violation count is even when duplicates arrive in pairs, because keep=False flags both copies.

Follow-up

Rule 5 fires on 0.4 percent of rows, all from one app_version, all with reported_at days after started_at. Is that a bug, and what do you do about the daily numbers already published?
How would you turn these into a blocking check in the pipeline without failing the load every time one client version misbehaves?
Which of the five would you expect to fire on live events specifically, and how do you keep them out of the denominator?

Paid retention triangle with monthly and annual curves kept apart

Medium · cohorts · retention · subscriptions

fct_subscription_period has subscription_period_id, account_id, period_index, billing_interval, period_start_ts, period_end_ts, payment_status, cancel_requested_ts and renewal_outcome. dim_account has account_id, first_paid_ts and is_test_account. Build a paid-retention triangle: label non-test accounts with the calendar month of first_paid_ts, and for month offsets 0 through 6 report the share of the cohort still holding a period with payment_status in ('paid','retried_paid') that contains the instant first_paid_ts + offset months. The anchor is each account's own first payment, not the first day of the offset calendar month. Report monthly and annual billing_interval as separate curves. Exclude cohorts not yet mature at offset 6.

Approach

Take the cohort key from dim_account.first_paid_ts, not from MIN(period_start_ts). The period table includes trial periods as zero-amount rows with payment_status = 'trial_no_charge', so the minimum start shifts trialling accounts a cohort early and inflates the youngest cohort's offset-0 denominator.
Anchor every offset on that same first_paid_ts; the calendar month is only the row label on the triangle. Anchoring survival on the first day of the offset month instead breaks offset 0 for everyone who did not first pay on the 1st — an account that first paid on the 20th holds no paid period covering that month's 1st, so its offset-0 cell reads as churned and the curve rises from offset 0 to offset 1. A retention chart whose first step goes up is almost always this. Two conventions to write down while you are here: PostgreSQL clamps timestamp + INTERVAL '1 month' at month end, so an account first paid on the 31st is tested on the 28th or 30th at some offsets; and the whole construction assumes first_paid_ts falls inside its own first paid period, which fails if the platform charges shortly before period_start_ts, in which case anchor on that period's period_start_ts instead.
Build the offsets 0..6 as an explicit ordered set and cross-join it to the cohort list. A grid built by aggregating survivors alone loses any (cohort, offset) cell with no survivors, so a collapsing cohort reads as a missing row instead of a zero.
Test survival as interval containment, not date equality: EXISTS a period for that account with payment_status in ('paid','retried_paid') where first_paid_ts + offset months falls inside [period_start_ts, period_end_ts). One annual row contains seven consecutive anniversaries on its own; an equality join against period_start_ts finds it at offset 0 and nowhere else.
Partition everything by billing_interval and never pool. An annual account has had no opportunity to churn before day 365, so pooling makes an annual-heavy cohort read as retentive when what it actually is, is un-renewed. billing_interval lives on the period and not on the account, so take the cohort's interval from the period containing first_paid_ts and hold it fixed for all seven offsets; an account that switches monthly to annual at offset 4 otherwise appears in both curves and is counted twice in the denominators.
Apply maturity last, and against the anchor rather than the label: keep a cohort only when the last day of that cohort month plus six months is at or before the last fully closed day in the data, so every account in it has actually reached its offset-6 anniversary. Otherwise the newest cohorts print 0% at the far offsets for a reason that is entirely calendar.

Worked solution (40 min)

Build the cohort list as (cohort_month, billing_interval, account_id, first_paid_ts) over non-test accounts, taking billing_interval from the period that contains first_paid_ts, and record each cohort's size.
Cross-join cohorts to offsets 0..6 to produce the dense grid before any survivor logic touches it.
Attach a survivor flag per (account_id, offset) with the EXISTS containment test at first_paid_ts + offset months, then aggregate to a share within (cohort_month, billing_interval, offset).
Read offset 0 before anything else: it must be 1.0 in every cell. Where it is not, list those accounts' first_paid_ts beside their first period's [period_start_ts, period_end_ts) — the anchor and the period data disagree, and the fix is in the anchor, not in the survival test.
Apply the maturity filter and pivot offsets into columns so the triangle can be read a row at a time.

Expected result: A grid of (cohort_month, billing_interval, offset 0..6) retention shares. Offset 0 is 1.0 in every cell, because the anchor is the account's own first payment and the period containing it is paid by definition; the annual curves sit at 1.0 through offset 6 unless a period's payment_status was later reversed, since no annual account has reached a renewal decision by month six.

Verify it yourself

Offset 0 equals 1.0 in every cell. Anything less means first_paid_ts falls outside the first paid period it belongs to — a charge recorded shortly before period_start_ts is the usual cause — and the anchor moves to that period's period_start_ts.
Cohort sizes sum to the distinct count of non-test accounts with a first_paid_ts in the covered months, and no account appears in both the monthly and the annual curve.
Each monthly curve is non-increasing in offset unless win-backs are in scope, in which case you can point to the specific rows that produce the rise.

Follow-up

An account migrates plan mid-cohort with renewal_outcome = 'migrated_plan'. Retained or churned, and what does your choice do to the revenue story told beside this chart?
A period fails and a retry succeeds as a new row. Does your survival test find the account at that offset, and should it?
Can you produce a curve for annual accounts that is comparable to the monthly one before twelve months have elapsed, and what do you give up?

Home-row impressions that never converted, and an empty result

Medium · anti-join · null semantics · attribution

fct_impression has impression_id, profile_id, content_version_id, surface, slate_position, rendered_at and viewport_visible_ms. fct_stream has stream_id, profile_id, content_version_id, started_at, is_qualified and a nullable impression_id, populated only when the play started from a rendered slate. For surface = 'home_row' impressions rendered yesterday with viewport_visible_ms > 0, return content_version_id and the number of impressions that produced no qualified stream within 30 minutes. A colleague's draft filters WHERE impression_id NOT IN (SELECT impression_id FROM fct_stream) and returns zero rows on data that plainly contains misses. Explain why, then write the correct query.

Approach

Name the mechanism rather than the symptom. fct_stream.impression_id is nullable, so the subquery returns a set containing NULL. In SQL's three-valued logic x NOT IN (..., NULL) evaluates to UNKNOWN whenever x matches nothing else, never to TRUE, so the WHERE clause admits no rows. This is correct engine behaviour on correct data.
Fix with NOT EXISTS, which is a correlated existence test and is NULL-safe by construction, or with LEFT JOIN ... WHERE s.impression_id IS NULL. Adding IS NOT NULL to the subquery also works but leaves the same landmine armed for whoever edits the query next.
Put is_qualified = true and the 30-minute window inside the join or the correlated predicate, never in an outer WHERE over a LEFT JOIN. An outer filter on a right-side column turns a LEFT JOIN back into an inner join and returns the exact complement of the set you were asked for.
Match on impression_id, not on (profile_id, content_version_id). The second is a many-to-many between two event tables and will count one stream against every impression of that title for that profile.
Keep viewport_visible_ms > 0. A row that was never scrolled into view is not a recommendation that failed; it is a recommendation that was never made, and including it depresses every title's conversion by whatever share of the row went unseen.

Follow-up

Two home rows on the same screen render the same title, and the stream carries one impression_id. Should the other impression count as a miss? Defend whichever rule you pick.
Would NOT EXISTS and LEFT JOIN ... IS NULL produce the same plan here, and is there a case where you would prefer one over the other?
What attribution window between rendered_at and started_at do you allow, and why does an unbounded window overstate conversion rather than merely add noise?

How would you optimize a data pipeline for performance?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

Given a dataset, how would you identify outliers and what actions would you take?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you approach a project to improve customer retention for a financial product?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

How do you prioritize tasks when managing multiple projects?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Design a switchback around a shared popularity feature

Hard · interference · switchback · spillover · clustered design

A ranker change boosts items using a global popularity feature computed from every account's streams over a trailing 24 hours. An account-randomised A/B is proposed. Treated accounts' plays feed that feature, so control accounts are shown items the treatment arm made popular. Design an experiment that identifies the effect on qualified hours. Specify the randomisation scheme, slice length, burn-in, the analysis unit and its standard error, and how you would measure the interference you cannot remove. The surface refreshes hourly and 74 percent of qualified hours fall between 17:00 and 24:00 local.

Approach

Name the violation: SUTVA fails because one account's assignment changes another's exposure through the shared feature. The account-randomised estimate is attenuated by an unknown factor, so a null cannot distinguish no effect from complete spillover.
Rule out cluster designs by reading the feature's partition key rather than by reaching for geography. Popularity computed globally has no containing cluster smaller than the whole population: a treated account in one territory moves the same feature value a control account in another territory reads. The general rule is that any cluster or saturation scheme must be randomised at or above the unit the shared feature is computed over, which here leaves only time.
Choose a switchback: randomise time, put the entire population on one arm per slice, and the shared feature becomes internally consistent within a slice. The price is that only outcomes resolving inside a slice are measurable, so this design reads hours and cannot read retention.
Set slice length against carryover rather than convenience. A 24-hour popularity window means each slice inherits up to 24 hours of the previous arm's influence, so burn-in must be at least 24 hours and a one-hour slice is meaningless. The engineering trade worth proposing is shortening the feature window for the duration of the test.
Block assignment against the daypart. With 74 percent of hours inside a seven-hour evening band, unblocked random slices will hand one arm more evenings than the other; randomise within blocks of complete days and verify each arm's realised share of evening and weekend hours.
Measure the residual spillover with a saturation ladder built on the same slice schedule, since time is the only unit above a global feature. Run some slices at intermediate treated shares and read control accounts' outcomes as a function of the slice's treated share: the slope is the spillover, and it is identified precisely because the feature is global.
Analyse at the slice, not the account. With tens of slices per arm, use a permutation test over the actual slice-assignment mechanism rather than a t-test on millions of correlated account rows, and allow for correlation between adjacent slices.

Worked solution (45 min)

Read the feature definition to confirm its partition and window rather than assuming: global versus per-territory decides whether cluster randomisation is even available, and whether a saturation design can be run on anything but time.
Cost the design under the native 24-hour window: a 24-hour burn-in forces 48-hour slices, so 28 days gives 672 / 48 = 14 slices, seven per arm, each contributing 24 measured hours. Write down explicitly that this is unreadable for anything but a very large effect.
Propose shortening the feature window to 4 hours for the test. A 4-hour burn-in then allows 8-hour slices: 672 / 8 = 84 slices over 28 days, each contributing 4 measured hours.
Allocate the 84 slices across treated shares rather than spending all of them on the two extremes: 24 slices at 0 percent treated, 24 at 100 percent, and 12 each at 25, 50 and 75 percent. The primary contrast is 0 versus 100 with 24 slices an arm instead of 42, which widens its MDE by sqrt(42 / 24) = 1.32. That 32 percent is the price of being able to measure the spillover at all rather than assuming it away.
Block the slices into consecutive day pairs so every treated share receives the same number of 17:00 to 24:00 bands and the same number of weekend bands, and verify the realised balance before unblinding.
Compute the slice-level outcome as qualified hours per active account inside each slice's measured portion, then estimate the 0 versus 100 contrast with a permutation test over 10,000 re-randomisations of the blocked assignment.
Regress control accounts' qualified hours per active account on the slice's treated share across the 0, 25, 50 and 75 percent rungs. The slope is the spillover the switchback exists to avoid, measured on the same schedule and the same metric.

Expected result: A blocked switchback rather than an account-randomised A/B. Under the native 24-hour feature window the design yields only seven slices per arm over 28 days and is not readable; shortening the feature window to 4 hours for the test gives 8-hour slices with a 4-hour burn-in and 84 slices over 28 days, analysed at the slice level with a permutation test. Twenty-four of those slices go to each extreme and 36 to intermediate treated shares, widening the primary contrast's MDE by a factor of 1.32 in exchange for a direct spillover estimate. The spillover is read as the slope of control accounts' hours against the slice's treated share, and it is measured on a time ladder rather than on territory clusters because a globally computed feature admits no saturation unit smaller than the whole population. The limitation stated up front is that this measures within-slice consumption and cannot measure retention.

Verify it yourself

An A/A switchback over the same slice schedule must reject at close to the nominal rate. Rejection above nominal means the burn-in is too short or adjacent slices are more correlated than the test allows.
Split each measured window into halves: after adequate burn-in the two halves must give the same estimate, and a gradient between them means carryover is still bleeding through.
Each arm's share of qualified hours inside 17:00 to 24:00 and its share of weekend hours must both sit within one percentage point of 50 percent across the realised schedule.
Control accounts' hours must be flat against the slice's treated share if the shared feature carries no spillover, and a positive slope across the 0, 25, 50 and 75 percent rungs is the spillover measured directly. Read a flat line only to the width of its confidence interval: 12 slices a rung is a diagnostic, not a precise estimate, so report the interval and say what magnitude of spillover it still admits rather than calling the null zero.
The ladder must sit on the same unit the feature is computed over. A saturation design run on territory clusters while popularity is global is uninformative by construction, because every cluster reads the same feature value whatever its local treated share, so a null there says nothing about spillover.

Follow-up

A stream starts at 07:55 and ends at 08:40 across a slice boundary. Write the rule, and say which direction your choice biases the estimate.
The change is also expected to move 30-day retention. Can this design say anything about that, and if not, what is the least-bad alternative?
Bound the attenuation in the account-randomised test instead: with 50 percent of accounts treated, what share of the popularity feature is treatment-derived, and what does that imply about the observed effect relative to the true one?

Trial conversion fell for cohorts that have not matured

Hard · cohort-maturity · mix-shift · acquisition-quality

Trial-to-paid conversion — cohort accounts with a period_index = 1 row in fct_subscription_period whose payment_status is 'paid' or 'retried_paid', over cohort size — is reported weekly by trial_start_ts cohort and has fallen from 38% to 29% across five weeks. Trial length is 14 days. You have dim_account (trial_start_ts, signup_channel, signup_platform, signup_country, is_test_account, first_paid_ts) and fct_subscription_period (account_id, period_index, payment_status, dunning_attempts, period_start_ts). Establish whether conversion fell and for whom. Deliverable: an age-aligned series plus a composition-adjusted estimate of the real change.

Approach

Align cohort age first. No period_index = 1 row can exist before the trial ends, so cumulative conversion is identically zero through day 13 and the whole curve lives between day 14 and the close of the dunning horizon — roughly 14 plus 21 days. The three most recent cohorts sit inside that window and cannot read anything but low. Plot each cohort's cumulative conversion against days since trial_start_ts and read every cohort at one common age.
Quantify how much maturity alone explains. Take the conversion-by-age curve from fully matured cohorts, read off the fraction of final conversion it has reached at each age, and multiply that fraction by a mature cohort's final rate to predict what a young cohort should be showing today. Whatever the prediction reproduces is not a regression and should never have been plotted as one.
Decompose the residual across signup_channel, signup_platform and signup_country. Paid social and carrier bundles convert at very different rates from organic, so a weight shift between them moves the blended figure with every channel's own rate flat.
Make the composition adjustment explicit rather than rhetorical: reweight recent cohorts to the channel-by-country mix of a fixed reference cohort and recompute, then publish the reweighted series beside the raw one and name which channels moved and by how much.
Only then look for a within-channel product cause, and rule out the instrumentation confounders first: is_test_account leakage, attribution opt-out inflating the organic bucket at a platform-specific rate, and app-store payment_status rows arriving later than card-direct ones.
Deliver a number with an interval rather than a direction: the age-aligned, composition-adjusted change, with the cohorts too young to read named explicitly as excluded.

Follow-up

Attribution opt-out shifts accounts into the unattributed bucket at a platform-specific rate. How does that corrupt your reweighting, and what do you do about it?
If the residual concentrates in one paid channel, what do you need from marketing before calling it acquisition quality rather than a product regression?
How many mature cohorts do you need before the composition-adjusted change clears the noise floor?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Getting an analysis acted on

Work that nobody used is a common and unflattering pattern in data careers, and interviewers probe for it. Have a story about an analysis that changed a decision, and be specific about how you got it in front of the person who could act. Also have one about work that went nowhere, with your reading of why.

Describe a time you faced a significant challenge in a team project. How did you handle it?

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Pick a story where you drove the decision, not one where you observed it.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Give an example of how you have influenced a decision in your previous roles.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Name the disagreement or constraint, and how you resolved it with evidence.
Quantify the outcome, including what you would not claim credit for.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Walk through an analysis you shipped that was wrong

Medium · error ownership · data quality · postmortem

Describe a case where you delivered a result that was later shown to be wrong, and it had already been acted on. Cover how the error surfaced, whether you or someone else found it, what the wrong number caused, and what you changed afterwards. Prepare an example whose root cause was a definition, a denominator or a join — not a transcription slip. The interviewer will push on the mechanism, not the apology. Deliverable: a four-minute account that ends with a specific control now running in a pipeline.

Approach

The probe is whether your account has a mechanism in it. Choose an error that generalises — a denominator that silently changed population, a join that fanned rows, a metric partitioned on event_date while offline playback arrived days late and landed in the wrong partition — rather than one that only teaches you to check your typing.
State the blast radius factually and early: which decision was taken, how long the number stood, what it cost. A candidate who softens this is answering a different and easier question, and the interviewer can hear the substitution.
Explain how it surfaced without adjusting who found it. The generalisable detail is why your own checks did not catch it, which is a statement about your checks rather than about your luck.
Name the control you added and where it now lives: a row-count assertion after the fan-out join, a reconciliation that recomputes a closed day after late-arriving offline playback and alerts above a threshold, a denominator assertion inside the query. A fix that lives in a pipeline is different in kind from a resolution to be more careful.
Close with whether the control has fired since, or how you tested that it would. That single sentence is what separates a fix from an intention, and interviewers ask for it when candidates do not offer it.

Follow-up

Why didn't your own review catch it? Be specific about what you did check.
What class of error would that control still not catch, and what would you add next?
Have you found an error in someone else's published analysis since? How did you raise it?