Interview guides · Hp Iq

Hp Iq Data Scientist Interview Questions & Guide 2026

Hp Iq Data Scientist interview guide: reported rounds, practice questions with worked solutions and a 7-day prep plan.

The 60-second brief

As a Data Scientist at Hp Iq, you will play a pivotal role in shaping the future of technology through data-driven insights. This position is critical for translating complex data sets into actionable strategies that enhance product offerings and improve user experiences. By leveraging statistical methods, machine learning algorithms, and data mining techniques, you will contribute to projects that have a direct impact on various products, including laptops, printers, and software solutions.

Most of the loop measures decision-making under uncertainty rather than recall. You are scored on whether you state your assumptions, commit to an estimate you can defend, and say explicitly what evidence would change it.

Focus areas

Separate release-calendar spikes from underlying trend
Define a qualified stream and defend it
Split voluntary churn from involuntary payment failure

Overview

Your work as a Data Scientist will not only influence the design and functionality of Hp Iq's products but also contribute to strategic decisions that drive the company's growth and innovation. The role demands a blend of technical expertise and creative problem-solving skills, allowing you to engage with complex challenges and collaborate with cross-functional teams, including engineering, product management, and operations. Expect to work on a diverse array of projects that are both intellectually stimulating and impactful, making this position both rewarding and essential.

The Hp Iq interview process

Hp Iq candidates report 6 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Interviews

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Problem-Solving Sessions

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Case Studies

A case has a fixed clock, and a good deal of what is being scored is how you spend it. Thirty to forty-five minutes buys one pass across the whole problem or a deep read of one part of it, and choosing between those is the work rather than a compromise forced on you. Announce the shape early: the structure you are using, the branch you think carries the decision, and what you are setting aside. An answer that is thorough for the first third and silent on the recommendation reads worse than one that is rougher throughout and lands.

What to demonstrate

Whether a visible structure appears in the opening minutes and survives the rest of the case
Whether the depth goes to the branch that carries the decision, rather than the branch you find most comfortable
Whether you say what you are leaving out and why, instead of quietly omitting it and hoping nobody asks

How to prepare

After each practice case, write down the branches you chose not to open and the reason for each, then check whether you said any of them out loud while the case was running. A branch you only cut privately reads to the interviewer as one you missed.
Redo a case you have already worked in half the time, deciding in advance which single branch you keep, then compare which version a listener would find more useful.
Write a two-sentence opening you can reuse, holding the restated question and your plan for the available time, and deliver it within the first ninety seconds of every practice run.

Coding Challenges

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Behavioral Discussions

This round decides whether you owned a decision or watched one happen nearby. Interviewers for data roles listen for the point where the analysis stopped being a report and started changing what someone did, so build each story around that hinge: what was going to happen by default, what you found, and what happened instead. The most common weakness is a story that ends at delivery. If you can name the decision your work changed and the number that moved because of it, most follow-ups become easy.

What to demonstrate

Whether the decision was yours to influence, or whether you are narrating a team outcome in the first person
The counterfactual: what would have been done without your analysis, and why that default was worse
How far your involvement ran past the handoff, and whether you checked that the change did what you predicted

How to prepare

Pick three projects and write one sentence for each naming the decision-maker, the choice in front of them, and what they chose after seeing your work. If you cannot name a person and a choice, the story is not ready for this round.
Reconstruct the baseline for your strongest project from the original query or dashboard rather than memory, so the before-number survives a follow-up asking where it came from.
Prepare an honest version of a project where your recommendation was overruled, including what you did with the analysis afterwards.

Where candidates lose points

Testing hours, revenue or completion with a difference in means on a heavy-tailed distribution.

Listening and viewing hours per account are strongly right-skewed and content popularity is close to power-law, so the variance of a sample mean is dominated by a few accounts and the central limit approximation converges slowly at realistic sample sizes. A t-test on mean hours can flip sign when one heavy account's week changes, and an experiment can appear significant because a single title released into one arm's window. Capping at a pre-registered percentile, or decomposing into a rate (did they stream at all) and a conditional intensity, controls the variance, at the stated cost that capping biases toward zero exactly when the true effect lives in the tail.

Treating the account as the person, or the profile as the person.

A household account carries several people, profiles frequently are not switched, and shared-screen, car and speaker playback often lands on a default profile with no user behind it. Personalisation trained on a profile therefore learns a mixture, retention regressions attribute one member's behaviour to another, and a per-account taste statistic describes a household composite. The practical consequence is that apparent personalisation wins can be device or context effects, so any identity-level claim needs a stated unit and an acknowledgement of what that unit actually aggregates.

Explaining an aggregate move without decomposing the mix shift

Split the change in the aggregate into within-segment movement and movement in segment weights before you explain it. Every segment's rate can fall while the overall rate rises, purely because volume shifted toward segments that already had higher rates.

Treating a non-significant result as proof of no effect

Say whether the confidence interval excludes the effect sizes you would have cared about. If it does not, the honest reading is that the test was underpowered, so report the minimum detectable effect the design could have found and what sample size would resolve it.

Questions & practice

How would you design an architecture for a machine learning pipeline?

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Set a baseline first, so any model has something honest to beat.
Check what information would not exist at prediction time, and exclude it.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

What are the key metrics you use to evaluate the performance of a machine learning model?

Medium · machine learning and modelling

Approach

Check what information would not exist at prediction time, and exclude it.
Set a baseline first, so any model has something honest to beat.
Say how the offline result would be validated online before it is trusted.

Follow-up

What would you monitor after launch to know the model is still valid?
How would you choose the decision threshold, and who owns that choice?

Sessionise playback heartbeats into streams under an idle-gap rule

Hard · sessionisation · gaps-and-islands · pandas

heartbeats has profile_id, device_id, content_version_id, heartbeat_ts, playhead_seconds, is_paused and reported_at; clients emit one every 30 seconds while in the foreground. Collapse it into stream rows keyed on (profile_id, device_id, content_version_id) under a 300-second idle gap. Emit started_at, ended_at, max_position_seconds and played_seconds, where played_seconds counts advancing playback only and so excludes pause, seek and rebuffer time. Flag streams whose reported_at trails heartbeat_ts by more than an hour. Then re-run at a 60-second gap and report what moved.

Approach

Sort by the key and heartbeat_ts, take the per-row time delta inside the key with groupby().diff(), mark a new stream where that delta is null or above 300 seconds, and cumulative-sum the flag into a stream ordinal. This is gaps-and-islands, and it is O(n log n) after the sort with no per-group Python loop.
Compute played_seconds from consecutive heartbeats as min(max(playhead_delta, 0), wall_delta), summed inside the stream. The lower clamp discards a backward seek; the upper clamp is what stops a forward seek to the end of a forty-minute episode from crediting forty minutes of playback to one thirty-second interval. Note what this makes the column mean: a rewind and replay inside one stream counts the replayed interval twice, because played_seconds measures time spent advancing the playhead, not distinct coverage of the item.
Accept that the first heartbeat of each stream has no predecessor and contributes nothing, which undercounts by up to one heartbeat interval per stream. Quantify that bias as stream count times 30 seconds against the total instead of leaving it unsaid.
Take ended_at as the last heartbeat_ts, but keep it null when that heartbeat sits within one idle gap of the data cutoff: such a stream may still be running, and giving it an ended_at manufactures a completed stream.
For the offline flag, work on reported_at minus heartbeat_ts rather than on event_date, and report the distinct dates the flagged streams belong to, because that list is the recompute list.
Re-run at 60 seconds and report both the stream count and the change in total played_seconds: the count can only rise, and the total falls by exactly the contributions that now straddle a split.

Worked solution (40 min)

Sort by (profile_id, device_id, content_version_id, heartbeat_ts); compute wall_delta as the grouped heartbeat_ts diff in seconds and pos_delta as the grouped playhead_seconds diff.
new_stream = wall_delta.isna() OR wall_delta above 300; stream_ordinal = new_stream grouped-cumsum; group on the key plus stream_ordinal from here on.
contrib = np.minimum(np.maximum(pos_delta, 0), wall_delta), forced to 0 on the first row of each stream; played_seconds is contrib summed per stream.
Aggregate started_at as min(heartbeat_ts), ended_at as max(heartbeat_ts), max_position_seconds as max(playhead_seconds), and late_offline as max(reported_at minus heartbeat_ts) above 3600 seconds.
Null ended_at where it falls within 300 seconds of the data cutoff.
Re-run with the gap as a parameter at 60 seconds, and diff the two summaries on stream count and total played_seconds.

Expected result: One row per stream whose played_seconds never exceeds ended_at minus started_at in seconds, since each interval contributes at most its wall_delta and the wall_deltas inside a stream telescope to exactly that span. At a 60-second gap the stream count is greater than or equal to the 300-second count, and total played_seconds is lower by exactly the sum of the contributions that now straddle a split.

Verify it yourself

Assert played_seconds at or below (ended_at minus started_at) in seconds for every stream; any violation means the clamp against wall_delta was dropped.
No stream spans two content_version_ids or two device_ids. Group the heartbeats back by stream ordinal and check nunique equals 1 on both.
A hand-built three-heartbeat case with a backward seek (playheads 600, 300, 330 at 30-second spacing) yields played_seconds of 30, not 60 (the wall clock) and not -270 (the raw playhead deltas).
The distinct event_date partitions of the late_offline streams are exactly the days that need recomputing, and that set should be small and old rather than spread evenly across the window.
Do not assert played_seconds against an item's runtime. The heartbeats input carries no duration_seconds, and the bound does not hold even if you join one: a backward seek followed by a rewatch advances the playhead over the same interval twice inside one stream, and both passes are genuine playback. max_position_seconds is the column that compares against a joined runtime, and then only within the segment-overshoot tolerance of a second or two.

Follow-up

A client dies mid-playback and stops reporting. What does its stream look like under this rule, and what does that do to mean stream duration?
Which of these numbers determines a per-stream payout, and what does moving the idle gap do to somebody's money?
The same profile plays the same track on a phone and a TV at once. Your key separates them. When would you want it not to?

Write a function to perform a join operation on two datasets.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Solve a LeetCode-style problem related to arrays or strings.

Medium · sql and data manipulation

Approach

Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Paid retention triangle with monthly and annual curves kept apart

Medium · cohorts · retention · subscriptions

fct_subscription_period has subscription_period_id, account_id, period_index, billing_interval, period_start_ts, period_end_ts, payment_status, cancel_requested_ts and renewal_outcome. dim_account has account_id, first_paid_ts and is_test_account. Build a paid-retention triangle: label non-test accounts with the calendar month of first_paid_ts, and for month offsets 0 through 6 report the share of the cohort still holding a period with payment_status in ('paid','retried_paid') that contains the instant first_paid_ts + offset months. The anchor is each account's own first payment, not the first day of the offset calendar month. Report monthly and annual billing_interval as separate curves. Exclude cohorts not yet mature at offset 6.

Approach

Take the cohort key from dim_account.first_paid_ts, not from MIN(period_start_ts). The period table includes trial periods as zero-amount rows with payment_status = 'trial_no_charge', so the minimum start shifts trialling accounts a cohort early and inflates the youngest cohort's offset-0 denominator.
Anchor every offset on that same first_paid_ts; the calendar month is only the row label on the triangle. Anchoring survival on the first day of the offset month instead breaks offset 0 for everyone who did not first pay on the 1st — an account that first paid on the 20th holds no paid period covering that month's 1st, so its offset-0 cell reads as churned and the curve rises from offset 0 to offset 1. A retention chart whose first step goes up is almost always this. Two conventions to write down while you are here: PostgreSQL clamps timestamp + INTERVAL '1 month' at month end, so an account first paid on the 31st is tested on the 28th or 30th at some offsets; and the whole construction assumes first_paid_ts falls inside its own first paid period, which fails if the platform charges shortly before period_start_ts, in which case anchor on that period's period_start_ts instead.
Build the offsets 0..6 as an explicit ordered set and cross-join it to the cohort list. A grid built by aggregating survivors alone loses any (cohort, offset) cell with no survivors, so a collapsing cohort reads as a missing row instead of a zero.
Test survival as interval containment, not date equality: EXISTS a period for that account with payment_status in ('paid','retried_paid') where first_paid_ts + offset months falls inside [period_start_ts, period_end_ts). One annual row contains seven consecutive anniversaries on its own; an equality join against period_start_ts finds it at offset 0 and nowhere else.
Partition everything by billing_interval and never pool. An annual account has had no opportunity to churn before day 365, so pooling makes an annual-heavy cohort read as retentive when what it actually is, is un-renewed. billing_interval lives on the period and not on the account, so take the cohort's interval from the period containing first_paid_ts and hold it fixed for all seven offsets; an account that switches monthly to annual at offset 4 otherwise appears in both curves and is counted twice in the denominators.
Apply maturity last, and against the anchor rather than the label: keep a cohort only when the last day of that cohort month plus six months is at or before the last fully closed day in the data, so every account in it has actually reached its offset-6 anniversary. Otherwise the newest cohorts print 0% at the far offsets for a reason that is entirely calendar.

Worked solution (40 min)

Build the cohort list as (cohort_month, billing_interval, account_id, first_paid_ts) over non-test accounts, taking billing_interval from the period that contains first_paid_ts, and record each cohort's size.
Cross-join cohorts to offsets 0..6 to produce the dense grid before any survivor logic touches it.
Attach a survivor flag per (account_id, offset) with the EXISTS containment test at first_paid_ts + offset months, then aggregate to a share within (cohort_month, billing_interval, offset).
Read offset 0 before anything else: it must be 1.0 in every cell. Where it is not, list those accounts' first_paid_ts beside their first period's [period_start_ts, period_end_ts) — the anchor and the period data disagree, and the fix is in the anchor, not in the survival test.
Apply the maturity filter and pivot offsets into columns so the triangle can be read a row at a time.

Expected result: A grid of (cohort_month, billing_interval, offset 0..6) retention shares. Offset 0 is 1.0 in every cell, because the anchor is the account's own first payment and the period containing it is paid by definition; the annual curves sit at 1.0 through offset 6 unless a period's payment_status was later reversed, since no annual account has reached a renewal decision by month six.

Verify it yourself

Offset 0 equals 1.0 in every cell. Anything less means first_paid_ts falls outside the first paid period it belongs to — a charge recorded shortly before period_start_ts is the usual cause — and the anchor moves to that period's period_start_ts.
Cohort sizes sum to the distinct count of non-test accounts with a first_paid_ts in the covered months, and no account appears in both the monthly and the annual curve.
Each monthly curve is non-increasing in offset unless win-backs are in scope, in which case you can point to the specific rows that produce the rise.

Follow-up

An account migrates plan mid-cohort with renewal_outcome = 'migrated_plan'. Retained or churned, and what does your choice do to the revenue story told beside this chart?
A period fails and a retry succeeds as a new row. Does your survival test find the account at that offset, and should it?
Can you produce a curve for annual accounts that is comparable to the monthly one before twelve months have elapsed, and what do you give up?

How would you communicate complex technical information to a non-technical audience?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize your tasks when facing tight deadlines?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

What methods do you use for feature selection?

Medium

Approach

Clarify what is being asked and what a complete answer would contain.
Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Define success before a startup-latency infrastructure change

Easy · metric design · quality metrics · guardrails · mix effects

An infrastructure change is expected to cut startup_latency_ms on tv_app playback. You must define success before the rollout. Available: fct_stream (startup_latency_ms, rebuffer_seconds, played_seconds, peak_bitrate_kbps, end_reason, start_source, device_type, cdn_pop, network_type). Propose one primary outcome metric, the latency statistic you will report and at which quantile, and two guardrails. State how you will stop a mix shift across device_type and cdn_pop from reading as an improvement, and name one way a latency win could be bought rather than earned.

Approach

Separate the input from the outcome out loud: startup_latency_ms is the thing the change moves, but success is whether consumption improved, so the primary metric is qualified hours per active account-week on tv_app and latency is the mechanism metric that explains it.
Choose the latency statistic for the shape of the distribution: startup latency is right-skewed with a long tail from cold caches and poor networks, so report p95 and p99 rather than a mean, because the mean moves with tail mass that a handful of streams contribute and the tail is where abandonment lives.
Kill the mix effect by construction: compute the latency quantiles inside (device_type, cdn_pop, network_type) cells and aggregate with cell weights fixed from a pre-period, so that a rollout reaching wifi and well-provisioned points of presence first cannot show a win that is entirely composition.
Guardrail one on how a latency win is bought: peak_bitrate_kbps and the rebuffer ratio, sum(rebuffer_seconds)/sum(played_seconds) with played_seconds = 0 rows excluded from both sides, because starting playback at a lower ladder rung lowers time to first frame and costs picture quality instead.
Guardrail two on what the ratio hides: start-failure rate over streams with end_reason = 'playback_error' and played_seconds = 0, counted separately, since a point of presence that fails before first frame contributes to neither side of the rebuffer ratio and would otherwise report perfectly.
State the exclusion: cut or exclude start_source = 'resume', because resumed playback starts from a warm buffer and a rollout that shifts the resume share moves pooled latency without any engineering change.

Worked solution (20 min)

Write the primary metric with numerator, denominator and window, restricted to device_type = 'smart_tv' and its streaming-stick equivalents, and state that it is the outcome the change is judged on.
Specify the latency readout: p95 and p99 of startup_latency_ms computed within (device_type, cdn_pop, network_type) cells, aggregated with pre-period cell weights, reported daily against the same cells pre-rollout.
Construct one numerical counterexample showing the mix trap: two points of presence with unchanged within-cell latency but a shift in traffic share between them, and compute that the pooled mean moves while every cell is flat.
Write the two guardrails: peak_bitrate_kbps distribution plus rebuffer ratio with the played_seconds = 0 exclusion stated, and start-failure rate over end_reason = 'playback_error'.
State the exclusion rule for start_source = 'resume' and the reason, and write the one-line ship rule combining primary and guardrails.

Expected result: Primary: qualified hours per active account-week on TV devices. Mechanism metric: p95 and p99 startup_latency_ms within (device_type, cdn_pop, network_type) cells with fixed pre-period weights. Guardrails: peak_bitrate_kbps together with the rebuffer ratio excluding played_seconds = 0, and a separately counted start-failure rate over end_reason = 'playback_error'. The bought win is opening at a lower bitrate rung.

Verify it yourself

The worked mix counterexample shows every cell flat while the pooled mean moves, which confirms the weighting is doing real work rather than being ceremony.
Streams with played_seconds = 0 appear in exactly one metric, the start-failure rate, and in neither side of the rebuffer ratio.
The bitrate guardrail is stated as a distribution, not a mean, so a shift of the low tail onto a lower ladder rung is visible.
Resume-sourced streams are either excluded or reported as their own cut, so a change in resume share cannot move the headline latency.

Follow-up

p99 falls but the mean rises. What do you conclude, and which cut would you look at first?
The change only reaches accounts on app versions above a floor. How does that change what your comparison can claim?
How large a latency improvement would you need before you expect qualified hours to move at all, and how would you size that before running it?

Trial conversion fell for cohorts that have not matured

Hard · cohort-maturity · mix-shift · acquisition-quality

Trial-to-paid conversion — cohort accounts with a period_index = 1 row in fct_subscription_period whose payment_status is 'paid' or 'retried_paid', over cohort size — is reported weekly by trial_start_ts cohort and has fallen from 38% to 29% across five weeks. Trial length is 14 days. You have dim_account (trial_start_ts, signup_channel, signup_platform, signup_country, is_test_account, first_paid_ts) and fct_subscription_period (account_id, period_index, payment_status, dunning_attempts, period_start_ts). Establish whether conversion fell and for whom. Deliverable: an age-aligned series plus a composition-adjusted estimate of the real change.

Approach

Align cohort age first. No period_index = 1 row can exist before the trial ends, so cumulative conversion is identically zero through day 13 and the whole curve lives between day 14 and the close of the dunning horizon — roughly 14 plus 21 days. The three most recent cohorts sit inside that window and cannot read anything but low. Plot each cohort's cumulative conversion against days since trial_start_ts and read every cohort at one common age.
Quantify how much maturity alone explains. Take the conversion-by-age curve from fully matured cohorts, read off the fraction of final conversion it has reached at each age, and multiply that fraction by a mature cohort's final rate to predict what a young cohort should be showing today. Whatever the prediction reproduces is not a regression and should never have been plotted as one.
Decompose the residual across signup_channel, signup_platform and signup_country. Paid social and carrier bundles convert at very different rates from organic, so a weight shift between them moves the blended figure with every channel's own rate flat.
Make the composition adjustment explicit rather than rhetorical: reweight recent cohorts to the channel-by-country mix of a fixed reference cohort and recompute, then publish the reweighted series beside the raw one and name which channels moved and by how much.
Only then look for a within-channel product cause, and rule out the instrumentation confounders first: is_test_account leakage, attribution opt-out inflating the organic bucket at a platform-specific rate, and app-store payment_status rows arriving later than card-direct ones.
Deliver a number with an interval rather than a direction: the age-aligned, composition-adjusted change, with the cohorts too young to read named explicitly as excluded.

Follow-up

Attribution opt-out shifts accounts into the unattributed bucket at a platform-specific rate. How does that corrupt your reweighting, and what do you do about it?
If the residual concentrates in one paid channel, what do you need from marketing before calling it acquisition quality rather than a product regression?
How many mature cohorts do you need before the composition-adjusted change clears the noise floor?

Suggested practice checklist

For a candidate whose interviews will centre on A/B testing, metric movement and causal claims. Design comes before arithmetic, arithmetic before analysis, and the week ends by rehearsing the readout rather than the derivation.

Day 1 — Design one test end to end on paper

Take a single feature change and write the full design: randomization unit, the exact point of exposure, the primary metric with its grain, guardrails, allocation, planned duration, and the decision rule committed before any data exists.
Write why the randomization unit must sit at or above the level where treatment can spill over, and give one case where user-level randomization is still contaminated (shared accounts or devices, or two participants in the same marketplace).
State in advance what you will do if the primary metric is flat while a secondary metric is significant.

Deliverable: A one-page test design with a decision rule written before launch.

Day 2 — Power arithmetic until it is automatic

Compute required sample size per arm for a binary metric with the normal approximation, n is approximately 2 times (z for alpha/2 plus z for power) squared times p(1 minus p) divided by delta squared, for baselines of 2, 10 and 40 percent at a 5 percent relative lift, and note that for a fixed relative lift the requirement falls as the baseline rises because delta grows proportionally with p.
Redo the calculation for a continuous metric using variance in place of p(1 minus p), and show why a heavy-tailed quantity such as revenue per user needs either far more traffic or a capped version with a stated cap.
Convert one of the results into weeks given a weekly eligible traffic figure, then list the two honest ways to shorten it (accept a larger detectable effect, or reduce variance) and write why quietly lowering the power target is a decision to miss more real wins, not a speedup.

Deliverable: A small script or sheet that maps baseline, minimum detectable effect, alpha and power to sample size and weeks, cross-checked against a published calculator.

Day 3 — Variance and the unit-of-analysis problem

Take a ratio metric whose denominator is not the randomization unit (clicks per session, randomized by user) and compute the standard error twice, once naively at session level and once by the delta method or a user-level bootstrap, then record how much the naive version understates it.
Implement CUPED on simulated data: choose a pre-period covariate X measured before assignment, estimate theta as Cov(Y, X) divided by Var(X), and analyse Y minus theta times (X minus its mean) in place of Y. Confirm the variance of the adjusted outcome equals the raw variance multiplied by one minus the squared correlation between Y and X, so a correlation of 0.45 removes about 20 percent of the variance and not 80.
Now run that simulation a few hundred times and confirm the adjusted effect estimate is unbiased for the same effect rather than numerically identical to the raw one. Within any single run the two differ, sometimes by a large fraction of the true effect, because the two arms' pre-period covariate means never coincide exactly in a finite sample; they agree in expectation, which is the property that matters and the one to state out loud.

Deliverable: A notebook showing the adjusted estimator with a measurably smaller variance than the raw one, plus a repeated-simulation table showing the two estimators agreeing on average while differing run by run.

Day 4 — Validity threats you can actually test for

Run a sample ratio mismatch check as a chi-square goodness-of-fit test against the intended allocation, and write the three causes you would chase first (assignment logged before exposure, an arm-specific redirect or load failure, bot filtering applied asymmetrically).
Simulate peeking: generate A/A data, test daily at alpha 0.05 across 14 looks, record the inflated false positive rate, then apply an alpha-spending boundary or commit to a fixed horizon and confirm the rate returns to nominal.
Write how you would separate a novelty effect from a durable lift using the treatment effect plotted against days since first exposure, and what shape would change your recommendation.

Deliverable: One table showing the peeking false positive rate before and after correction, plus a written SRM triage list.

Day 5 — When randomization is not available

Write the identifying assumption for difference-in-differences (parallel trends in the absence of treatment), then plot pre-period trends for two candidate control groups and justify rejecting one of them.
Design a switchback test for a change where user-level randomization would leak across participants, choosing a time-block length against the carryover you expect and saying how you would detect carryover in the data.
List what an interrupted time series or a synthetic control buys you and the one thing neither can rule out: an unobserved shock that coincides with the launch.

Deliverable: A one-page memo recommending a single quasi-experimental design and naming its weakest assumption explicitly.

Day 6 — The readout query

Write the assignment-to-exposure join that returns exactly one row per unit per experiment, and handle units appearing in both arms by excluding and counting them rather than silently keeping one.
Compute the per-arm metric, its variance and the relative lift with a confidence interval in SQL, then reproduce the identical numbers in a notebook as a cross-check.
Add a segment breakdown and write the sentence that keeps it from being p-hacking: segments declared in advance, everything else reported as exploratory and corrected for multiplicity.

Deliverable: A single query that outputs the full readout table, matched to a notebook recomputation.

Day 7 — Present it to someone who will not read the appendix

Give a 10-minute readout of a real or simulated experiment in the order decision, number, uncertainty, caveat.
Have your listener ask "can we ship it" in the case where the primary is flat and a guardrail moved, and answer with a recommendation rather than a request for more data.
Rewrite your opening line so the recommendation lands before any methodology.

Deliverable: A one-page readout whose first line is the recommendation.

What the behavioural round is actually scoring

Interviewers here are not checking whether you can describe a project. They want the decision you made, why you made it under the information you had, and what changed afterwards that someone else could measure. A story that ends at 'I built a model' has no ending. Say what the model caused, or what you stopped doing because of it.

How do you handle missing data in a dataset?

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Pick a story where you drove the decision, not one where you observed it.
Close with what you would do differently, concretely.

Follow-up

How did you know the outcome was caused by your change?
What would you do differently if you ran that project again?

Describe a situation where you had to work collaboratively with a team. What role did you play?

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

State the impact of your last year without inflation

Hard · self-assessment · attribution · counterfactual reasoning

You are asked what your work was worth over the last year. Two candidates are on the table. First: a dunning-schedule change you analysed, shipped in March with no holdout, after which the involuntary share of gross churn fell three points. Second: a metric-definition change you drove, which nobody can attribute revenue to. Deliverable: the impact claim you make for each, the counterfactual attached to each, and which one you lead with in a performance review — with the reason.

Approach

The probe is whether you apply the same causal standard to your own work as to a product experiment. Attach the counterfactual before the claim: involuntary share of gross churn responds to card-updater coverage, billing_provider mix, retry schedules and the dunning horizon you read it at, any of which can move three points with the schedule change contributing nothing.
Say which evidence would separate those, then say plainly which you actually have. A staggered rollout by billing_provider, a holdout, or at minimum a flat pre-period series would each support a different strength of claim; with none of them, the honest claim is a contribution to a favourable movement, stated as such.
Check the classification itself before claiming the movement is real: the split between cancelled_voluntary and failed_involuntary must be computed only after the dunning horizon has closed for every period in the month, or retries still in flight are counted as churn and the share reads too involuntary in one direction and corrects in the other.
Price your contribution rather than the outcome. The value of analysis that changed a decision is the decision's delta multiplied by the probability the decision would not have been taken without it, and being explicit about that second factor is most of what honesty means here.
Make the definition-change claim concrete rather than apologetic: name the decisions that would have been taken on the wrong number, the reports it reconciled, the recurring argument it closed, with dates. Then lead with whichever claim survives questioning, not whichever carries the larger number, because an inflated first claim makes the second unbelievable.

Follow-up

Your manager writes up the churn improvement and credits it to you. Do you correct it, and to whom?
How would you have designed the March rollout so that attribution was possible, at what cost in delay?
Name something you worked on last year that had no impact, and say what you learned from that rather than from the wins.