Interview guides · Georgia-Pacific

Georgia-Pacific Data Scientist Interview Questions & Guide 2026

Modeling drills for a Georgia-Pacific data scientist interview: baselines, regression interpretation, class imbalance, model defense.

The 60-second brief

At Georgia-Pacific, a Data Scientist plays a pivotal role in driving data-informed decision-making across various business units. This position is critical for enhancing operational efficiency, optimizing product offerings, and improving customer engagement through advanced analytics and machine learning techniques. As a Data Scientist, you will leverage vast datasets to glean insights that influence strategies, contribute to product development, and enhance the overall user experience.

When randomisation is off the table, the skill being checked is naming an identification strategy together with the assumption it rests on: parallel trends for difference-in-differences, relevance and exclusion for an instrument, overlap and conditional ignorability for matching. Say the assumption out loud and say how you would try to break it.

Focus areas

Size an experiment before anyone launches it
Define numerator, denominator and window precisely
Separate novelty effects from durable behaviour change

Overview

You will be part of interdisciplinary teams focusing on diverse areas such as supply chain management, manufacturing processes, and customer analytics. Here, your analytical expertise will significantly impact business outcomes, making your role not only vital but also exciting. The scale and complexity of the data you will work with provide ample opportunities for innovation and strategic influence, enabling you to contribute to projects that have real-world implications for Georgia-Pacific and its customers.

The Georgia-Pacific interview process

Georgia-Pacific candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Online HireVue Interview

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Coding/Data Science Challenge

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

In-Person Interview

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Where candidates lose points

Slicing a flat experiment until a segment reaches significance

Testing one metric across twenty segments at a nominal 5% level produces a significant result about two thirds of the time when nothing is happening anywhere, and the segment that surfaces is by construction the one with the most favourable noise. The reported effect in that slice is then badly overstated, because selection on significance conditions the estimate on being large. What makes it dangerous rather than merely wrong is that a post-hoc segment always has a plausible story attached, so it survives the meeting. The controls are declaring the small number of segments of interest before launch, correcting across the ones tested, and treating anything discovered afterwards as a hypothesis that needs its own adequately-powered test rather than a finding.

Counting on an identity key that changes underneath the metric

visitor_id is per browser and per device, and it resets on cookie clearance, private browsing and platform privacy changes, so the distinct-visitor count drifts upward for reasons unrelated to reach. Any rate with visitors in the denominator therefore decays over time even when behaviour is constant, and any rate with visitors in the numerator inflates. The stitching at signup makes it worse in both directions: a user who signed up on mobile and returns on desktop is two visitors and one user, while a shared device is one visitor and several users. Decide which key each metric is counted on, write it into the definition, and when comparing a period before and after a platform privacy change, expect a level shift in every visitor-keyed metric and do not attribute it to the product.

Comparing periods without accounting for seasonality or day-of-week

Compare whole weeks against whole weeks and check whether the same swing appeared in prior cycles or prior years before attributing it to anything you changed. Weekday and weekend populations often differ enough that a Tuesday-to-Saturday comparison is meaningless.

Dropping rows with missing values without naming the mechanism

Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.

Questions & practice

Write a function to calculate the mean and standard deviation of a list of numbers.

Medium · statistics and probability

Approach

Sanity-check the answer against a simple bound or a simulated case.
Translate the result into the decision it informs, in one plain sentence.
Write down the assumption the method needs before you use the method.

Follow-up

Which assumption here is most likely to be violated in practice?
How would you explain this result to someone who does not know statistics?

Describe a machine learning algorithm and its application.

Medium · machine learning and modelling

Approach

Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.
Check what information would not exist at prediction time, and exclude it.

Follow-up

Where could label leakage enter this setup?
What would you monitor after launch to know the model is still valid?

What steps would you take to improve an underperforming model?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

Permutation test for a difference in conversion rates

Medium · permutation test · resampling · inference

Write a two-sided permutation test from scratch for a difference in conversion rates, using no scipy hypothesis function. Input: a DataFrame with unit_id, variant in {control, treatment} and converted in {0,1}, one row per randomisation unit. Compute the observed difference in proportions, then build the null distribution by reshuffling the variant labels while holding each arm's size fixed. Report the p-value as (1 + the count of permuted statistics at least as extreme in absolute value) / (B + 1) with B at least 10,000, and return the permutation distribution.

Approach

Name the null being tested: the sharp null that each unit's outcome is the same under either label. That is what licenses permuting labels, and it is stronger than the null of equal means, which matters when someone asks whether the test is valid under unequal variances.
Extract converted to a single numpy array of 0s and 1s and record n_treatment. Every permutation is then just a reshuffle of one array, and the treatment mean is the mean of the first n_treatment entries of the shuffled array.
Vectorise the B permutations with rng.permuted on a tiled 2-D array, or with argsort of a (B, n) random matrix. A Python loop calling np.random.shuffle B times is correct but roughly an order of magnitude slower and often runs past the time limit.
Use the +1 correction in both numerator and denominator. Without it a p-value of exactly 0 is reportable, which is false: the observed labelling is itself one of the permutations, so the smallest attainable p-value is 1/(B+1).
Compare the resulting p-value against a two-proportion z-test as a sanity check. At these sample sizes they should agree closely; a large divergence means the statistic or the shuffle is wrong, not that the permutation test found something subtle.

Worked solution (25 min)

y = df['converted'].to_numpy(); n_t = (df['variant'] == 'treatment').sum(); obs = y[treat_mask].mean() - y[~treat_mask].mean().
Build the null: for B draws, shuffle y and take the mean of the first n_t entries minus the mean of the rest.
p = (1 + (np.abs(null) >= abs(obs) - 1e-12).sum()) / (B + 1), with the small tolerance so exact ties count as at least as extreme.
Return obs, p and the null array; plot or describe the null to confirm it is centred at 0.

Expected result: The observed difference, a p-value in [1/(B+1), 1], and a null distribution whose mean is approximately 0 and whose standard deviation approximately matches the analytic pooled standard error sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2)).

Verify it yourself

The null distribution's mean is within a few Monte Carlo standard errors of 0; if it is centred on the observed difference, the code bootstrapped instead of permuted.
The null's standard deviation agrees with the analytic pooled standard error to within a couple of percent.
The p-value agrees with a hand-computed two-proportion z-test p-value to within about 0.01 at these sample sizes.
Setting converted identical in both arms by construction yields a p-value near 1, and a planted large effect yields the floor of 1/(B+1).

Follow-up

The arms are 200 and 20,000 units. Does the permutation test stay valid, and what happens to its resolution at B = 10,000?
Give a 95 percent confidence interval for the difference. Can you get it from this permutation distribution, and if not, what would you run instead?
The randomisation unit is user_id but the outcome is per session. What breaks, and what is the fix?

Given a dataset, write a SQL query to extract relevant insights.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Find reactivation gaps in account paid-period history

Hard · gaps-and-islands · window-functions · subscriptions

fct_subscription_period holds account_id, subscription_id, period_start_utc, period_end_utc, period_status and change_reason. A mid-period plan or seat change closes one row and opens another, so a single continuous paid tenure is often many rows, and an account may hold two overlapping subscriptions. Collapse rows with period_status in ('active','past_due') into continuous tenures per account, treating gaps of three days or less as continuous. Return account_id, tenure_start, tenure_end, and for every tenure after the first, the gap in days that preceded it.

Approach

Filter to paid rows only: period_status IN ('active','past_due'). Trialing periods are not tenure, and including them turns every trial that never converted into a one-period tenure followed by a fake churn.
Order by period_start_utc and take a running maximum of all prior ends: MAX(period_end_utc) OVER (PARTITION BY account_id ORDER BY period_start_utc, period_end_utc, subscription_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING). Those three columns are the only stable ordering this schema exposes, so check first that they are unique within an account; if rows tie on all three, the island numbering is order-dependent between runs and you need a real row key before the result is reproducible.
LAG on its own is wrong here because with overlapping or nested periods the immediately preceding row by start date is not the one that ends latest, so the running maximum is the part that cannot be shortcut.
Flag a new island when prior_max_end IS NULL OR period_start_utc > prior_max_end + interval '3 days', then number islands with a running SUM of the flag over the same ordering and an explicit ROWS frame.
Group to (account_id, island) taking MIN(period_start_utc) and MAX(period_end_utc), then LAG(tenure_end) OVER (PARTITION BY account_id ORDER BY tenure_start) to compute the preceding gap in days for every tenure after the first.
Sanity-check with change_reason, which is the only lineage this schema carries: list its distinct values first, then confirm that rows recording a plan or seat change sit inside a tenure rather than opening one, and that every tenure after the first opens on a row whose reason records a restart rather than an ordinary renewal. Do not reconcile against a churn timestamp on dim_account, which this schema does not define; and where such a column does exist, a cancellation timestamp records when the request was made and routinely sits weeks before the period it ends.

Worked solution (35 min)

Find an account with a known mid-period upgrade and dump its period rows to use as the trace case.
Check that (period_start_utc, period_end_utc, subscription_id) is unique per account, since the whole ordering rests on it.
Write the paid-rows CTE and the running MAX with the explicit frame.
Add the island flag and the running SUM, then verify the trace account yields one island.
Group to tenures and add the LAG-based gap in days.
List the distinct change_reason values, then count accounts with more than one tenure and compare against the count of accounts carrying a restart-flavoured reason anywhere in their history.

Expected result: One row per (account_id, tenure). An account with a continuous history returns a single row with a NULL preceding gap. Every non-first tenure has gap_days strictly greater than 3 by construction.

Verify it yourself

No two tenures for the same account overlap or touch within the grace window.
MIN(tenure_start) per account equals MIN(period_start_utc) per account in the filtered input.
The account with a known mid-period upgrade produces one tenure, not two.
(period_start_utc, period_end_utc, subscription_id) has no duplicates within an account; if it does, say how many accounts are affected before reading the output.
The share of accounts with more than one tenure is small; if it is most accounts, the status filter or the grace window is wrong.

Follow-up

Why three days of grace? What do 0 and 30 days each do to the count of accounts classed as reactivated?
An account runs two concurrent subscriptions for different teams. One tenure or two, and what does the revenue reader expect?
How would you turn these tenures into a monthly gross logo churn series without double-counting an account that churned and returned in the same month?

How would you approach forecasting demand for a new product line?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Given a dataset of customer transactions, how would you identify purchasing trends?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

What techniques do you use for feature selection?

Medium

Approach

Work from the decision backwards to the evidence you would need.
Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

A surrogate for twelve-month value inside a three-week test

Hard · surrogate metrics · experimentation · long-term value

A monetisation change — an earlier and harder paywall — will be tested for three weeks. The decision quantity is twelve-month cumulative net revenue per acquired account, which cannot be observed in three weeks. Available: fct_event, fct_session, fct_subscription_period (mrr_cents_constant_fx, period_status, change_reason, is_first_paid_period), dim_account, and fct_experiment_exposure (unit_type, unit_id, variant, first_exposed_at_utc, is_in_analysis_population). Construct a surrogate index readable at three weeks, state the assumption that makes it valid, name the mechanism that breaks it, and give the pre-registered rule for refusing to decide. Deliverable: the index, the assumption, and the refusal rule.

Approach

State the surrogacy condition before building anything: a surrogate is valid only if the treatment's entire effect on twelve-month revenue runs through it. Then say where it fails here, because a paywall moves short-run revenue directly and long-run revenue through churn and through who becomes a payer at all.
Fit the index on history rather than on intuition: regress twelve-month cumulative net revenue per account on features observable by day 21 — first-paid flag, activation days, week-3 core-action days, seats billed — using cohorts old enough to have a twelve-month outcome, and hold out a later cohort.
Report the out-of-sample fit as the headline rather than the point estimate: the held-out R-squared and the calibration of predicted against actual deciles are what license any use of the index, and a decile plot catches the mis-calibration an R-squared hides.
Name the bias direction explicitly: the relationship was estimated under the old paywall, so under a harder paywall the marginal payer is a different person and the index over-predicts their value, biasing the treatment arm optimistic.
Surround the index with the direct three-week readouts it cannot contain — revenue per exposed unit, cancel-within-first-period rate, and the free-side signup and activation counts — and pre-register a numeric refusal rule, because the failure mode of a surrogate is being used confidently in exactly the case it was not fit for.

Worked solution (40 min)

Assemble the training set: accounts with first_paid_at_utc at least twelve months old, their realised twelve-month cumulative net revenue, and the day-21 features.
Fit on pre-change cohorts, evaluate on a held-out later cohort, and report both out-of-sample R-squared and predicted-versus-actual decile calibration.
Write the surrogacy assumption as a falsifiable sentence, then write the specific mechanism in this treatment that violates it.
Specify the direct arm readouts on fct_experiment_exposure with is_in_analysis_population = TRUE: revenue per exposed unit, cancel-within-first-period rate, and free-side signup and activation counts.
Write the refusal rule with numbers attached, covering both the fit threshold and the free-side divergence case.

Expected result: A fitted surrogate with a reported out-of-sample R-squared and calibration, a written surrogacy assumption plus the mechanism that violates it, the resulting bias direction, and a numeric refusal rule agreed before launch.

Verify it yourself

Standard errors are clustered on fct_experiment_exposure.unit_id, the identifier of the randomisation unit. unit_type is the kind label for that unit — a single value such as 'user' across the whole table — so clustering on it collapses the design to one or two clusters and carries no information about who was randomised together. A per-event or per-session standard error on a user-randomised test is the opposite error: too small, and it will manufacture significance.
The analysis population is the exposed set with is_in_analysis_population = TRUE, and a sample-ratio check against the intended split passes before any outcome is read.
The index applied to the control arm should reproduce the historical relationship. If control is already mis-calibrated, the treatment estimate is not interpretable at all.

Follow-up

What evidence would make you trust a surrogate for this particular change rather than a different one?
The index reads plus 8 percent while three-week cancel-within-period is up. Which do you act on, and what do you tell the decision-maker?
How would you size this test, on what unit, and what does that do to the three-week horizon?

Separate a definition edit from a behaviour change

Easy · definition-change · metric-versioning · cohorts

Seven-day activation rate is documented as core actions on at least 2 distinct UTC dates inside the first seven days after account_created_at_utc, and every cohort published through last week was computed on that definition. The dashboard now reads 38% for the signup cohort that started Monday, against 31% for the cohort before it. Two things landed in that week: an onboarding change, and a commit to the metric's SQL that you have not read yet and that came with no documentation update. You have dim_user, fct_event and the metric's version history. Deliver two numbers: how many of the 7 points are behaviour and how many are the definition. Then say which published cohorts are no longer comparable.

Approach

Diff the metric SQL across the commit boundary and enumerate every changed predicate individually: the distinct-day threshold, the is_internal exclusion, the is_core_action source, the window bounds, and the identity key. A single commit routinely changes more than one, and each has its own sign.
Dual-run both definitions over the same 26 cohort weeks. The definition effect is the level shift between the two series computed on identical data, which isolates it from anything that happened in the world.
Read the behaviour effect only from the old-definition series, comparing the new cohort to its own trailing cohorts. The new-definition series cannot answer the behaviour question because it has no pre-period.
Explain the mechanism of the changed predicate rather than only its size. Lowering a distinct-day threshold from 2 to 1 admits every user who acted once in a single session, which is a large and low-intent population, so a several-point jump is the expected magnitude and not a surprise.
State the restatement plan: pick one definition, backfill the full history on it, and annotate the break date. A series carrying two definitions is not a time series.

Follow-up

The onboarding team wants the 1-day threshold kept because it is easier to move. What is your argument, and what would change your mind?
How would you make a definition change visible to every consumer of this metric without relying on people reading a changelog?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

The project you argued against

Saying no well is a senior skill and it is rarely rehearsed. Think of a time you told someone their analysis was not worth doing, or that the experiment could not answer their question at the sample size available. Explain what you offered instead. Refusal without an alternative reads as obstruction rather than judgement.

Tell me about a time you led a project with cross-functional teams.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

How did you know the outcome was caused by your change?
What did you decide not to do, and why?

How do you handle missing data in a dataset?

Medium · behavioural and stakeholder questions

Approach

Quantify the outcome, including what you would not claim credit for.
Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Disagree with a product manager's roadmap claim using data

Medium · causal inference · selection bias · stakeholder

A product manager proposes building a feature on the argument that accounts connecting an integration in week one retain three times better at week four. The figure is correctly computed from dim_user and fct_event, and it has already been shown to leadership. You have one scheduled 1:1 before the roadmap locks. Deliver the specific analysis you would run to test whether the relationship is causal, the result that would change your own mind, and how you open the conversation so that the PM is not put in the position of defending the number in public.

Approach

Recognise what is being probed: whether you can separate a number being right from an inference being wrong, and do it without costing the PM face. The generic answer recites that correlation is not causation; the strong one names the specific confound and proposes the cheapest design that could distinguish the explanations.
State the alternative concretely. Accounts that connect an integration in week one are accounts that already have a workflow and a technical owner, so week-one intent plausibly drives both the connection and week-four retention. The selection is on intent, which no amount of post-hoc adjustment observes.
Order the discriminating analyses by cost. First, condition on pre-connection activity by comparing retention within strata of week-one core-action count, which removes the crude version of the confound but not unobserved intent. Second, look for variation in integration availability that was unrelated to intent, such as a staggered release or an outage window. Third, an encouragement design that randomises a prompt to connect and reads the intent-to-treat effect on week-four retention, which is the only version that identifies an effect.
Run the timing check, because it is nearly free and it is the most persuasive single piece of evidence. If the retention advantage among connectors is already visible before any of them connected, the causal story is largely finished.
Pre-commit to what would change your mind and say it before you show anything: if the gap survives stratification and the encouragement arm moves week-four retention at all, the feature has a case and you will say so.
Open the 1:1 by agreeing with the true part, that the correlation is real and worth chasing, then ask what effect size the roadmap plan assumes. That makes the size of the claim the topic instead of its authorship.

Follow-up

The encouragement test needs six weeks and the roadmap locks in two. What do you recommend in the interim?
Stratifying on week-one activity closes half the gap. What do you conclude, and what do you still not know?
How would you word this in the roadmap document so the PM's original number is reframed rather than deleted?