Causal Mediation Interview Questions: Post-Treatment Variables and Misleading Controls

Causal mediation interview questions test whether you can distinguish an intervention’s total effect from a proposed mechanism. Adding a post-treatment variable to a regression does not automatically reveal a direct effect. It can remove part of the effect you wanted to measure, open a biased comparison, or change the estimand without telling you.

For data scientist and experimentation interviews, start with a causal diagram, define the effect of interest, and state what identifies it. Randomizing treatment helps identify its total effect; it does not also randomize the mediator.

The examples below are original synthetic exercises about identification, not evidence that a particular product mechanism exists or a prediction of an employer’s interview.

What is causal mediation analysis trying to estimate?

A mediator is a variable through which treatment may affect an outcome. Causal mediation analysis asks how an intervention’s effect would change under specified mediator counterfactuals or interventions. An observed association between the mediator and outcome is not enough to establish that mechanism.

Let T be a reminder, M subsequent engagement, and Y renewal. A proposed mechanism is T → M → Y; the reminder may also affect renewal through another path, T → Y. The word “proposed” matters: the diagram is a causal hypothesis, not something proved by a correlation matrix.

The total effect compares renewal under the reminder with renewal under no reminder, allowing engagement to take its corresponding value in each condition. In an appropriately analyzed randomized experiment, this is the primary contrast you can estimate without conditioning on engagement.

A direct-effect question asks what treatment changes while restricting a mediator pathway. Specify how that restriction works. Fixing everyone’s engagement to a selected level and allowing engagement to follow its untreated counterfactual define different quantities.

Kosuke Imai, Luke Keele, and Dustin Tingley lay out potential-outcome definitions and identification assumptions in A General Approach to Causal Mediation Analysis. The essential interview lesson is to define the counterfactual before choosing a regression.

Total, controlled direct, and natural effects are different

Write M(t) for the mediator under treatment status t, and Y(t,m) for the outcome if treatment and mediator were set to t and m. Then the total effect is the population average of:

Y(1, M(1)) - Y(0, M(0))

A controlled direct effect at level m compares Y(1,m) with Y(0,m). It asks about an intervention that holds the mediator fixed at a specified value. Its relevance depends on whether that mediator intervention is meaningful and supported by the data.

A natural direct effect can hold the mediator at the value it would have taken under control: Y(1,M(0)) - Y(0,M(0)). The corresponding natural indirect effect, holding treatment at 1, compares Y(1,M(1)) with Y(1,M(0)). These two contrasts add to the total effect.

That decomposition is algebra on potential outcomes. Identifying the pieces from observed data is a separate problem. The nested quantity Y(1,M(0)) combines treatment with a mediator value from another treatment condition for the same unit; you do not observe that combination automatically.

With treatment–mediator interactions, direct and indirect effects can depend on which treatment state is held fixed. Do not present one product of regression coefficients as a universally valid natural effect. The mediation software paper discusses estimation for defined causal mediation quantities rather than treating every adjusted coefficient as causal.

Why is a post-treatment control dangerous?

Suppose treatment increases engagement, and engagement genuinely increases renewal. If your goal is the total reminder effect, controlling for engagement can block the pathway you intended to include. The adjusted model is answering another question even in a benign causal setting.

Now add an unobserved motivation variable U that affects both engagement and renewal. The structure includes T → M ← U → Y. Here M is a collider: two arrows meet at it. Conditioning on it can make randomized treatment associated with motivation within the selected engagement levels.

Think through the comparison. Among users with the same engagement, those who received a reminder may need less underlying motivation to reach that engagement. Treated and control users inside the conditioned group can therefore differ in an outcome-relevant hidden characteristic, despite balanced motivation before conditioning.

The bias is not fixed by having a large randomized sample. Large samples can estimate a biased adjusted association more precisely. Nor does including every available post-treatment field necessarily help; it may introduce more selection paths.

The identification discussion in Imai, Keele, and Tingley explicitly separates treatment ignorability from mediator ignorability. Their sensitivity framework is relevant because the second condition cannot be inferred merely from random treatment assignment.

Worked counterexample: a mediator coefficient with no mediator effect

Consider original structural equations:

T is randomized independently of U
M = T + U
Y = 2*T + U

There is no causal M → Y term. Changing M alone, while holding its causes fixed, does not change Y in this model. Nevertheless, M and Y are associated because they share causes.

The marginal treatment effect is 2 outcome units. But substituting U = M - T into the outcome equation gives Y = T + M. A regression controlling for M therefore assigns coefficient 1 to treatment and coefficient 1 to the measured mediator.

In this original example, the measured mediator has no causal arrow to the outcome. Conditioning on it changes the treatment comparison and can invent an apparent mechanism.

We verified this with six synthetic rows: treatment 0 and 1 each paired with hidden U values -1, 0, and 1. Least squares gives an unadjusted treatment coefficient of 2 and an adjusted coefficient of 1, up to floating-point rounding. This enumeration establishes the example’s algebra, not an identification result for real user data.

An interviewer might call the difference “50% mediated.” Reject that interpretation for this model. The apparent reduction arose from conditioning on a collider, not an actual mediator pathway. You can show the wrong conclusion without debating an arbitrary significance threshold.

This is also why feature importance is insufficient. A prediction model may rely heavily on M, and explaining that model may show M as important. Neither result establishes what an intervention on M would do.

When does the product-of-coefficients shortcut work?

For contrast, consider a deliberately benign linear structural model:

M = a*T + error_M
Y = c_direct*T + b*M + error_Y

With suitable causal assumptions, independent errors, and no treatment–mediator interaction, the indirect effect is a*b and the total effect is c_direct + a*b. The assumptions distinguish this case from the collider example; the equations are not interchangeable descriptions of the same causal structure.

Choose a=2, b=3, and c_direct=-2. The indirect effect is +6 outcome units, the direct effect is -2, and the total effect is +4. The two pathways work in opposing directions.

The ratio 6/4 = 1.5 does not mean that 150% of people were mediated. It reflects cancellation between effects on a chosen scale. If the total effect is close to zero, a proportion-mediated ratio can become unstable or misleading even when the component effects are meaningful.

For nonlinear models, multiplying two coefficients can mix incompatible scales. A log-odds coefficient, a probability difference, and a mediator-unit change cannot be combined casually. Use the defined counterfactual contrast and a justified estimation procedure; report the effect scale explicitly.

The statsmodels Mediation reference describes a model-based implementation. Supplying models to software does not establish that the required causal assumptions hold, and this article does not claim a statsmodels execution test.

What assumptions should you say aloud?

Separate design conditions from modeling conditions. Treatment assignment may be randomized; mediator assignment generally is not. A useful answer identifies baseline confounders of the mediator–outcome relationship, measurement timing, support, and any treatment-induced common causes.

In plain language, ask whether treated and untreated people with comparable baseline characteristics are comparable for the treatment contrast. Then ask whether, within treatment and those baseline characteristics, mediator differences can be interpreted causally rather than as hidden motivation, health, or intent differences.

If treatment creates a variable that subsequently affects both mediator and outcome, standard natural-effect identification becomes more difficult. Merely adding that variable to the regression can fail to recover the intended quantity. Do not promise that “control for everything” resolves the problem.

Also require a well-defined mediator intervention. “Increase engagement” could mean sending more notifications, reducing friction, or changing content. Those interventions may have different direct effects on renewal. A mediator label can hide multiple causal versions of the action you are proposing.

Finally, consider measurement error and support. A noisy engagement measure may not represent the mechanism accurately. If a mediator level barely occurs in one treatment arm, a fixed-level contrast may require unsupported extrapolation. These limitations should narrow the claim rather than disappear inside a model specification.

How should you improve the evidence?

A sensitivity analysis asks how conclusions change under specified violations of the mediator assumptions. It does not prove those violations are absent. State which uncertainty is being varied and what magnitude would reverse the substantive conclusion.

Better design can help. Measure plausible baseline common causes before assignment, define a mediator window before the outcome window, and consider a separate randomized intervention targeting the proposed mechanism. That new intervention still needs its own causal interpretation; it may influence the outcome through pathways other than the mediator.

Use a mechanism claim ladder in the interview:

1. The original treatment changes the outcome: total-effect evidence.
2. Treatment changes the proposed mediator: evidence for the first pathway link.
3. Mediator and outcome are associated: descriptive evidence, not sufficient identification.
4. A causal mediator contrast is estimated under named assumptions: qualified mechanism evidence.
5. A redesigned intervention tests the mechanism more directly: additional evidence with its own boundaries.

Avoid replacing the first claim with the fourth just because the adjusted model has more columns. A precise recommendation can be: “The experiment supports a total effect; the mechanism remains a hypothesis, so I would not optimize the mediator as if its coefficient were causal.”

Finish with the estimand, not the adjusted coefficient

For causal mediation interview questions, the decisive skill is recognizing when a useful-looking control changes the causal comparison. Define total and direct effects, draw the paths, and explain why randomizing treatment does not identify every mechanism.

Give a total-effect answer first, then state what additional design or assumptions would justify a mechanism claim.

Sources and Further Reading

• Kosuke Imai, Luke Keele, and Dustin Tingley: A General Approach to Causal Mediation Analysis
• Journal of Statistical Software: mediation package methods
• statsmodels: Mediation reference

Research note: Primary sources checked October 3–4, 2026. The six-row collider example and opposing-path coefficients are original synthetic calculations, verified with NumPy 2.5.3. They do not establish assumptions for an observational dataset or identify any real product mechanism.