Articles

Missing Data Interview Questions: MCAR, MAR, MNAR, and Sensitivity Checks

Explain MCAR, MAR, and MNAR, avoid imputation traps, and test a missing-data conclusion with a concrete sensitivity calculation.

Sixty customers answer a satisfaction survey and give an average score of 8 out of 10. Forty customers do not answer. A dashboard reports “customer satisfaction: 8.” Before choosing an imputer, ask whether the goal is to describe respondents or estimate satisfaction among all one hundred invited customers.

That distinction drives a strong missing data interview answer. A completed table is not the same as a justified estimate. You need a target population, an explanation for why values are absent, and a check showing how plausible assumptions about those unseen values change the conclusion.

This guide uses an original survey exercise and a small Python sensitivity calculation. Authoritative statistical references support the definitions, official software documentation supports implementation guidance, and our examples are labeled as analysis. No candidate report is used to establish frequency.

Define what is missing and what you want to estimate

Start with the unit of analysis. Is one row a customer, session, transaction, or survey invitation? Then distinguish a missing feature from a missing outcome. Predicting churn with an absent age field is different from estimating churn when outcomes themselves are unavailable for some customers.

Also separate “unknown” from “not applicable” and from a real zero. A first-time customer has no previous-order delivery duration. Inventing a historical duration changes the meaning of the feature. A recorded duration of zero might instead indicate a genuine immediate event or an instrumentation defect. The data contract decides which interpretation is defensible.

For our original survey, every invited customer has a meaningful satisfaction score on a 1–10 scale at the reference time. We know who was invited and who responded. The estimand is the average score among those one hundred people. Generalizing beyond those people would introduce a separate sampling question.

Keep the invitation roster when joining answers. If you start from the response table, the forty missing outcomes disappear entirely and the apparent response rate becomes 100%. An imputation technique cannot correct a denominator you never retained.

Explain MCAR, MAR, and MNAR conditionally

Authoritative definitions: missing completely at random, or MCAR, means missingness does not depend on observed or unobserved data values. Missing at random, or MAR, allows dependence on observed information but not additional dependence on the missing values after conditioning on that information. Missing not at random, or MNAR, retains such dependence on unobserved information. See Stef van Buuren's introduction to the mechanisms.

These original examples make the conditioning visible:

Mechanism proposed	Survey explanation	Assumption you must defend
MCAR	An independently random selection of invitations loses its reply record	Record loss is unrelated to customer attributes and satisfaction
MAR given recorded device type	An older app version makes responses harder to submit	Within the observed device/version groups, response does not additionally depend on satisfaction
MNAR given available fields	Dissatisfied customers are less likely to respond even within the recorded groups	The unseen score still affects response probability after conditioning

MAR does not mean that values are missing “for no reason.” It describes which information is sufficient for modeling the missingness process. Adding useful observed variables can change the plausibility of a MAR analysis, although collecting more columns does not automatically establish it.

The labels are relative to the available data and analysis. A mechanism that depends on an unrecorded device failure can look different once reliable device-version information is recovered. Avoid classifying an entire company dataset as “MAR” without naming the variables, population, and missingness pattern.

Can the observed data prove the mechanism?

A missingness audit can show that response rates differ by device, signup week, geography, or prior engagement. Such findings can challenge a simple MCAR story and suggest variables to include in a model. They do not reveal the distribution of scores that nobody observed.

Authoritative limitation: van Buuren's formal discussion of incomplete data explains why MAR cannot generally be distinguished from MNAR using the observed data alone. Failure to reject a diagnostic test is also not proof that missingness is harmless.

For the survey, inspect whether the invitation was delivered, whether the form rendered, whether submissions reached the server, and whether answers were lost during ingestion. Compare counts across that chain before fitting a response model. A known recoverable pipeline failure calls for recovery of records, not immediate statistical invention.

Then investigate whether observed groups with lower response rates have different observed scores. That can motivate stratification or modeling, but keep the conclusion conditional: “MAR given these measured fields is our working assumption.” Do not upgrade it to an established fact because the adjusted and unadjusted means happen to agree.

A targeted follow-up sample of nonrespondents may provide more useful evidence than a more elaborate imputer. It still needs a design: if only the easiest nonrespondents answer the follow-up, the remaining selection problem persists.

Why deletion and mean filling can mislead

Complete-case analysis estimates from records with the required values present. Under suitable conditions it can be valid, and MCAR is a familiar sufficient condition for simple descriptive analyses. It also loses information. Whether it is valid under a weaker mechanism depends on the estimand and analysis model; “deletion is always biased unless MCAR” is too broad.

In our survey, replacing all forty missing scores with the respondent mean of 8 leaves the overall mean at 8 by construction. That operation adds no evidence about nonrespondents. Treating the filled values as forty genuine observations can make uncertainty look smaller than the information supports.

Filling with zero is worse for this example because zero is outside the defined 1–10 score scale. Filling with a random observed response avoids a spike at the mean but still imports the respondents' distribution into the missing group. It does not establish that the two groups are comparable.

For a predictive feature, a median fill plus a missingness indicator can be a useful baseline. A flexible model may benefit from learning that the value was absent. But a predictive association with missingness does not repair identification of an unseen outcome distribution or turn a regression coefficient into a causal effect.

Make the survey conclusion sensitive to an explicit assumption

The sixty observed scores sum to 60 × 8 = 480. Let u be the unknown mean score among the forty nonrespondents. The mean among all invited customers is:

mean_all = (480 + 40 × u) / 100 = 4.8 + 0.4 × u

This original calculation does not estimate u. It shows exactly what each proposed value implies. Suppose a product team uses an average of at least 7 as a descriptive planning threshold, not a statistical significance criterion.

Assumed nonrespondent mean u	Implied overall mean	At least 7?
8.0	8.0	Yes
6.0	7.2	Yes
5.5	7.0	Exactly at the threshold
5.0	6.8	No
3.0	6.0	No

The tipping point is u = 5.5. Equivalently, the missing group can average 2.5 points below respondents before the overall mean reaches 7. Ask whether that difference is plausible using domain knowledge, earlier follow-up surveys, or newly collected evidence.

Without additional assumptions beyond the valid score range, the overall mean lies between 5.2 and 8.8. Those endpoints assign every missing score 1 or 10, respectively. This is an identification range for the fixed invited group given the observed sum and bounds, not a 95% confidence interval.

A richer analysis could begin with conditional predictions under MAR and shift the missing outcomes by a sensitivity parameter. Keep shifts within the outcome's support and explain whether they vary by group. The simple table deliberately avoids pretending that a chosen sensitivity value was learned from the incomplete data.

Check the sensitivity arithmetic with Python

This standard-library example uses exact decimal arithmetic so the threshold comparison is transparent.

from decimal import Decimal as D

n_observed = D(60)
n_missing = D(40)
observed_mean = D(8)
n_total = n_observed + n_missing
threshold = D(7)

def overall_mean(missing_mean):
    if not D(1) <= missing_mean <= D(10):
        raise ValueError("Score mean must stay within 1..10")
    return (n_observed * observed_mean
            + n_missing * missing_mean) / n_total

for u in map(D, ["8", "6", "5.5", "5", "3"]):
    print(u, overall_mean(u), overall_mean(u) >= threshold)

tipping = (threshold * n_total
           - n_observed * observed_mean) / n_missing
assert tipping == D("5.5")
assert overall_mean(D(1)) == D("5.2")
assert overall_mean(D(10)) == D("8.8")
assert overall_mean(D(5)) == D("6.8")

A useful follow-up changes the missing count while holding the response mean fixed. As the missing fraction grows, the overall conclusion becomes more sensitive to its unknown mean. With no missing outcomes the sensitivity term disappears; with no observed outcomes this calculation contains no observed mean to anchor it. Production code should handle those cases explicitly rather than divide by zero while computing a tipping point.

Separate predictive preprocessing from statistical inference

Official implementation guidance: scikit-learn provides univariate and multivariate imputers, missingness indicators, and some estimators that accept NaN values directly. Its imputation guide explains their different behavior. These are tools, not automatic mechanism detectors.

For prediction, choose a baseline and evaluate it on the population the model will actually serve. Fit imputation statistics inside each training fold, then apply the fitted transformation to the held-out fold. Use time or group splits when those reflect deployment. The official common-pitfalls guide explains why preprocessing learned from held-out data leaks information.

For an inferential analysis, uncertainty about missing values must remain in the result. Proper multiple imputation creates plausible completed datasets, fits the analysis to each, and pools estimates with both within-imputation and between-imputation uncertainty. The original mice paper describes chained-equation imputation and pooling. More completed datasets do not eliminate bias from a poorly justified model.

Do not confuse repeated passes through an iterative imputer with proper multiple imputation. Likewise, averaging filled-in values into one supposedly complete dataset discards the variation that pooling is meant to retain. The imputation model must suit variable types, relationships, and the subsequent analysis.

In prediction, do not require the future target to fill features at serving time. In an inferential missing-data model, including an observed outcome among imputation predictors can be appropriate. The goal and availability constraints determine the distinction; “never include an outcome anywhere” is not a universal rule.

Explain what you would monitor after deployment

A missingness indicator can become a shortcut. Suppose optional-profile completion predicts retention during training, then the onboarding team makes the field mandatory. The model's learned relationship may no longer apply. Track missingness by source and cohort, model quality on incomplete records, and changes in the collection process.

Our interview recommendation is to report three separate things: what you observed, what you assumed about unseen values, and which decisions change under plausible alternatives. For the survey, that means reporting 60 responses with mean 8, the forty unknown scores, and the 5.5 nonrespondent-mean tipping point. It is more informative than saying “we handled missing data with imputation.”

These verified practice records develop related skills without claiming a current employer-specific process:

Practice question	Missing-data reasoning to demonstrate
Compare Missing-Value Strategies for Linear Models	Explain which distributional relationships a fill method preserves or changes
Handle Missing Values and Outliers in Machine Learning	Separate valid extreme observations, data defects, and unknown values
Design a Churn Model: Handle Missing Data and Justify	Respect feature availability and temporal validation
Diagnose a sudden KPI drop and validate causes	Investigate instrumentation before assigning a statistical mechanism
Diagnose Violations of Linear Regression Assumptions	Connect inference claims to the model and selection assumptions

Begin with Compare Missing-Value Strategies for Linear Models, then explain why the best predictive treatment need not answer the survey's population-mean question.