Articles
Decision Tree Interview Questions: Split Calculations, Pruning, and Unstable Predictions
Practice decision-tree interviews with an eight-row Gini split calculation, verified cost-complexity pruning path, and prediction-stability checks.
A decision-tree interview can turn on one arithmetic mistake. If the parent node has Gini impurity 0.50, two candidate splits with weighted child impurities 0.20 and 0.375 do not tie: their gains are 0.30 and 0.125. Choosing the wrong split changes every later branch. The stronger answer computes the candidate children, explains why a tiny later improvement may be pruned, and tests whether the resulting predictions survive a small data change.
This is an original eight-row practice fixture, not a report of a particular employer's prompt. scikit-learn's decision-tree guide supports the split, overfitting, and instability discussion; its cost-complexity pruning guide supports the pruning objective. One published Point72 candidate report describes a decision-tree follow-up about depth and overfitting, but that is one account, not a universal interview process.
Question 1: How do you calculate a candidate split?
Use binary classification with eight training rows. A and B are binary features. The label y has four zeros and four ones, so the root Gini impurity is 1 - (4/8)² - (4/8)² = 0.50. This example is intentionally small enough to audit by hand.
| Row | A | B | y |
|---|---|---|---|
| 1 | 0 | 0 | 0 |
| 2 | 0 | 0 | 0 |
| 3 | 0 | 0 | 0 |
| 4 | 0 | 1 | 0 |
| 5 | 0 | 0 | 1 |
| 6 | 1 | 1 | 1 |
| 7 | 1 | 1 | 1 |
| 8 | 1 | 1 | 1 |
For a split on A, the left child (A=0) contains four zeros and one one. Its Gini is 1 - (4/5)² - (1/5)² = 0.32. The right child (A=1) contains three ones, so its Gini is zero. Weight the children by their row counts: (5/8)*0.32 + (3/8)*0 = 0.20. The impurity decrease is 0.50 - 0.20 = 0.30.
For B, the B=0 child contains three zeros and one one, giving Gini 1 - (3/4)² - (1/4)² = 0.375. The B=1 child contains one zero and three ones, also 0.375. Each has four rows, so the weighted impurity remains 0.375; the gain is 0.50 - 0.375 = 0.125. A greedy Gini tree chooses A at this root because 0.20 is the smaller weighted child impurity. scikit-learn's tree guide defines this weighted impurity minimization for classification trees.
The common failure is to compare child impurities without weights. Another is to count one row twice or omit the minority label in A=0. Always restate the candidate partition as actual row IDs, count labels in each child, calculate impurity, then weight and compare. If a candidate creates an empty child, it is not a valid split. If all labels at a node are the same, its Gini is zero and further splitting cannot improve classification impurity.
Question 2: What happens after the root split?
The A=1 child is pure and becomes a class-one leaf. The A=0 child is mixed, with four zeros and one one. Splitting that child on B creates a four-row B=0 leaf containing [0,0,0,1] and a one-row B=1 leaf containing [0]. The four-row leaf has Gini 0.375, while the singleton is pure. Across the whole eight-row training set, the weighted leaf impurity becomes (4/8)*0.375 = 0.1875, down from 0.20 after the root split. The extra branch improved training impurity by only 0.0125.
This tree cannot isolate row 5 using only A and B: row 5 has the same feature pair (0,0) as rows 1–3 but a different label. More depth cannot make identical feature vectors separable. That is a valuable interview point because “grow the tree until training accuracy is 100%” is sometimes impossible even before considering overfitting. A leaf's predicted class is usually the majority label among its training rows under the standard classifier setup, and its estimated class proportions come from the leaf's observations, subject to sample weights and implementation rules.
| Candidate tree on this fixture | Leaves | Weighted leaf Gini | Increment beyond the prior tree |
|---|---|---|---|
| Root-only prediction | 1 | 0.50 | Baseline |
Split on A | 2 | 0.20 | Gain 0.30 |
Split A, then B in A=0 | 3 | 0.1875 | Extra gain 0.0125 |
The table is an original calculation. It does not say that the highest training gain is automatically the best model. The last split isolates one row and changes a small training-set statistic; whether it improves future predictions must be checked out of sample. If the interviewer switches to entropy, recompute impurity with the entropy formula rather than reusing these Gini numbers. Entropy and Gini often rank candidates similarly, but that is not a guarantee on every dataset.
Question 3: How do pre-pruning and post-pruning differ?
Pre-pruning limits growth while the tree is built. Examples include max_depth, min_samples_split, min_samples_leaf, and a minimum impurity decrease. A larger minimum leaf size would prevent the one-row leaf in this fixture; the resulting tree might stop after splitting on A. These are constraints on candidate partitions, so they can change which tree is reachable. State the exact setting and sample weights if asked to predict a library result.
Post-pruning first grows a candidate tree, then removes branches that are not worth their complexity cost. scikit-learn's minimal cost-complexity pruning uses an objective of the form R(T) + alpha * number_of_leaves, where R(T) is total sample-weighted leaf impurity. For our three candidate trees, the objectives are 0.50 + alpha, 0.20 + 2*alpha, and 0.1875 + 3*alpha. The extra B branch is favored over the two-leaf tree only when alpha is below 0.0125; the A split is favored over the root-only tree while alpha is below 0.30, ignoring exact tie-handling at the boundaries. These values are specific to this dataset and Gini criterion.
We checked the eight-row matrix with scikit-learn 1.7.2: an unrestricted DecisionTreeClassifier(random_state=0) used A at the root and B in the left child, produced three leaves, and its cost_complexity_pruning_path returned effective alphas approximately [0, 0.0125, 0.30] with corresponding impurities [0.1875, 0.20, 0.50]. The official pruning example explains how ccp_alpha selects subtrees. The exact path for another dataset or implementation must be computed, not transferred from this tiny fixture.
Choosing alpha from training impurity alone defeats the purpose. Evaluate candidate depths or alphas on validation folds that respect the sampling unit and time structure. If repeated rows from one user appear in both train and validation, apparent stability may be leakage. If observations arrive chronologically, a random split may overstate future performance. Pruning is a complexity control, not a cure for bad labels, drift, or an invalid evaluation design.
Question 4: Why can a decision tree's predictions be unstable?
A greedy tree makes discrete split choices. If two candidate gains are close, one changed row or a different bootstrap sample can change which feature wins at an upper node; every descendant partition can then change. Even when the root here has a clear 0.30-versus-0.125 margin, a new label or feature value can alter those counts. A test point near a threshold can also move to a different leaf after a small feature perturbation. scikit-learn's guide explicitly lists instability as a disadvantage of decision trees.
Diagnose instability with repeated train/validation splits or bootstrap samples, a fixed preprocessing pipeline, and comparison of predictions for the same held-out rows. Inspect whether root features and thresholds change, not just the average score. Report the spread of the metric and the slices where predictions flip. A stable mean score can conceal unstable individual decisions, which matters if a tree is used for credit, triage, or another decision requiring explanations.
There is a bias-variance trade-off. Pruning, minimum leaf sizes, and maximum depth may reduce variance but can miss real local structure. A random forest reduces variance by averaging or voting over decorrelated trees fitted on resampled data; it does not make every individual tree stable or guarantee calibrated probabilities. Gradient-boosted trees fit trees sequentially to improve an objective and have different regularization and learning-rate controls. Do not present “use XGBoost” as a direct answer to a single-tree split calculation. First complete the tree trace, then explain why an ensemble might fit the production goal better.
How to explain a changed prediction to an interviewer
For a new row with A=0, B=1, the three-leaf tree routes left at A, then to the B=1 singleton leaf, whose training label is zero. The two-leaf pruned tree routes to the five-row A=0 leaf, also majority zero. In this case pruning changes the path and leaf support but not the class prediction. For A=0, B=0, the three-leaf leaf contains three zeros and one one, again majority zero. The fixture therefore illustrates a complexity change without a changed predicted class on these feature combinations. That limitation is important: a pruning-path demonstration is not evidence that this particular dataset has unstable predictions.
To show instability, create a nearby training sample or test point and recompute. Do not assert that a tree is unstable merely because the method can be high variance. An interviewer may ask what you would log: the feature vector and data version, model version, chosen path, leaf support and class counts, and whether the row lies near a threshold. Those details let another person reproduce the prediction and distinguish model change from input-pipeline change.
A concise answer worth rehearsing
“For a classification tree, I count labels in each candidate child, compute impurity, weight by child size, and pick the greatest parent-to-child reduction. On this eight-row dataset, A reduces Gini from 0.50 to 0.20, while B reaches only 0.375. A later branch improves weighted training impurity by 0.0125, so I would test whether that complexity earns its place using held-out data or a cost-complexity path. I would evaluate prediction stability across resamples and inspect changed paths, not assume a low training impurity generalizes.”
Recreate the eight-row table and the three objectives without looking, then compare your result with the Gini split practice question. That exercise tests the arithmetic; the pruning and stability reasoning tests whether you can use a tree responsibly.