Image Segmentation Interview Questions: Dice, IoU, Class Imbalance, and Boundary Errors

Image segmentation interview questions become much easier when you stop treating Dice and intersection over union (IoU) as names to memorize. First decide what a positive pixel means. Count true positives, false positives, and false negatives for one class. Then state how empty masks, background, images, and classes enter the average. A model can score well on area overlap while missing a thin structure or shifting a clinically or operationally important boundary.

The worked masks below are original interview practice, not reports of a particular employer's questions. Metric behavior is grounded in the MONAI metric documentation and the primary Boundary IoU paper. For adjacent vision practice, Explain dataset size, generalization, and U-Net skips asks you to defend model and data choices.

Question 1: Calculate Dice and mask IoU without confusing them with box IoU

Assume one binary foreground class in a 100-pixel image. The ground-truth mask contains 10 positive pixels; the predicted mask contains 12. Eight pixels overlap. Therefore TP=8, FN=10−8=2, FP=12−8=4, and TN=100−8−2−4=86. The mask intersection is 8 pixels. The union is 10+12−8=14, so mask IoU is 8/14 ≈ 0.571. Dice is 2×8/(10+12)=16/22 ≈ 0.727. Pixel accuracy is (8+86)/100=94%.

Quantity	Computation	Result	What it says
IoU	TP/(TP+FP+FN)	8/14 ≈ 0.571	Fraction of the union correctly overlapped
Dice	2TP/(2TP+FP+FN)	16/22 ≈ 0.727	Overlap relative to both mask sizes
Pixel accuracy	(TP+TN)/all pixels	94/100 = 94%	Dominated by correct background here

For the same binary masks, Dice and IoU are monotonic transforms: Dice = 2 IoU/(1+IoU). They do not become numerically equal because both measure overlap. The identity applies to a single confusion-count set; do not apply it to an average of per-image IoUs and expect to recover the average of per-image Dice values. Nonlinear transforms and averaging do not commute.

This is pixel-mask IoU. The existing object-detection guide calculates overlap of bounding boxes and follows it with NMS and mAP. A box can overlap the target while its interior segmentation is wrong. In an interview, say whether you are comparing masks, boxes, instances, or semantic classes before you compute anything.

Question 2: Why can a high accuracy hide complete failure?

Imagine 1,000 pixels, with a ten-pixel lesion or defect. A model predicts background everywhere. It gets 990 pixels right, so accuracy is 99%. Yet foreground TP=0, FN=10, and FP=0: foreground Dice and IoU are both zero. A 99% headline cannot rescue a model that never finds the target. This is a simple arithmetic example of class imbalance, not a claim about medical performance.

Ask whether background is included in a macro class average. The MONAI Dice documentation exposes include_background; its explanation notes that small foreground regions can be overwhelmed by background. A segmentation report should make the choice visible. Report per-class Dice/IoU, target prevalence, positive-image and empty-image slices, plus false-positive volume. If missing a small target is expensive, include object-level recall or a task-specific detection check rather than relying only on mean foreground overlap.

Class weighting and generalized Dice losses are training choices, not substitutions for honest evaluation. Oversampling rare-positive images can help optimization but changes the training mix; it does not increase the number of independent patients, scenes, or devices. Augmentation can diversify appearance, but a geometric transformation must be applied consistently to image and mask. Practice this alignment with Implement and visualize in-place augmentations. A one-pixel misregistration can look like model error even when the predicted anatomy is sensible.

Question 3: What score do two empty masks receive?

If both prediction and truth are empty, raw Dice and IoU have a zero denominator. There is no universal number to silently plug in. Some evaluation protocols treat a correct empty case as perfect, some exclude the class for that image, and others report empty-case specificity or false positives separately. A prediction with foreground when truth is empty is a different failure from two empty masks.

The MONAI metric API makes the policy explicit: with ignore_empty=True, an empty ground-truth case becomes NaN for Dice/IoU aggregation; with ignore_empty=False, an empty prediction paired with empty truth can receive 1. The documented get_not_nans option helps reveal how many values entered an aggregate. State the library/version and settings you used rather than saying “MONAI Dice” as though that fully defines the result.

Consider a test set with 90 empty images and ten positive images. If all empty pairs score 1 and all positive cases score 0, an image-macro mean can be 0.90 even though every target is missed. If empties are excluded, the positive-case mean is 0. These are different estimands, not a software bug. Report positive-case overlap together with an empty-case false-alarm rate and counts. A metric without its eligible-case denominator is hard to audit.

Question 4: Does averaging by image, class, or pixel change the answer?

Yes. Define the reduction before comparing models. Global or micro aggregation adds TP, FP, and FN across eligible images and computes a single overlap score. Large objects and images with many positive pixels weigh more. Image-macro aggregation computes one score per eligible image, then averages; a tiny lesion image can carry the same weight as a large organ image. Class-macro aggregation computes separate class scores and gives each class equal weight. Libraries can combine these axes and handle missing classes differently.

Take two positive images. Image A has TP=90, FP=5, FN=10; image B has TP=1, FP=9, FN=0. Their Dice values are 180/195 ≈ 0.923 and 2/11 ≈ 0.182, so image-macro Dice is about 0.552. Pooled counts give TP=91, FP=14, FN=10, hence micro Dice 182/206 ≈ 0.883. Both calculations are correct under their stated weighting; the gulf tells you the model fails the small case while doing well on the large one.

Do not average batches independently if batch sizes or class presence differ. Carry the relevant counts or per-case scores and valid-case counts across the full evaluation set. When classes are mutually exclusive, specify whether the mask is produced by argmax; for multilabel segmentation, each class may have its own threshold and overlapping predictions. Decide the threshold on validation data only, then freeze it before test evaluation. The threshold can change Dice without changing the probability model, so report it.

Question 5: How can boundary errors hide behind a good overlap score?

Consider a 100-by-100-pixel square mask shifted horizontally by one pixel. Its overlap with the original is 99×100=9,900 pixels; its union is 101×100=10,100, so mask IoU is 99/101 ≈ 0.980. That looks excellent, yet every vertical boundary is displaced. On a thin vessel, road lane, or surgical margin, a one-pixel shift can be consequential. The same absolute shift affects a large mask's area score less than a tiny mask's; compare slices by object size and boundary thickness.

The Boundary IoU paper proposes a boundary-focused overlap metric and reports greater sensitivity to boundary quality than standard mask IoU in its studied settings. That supports using an additional boundary view, not replacing every overlap score with one new number. A boundary metric still needs a specified tolerance or boundary region, object matching convention, and behavior on empty masks. For medical images, distances also need physical pixel or voxel spacing. MONAI's Hausdorff metric documentation exposes spacing and a percentile option; a 95th-percentile distance and worst-point distance answer different questions.

In a review, show overlays of false-positive and false-negative edges beside scores. A thin missed structure may have low area and modest effect on an organ-level Dice, but it can determine whether the output is useful. Confirm whether the annotation itself is crisp enough to support subpixel arguments; disagreement among labelers can set a practical ceiling on boundary precision.

Question 6: How would you debug a sudden Dice drop?

Begin with the measurement contract. Has foreground mapping changed? Are images and masks resized with different interpolation rules? A categorical mask should not be treated as a continuous photo during resizing. Did an evaluation update alter ignore_empty, background inclusion, reduction, or threshold? Are IDs and spatial dimensions aligned after cropping? These checks can explain a score drop before you retrain a model.

Then stratify errors. Compare positive-case and empty-case counts, mask area, object size, device or site, image quality, and boundary distance. View a small, blinded set of overlays across both good and bad slices. If only small objects worsened, inspect resolution, crop coverage, sampling, and annotation quality. If the boundary shifted on all images, inspect registration or preprocessing. If false positives jumped on empty images, inspect threshold, new backgrounds, and calibration. Analyze vision model failures is a complementary production diagnosis prompt.

Finally, name a test that would falsify your preferred explanation. Rerun the same frozen predictions through the old and new metric implementations. Evaluate original masks before resizing. Compare the threshold sweep only on validation data. Audit whether patients, scenes, or near-duplicate frames crossed train/test boundaries. A model change is justified after the metric and data path are trustworthy, not simply because the dashboard line moved.

Question 7: Which threshold and model would you actually ship?

A sigmoid output is a probability-like score per pixel, not yet a binary mask. A threshold of 0.5 is conventional but not automatically optimal for a rare or costly target. On a held-out validation set, sweep candidate thresholds and record foreground Dice/IoU, empty-case false alarms, small-object recall, and boundary quality. If you choose a threshold to maximize one metric, name the trade-off it creates elsewhere. Then freeze that threshold and evaluate once on a truly held-out test set. Choosing it on test data makes the reported test score optimistic.

For a multi-class softmax model, argmax assigns one class per pixel; for a multilabel model, separate class thresholds can allow overlapping masks. These are different prediction contracts. Ask whether a minimum connected-component size or morphological cleanup is part of the deployed pipeline. Such postprocessing may erase small true objects while suppressing noisy false positives, so it must be included in the frozen validation and test pipeline, not added after seeing bad examples.

Compare a simple baseline, the candidate model, and an ablation with identical splits and metric settings. If the candidate gains two Dice points on large structures but loses small-object recall, do not call it a universal improvement. Put uncertainty around per-case changes and review representative masks with domain experts where the boundary has real consequences. The release decision depends on those use-case costs, not a single leaderboard number.

Sources and further reading

• MONAI metrics reference — Dice, IoU, background/empty policies, reductions, and Hausdorff spacing.
• Cheng et al., Boundary IoU — primary research on boundary-sensitive evaluation.
• Object Detection Interview Questions: IoU, NMS, mAP, and Error Analysis — separate box-evaluation intent for comparison.