Jensen's Inequality Interview Questions: Convexity, Expectations, and Counterexamples

Jensen's inequality compares transforming an average with averaging transformed values. For a convex function, the function of the expectation is no larger than the expected function value; for a concave function, the direction reverses. A useful interview answer names the function, checks its domain, states the required expectations, and then explains when equality can hold. Saying “noise increases the result” without those checks is incomplete.

This guide is for quantitative, data science, and ML candidates who need to reason aloud through a bound rather than recite it.

Evidence boundary: the mathematical statements are grounded in the linked MIT course materials. The distributions, arithmetic ledgers, counterexamples, and rehearsal suggestions are original preparation material. We do not use candidate reports to claim that an employer asks these exact questions.

State Jensen's inequality with its assumptions

Primary-source fact: for convex f, Jensen gives f(E[X]) <= E[f(X)]; for concave f, it gives the reverse inequality. MIT's probability lecture states the expectation form and illustrates how curvature governs the comparison. MIT 18.440, Lecture 32

For a clean interview formulation, assume X takes values in a convex interval on which f is convex, and both displayed expectations are finite. That avoids silently treating an undefined expression as a numerical bound. More general extended-value formulations exist, but state that extra scope before using them.

The finite weighted form is equally useful: f(sum(w_i*x_i)) <= sum(w_i*f(x_i)) when the weights are nonnegative and sum to one. A probability mass function supplies these weights. Unequal probabilities require their actual weights, not a simple unweighted average.

You do not need independent copies of X to state this theorem. Independence can matter when analyzing the variance of a sample mean, but it is not part of the basic curvature inequality. Distinguishing those two uses prevents a common detour into unnecessary assumptions.

A concise answer could be: “I am applying Jensen to the convex square function and a random variable with finite second moment, so (E[X])² <= E[X²]. The difference is the variance.” Now the transformation, direction, and existence conditions are all visible.

Prove the direction without memorizing an arrow

Primary-source fact: convexity means the function evaluated at a mixture is no greater than the same mixture of function values. MIT's convex-functions lecture presents that geometric definition and its extension to weighted combinations. MIT 6.079, Convex Functions

Take two outcomes a and b, with probabilities t and 1-t. Their expectation is t*a+(1-t)*b. Substituting into the convexity definition gives Jensen immediately. Repeatedly combining points extends the argument to a finite distribution.

There is also a short supporting-line proof for a differentiable convex function with the mean inside its domain. At mu=E[X], the function lies above its tangent: f(x) >= f(mu)+f'(mu)*(x-mu). Taking expectations removes the linear correction because E[X-mu]=0. What remains is the required lower bound.

The differentiability and interior-point assumptions belong to this particular proof. They are not universal requirements of Jensen. For instance, absolute value is convex but not differentiable at zero. In an interview, say which proof you are using instead of accidentally narrowing the theorem to smooth functions.

A useful verbal picture is a bowl and its chords: averaging the heights at two points puts you on the chord, while evaluating the bowl at their average location puts you below it. For a concave cap, reverse the picture. This is intuition after the domain check, not a replacement for it.

Work through one distribution three ways

Our original fixture takes X=1 or X=5, each with probability one half. Its mean is 3. Use this same distribution to test whether you understand both operations:

Transformation	Average transformed values: E[f(X)]	Transform the average: f(E[X])	Comparison
Square	(1+25)/2 = 13	3² = 9	13 is larger
Natural logarithm	log(5)/2 ≈ 0.805	log(3) ≈ 1.099	0.805 is smaller
Reciprocal	(1+1/5)/2 = 0.600	1/3 ≈ 0.333	0.600 is larger

The square and reciprocal are convex on the relevant intervals; the logarithm is concave on positive inputs. Every expression is defined. Our CPython 3.12.14 check used exact fractions for rational calculations and numerical logarithms for the displayed approximations.

For the square row, the Jensen gap is 13-9=4. Directly calculating the variance gives the same value: the two deviations from the mean are -2 and 2, so their squared average is 4. This is an independent arithmetic check, not a simulation-based proof.

Primary-source illustration: MIT's introductory probability material connects the square-function case with nonnegative variance. MIT RES.6-012, Jensen's Inequality

If probabilities change, recompute the mean and both transformed quantities. You cannot preserve the table by replacing the mean alone. This makes a useful follow-up exercise: assign probability one quarter to 1 and three quarters to 5, then check the direction again before calculating exact values.

When can equality hold without a constant variable?

Strict convexity rules out a nontrivial mixture attaining the same value as the mixture of function values. Under the finite-expectation setup, a strictly convex function has equality when X is constant almost surely. “Almost surely” allows differences on probability-zero events.

Weak convexity permits equality along regions where the function is affine. Our X is not constant, but f(x)=abs(x) is simply x throughout its positive support. Consequently, E[abs(X)]=3=abs(E[X]). There is no contradiction: absolute value is not strictly convex on that region.

A globally affine function gives equality for every integrable X. That is why linearity of expectation and Jensen answer different questions. Linearity moves a sum or scalar multiple through expectation exactly; Jensen compares a nonlinear transformation with its expected value.

A weak response says, “Equality only happens when there is no randomness.” Repair it by first specifying strict convexity on the relevant domain. Then give the positive-support absolute-value example to show what changes when that condition is removed.

Construct counterexamples by breaking one assumption

Counterexamples are useful because they reveal which condition carries the argument. Change one assumption at a time, and calculate both sides rather than asserting that a theorem “fails.”

Wrong curvature interval: take f(x)=x³ and equal-probability outcomes -2 and 1. Then E[X]=-0.5, f(E[X])=-0.125, and E[f(X)]=(-8+1)/2=-3.5. The claimed convex-direction inequality would require -0.125 <= -3.5, which is false. The cube is not convex throughout the interval spanning these outcomes.

Negative weights: keep the square function but use weights -1 and 2 for inputs 1 and 3. The weights sum to one, yet the weighted input is 5, whose square is 25. The weighted squares total 17. Nonnegative weights were essential; these coefficients do not define probabilities.

A broken domain: use a reciprocal with outcomes -1 and 2. The mean is 0.5, so the reciprocal of the mean is 2, while the mean reciprocal is -0.25. The interval between the outcomes contains zero, where the function is undefined. Pointwise evaluation at the two outcomes is not enough to establish convexity on their convex hull.

Existence deserves the same attention. Knowing that E[X] is finite does not guarantee a finite second moment, so an expected square can be infinite. A formal extended-value bound may still be meaningful, but it does not yield the finite expected cost a stakeholder asked you to calculate. Say what moment or tail condition is missing.

Does an unbiased estimate stay unbiased after transformation?

Our original estimator T takes values 1 and 3 with equal probability. It is unbiased for a target of 2: E[T]=2. Its reciprocal is not unbiased for the reciprocal target:

E[1/T]=(1+1/3)/2=2/3, whereas 1/E[T]=1/2.

The difference is one sixth. Jensen predicts the upward direction because the reciprocal is convex on positive inputs, but it does not supply that exact magnitude. We obtained the magnitude by evaluating the distribution.

The same estimator produces a downward log bias: E[log(T)]=log(3)/2≈0.5493, below log(2)≈0.6931. Neither result says the original estimator is defective. The transformation changed the quantity whose expectation you wanted.

If asked to approximate the gap, distinguish an approximation from the theorem. A second-order Taylor expansion around mean mu suggests E[f(T)]-f(mu) ≈ f''(mu)*Var(T)/2 when concentration, smoothness, and remainder control make the expansion useful. It is not a universal exact bias correction.

For T equal to 1.5 or 2.5, the exact reciprocal gap is 1/30≈0.03333; the second-order expression gives 1/32=0.03125. For the wider 1-or-3 distribution, the approximation is 0.125 while the exact gap is about 0.16667. Both calculations were checked locally. The weaker approximation under wider spread gives you a concrete reason to discuss the remainder.

What changes with conditional averaging or an ensemble?

Apply the same logic within a group before averaging groups. In our original four-outcome example, X is uniformly drawn from 1, 3, 5, and 7. Let G identify the lower pair or upper pair. Conditional means are 2 and 6, each occurring with probability one half.

For the square transformation, three quantities are distinct: (E[X])²=16, E[(E[X|G])²]=20, and E[X²]=21. Group averaging removes within-group variation, but it does not remove the variation between group means. This explains the chain 16 <= 20 <= 21 without treating a conditional mean as a fixed global mean.

For an ensemble, hold one outcome y fixed and average predictions under a loss convex in the prediction. With y=1 and predictions 0 and 2, each squared loss is 1; their average prediction is 1 and has squared loss 0. Jensen bounds the loss of the averaged prediction by the average individual loss at that same outcome.

This does not prove that every ensemble beats the best model, that every metric is convex, or that validation leakage disappears. Accuracy, a thresholded decision, and a training procedure involving nonlinear parameter averaging require separate reasoning. Our preparation recommendation is to name exactly what is being averaged: predictions, losses, parameters, or estimated probabilities.

Practice the proof and the boundary checks

The Jensen follow-ups are original extensions, not claims that each record explicitly asks for this theorem or predicts your next interview.

Practice question	Original Jensen follow-up
Solve probability and expectation problems	Separate linear expectation calculations from a nonlinear transformed expectation.
Calculate Expected Impressions and Probability for Users	Explain why knowing an expected count does not determine every nonlinear function of that count.
Find minimum of unknown convex function	State the convex domain and distinguish an optimization argument from an expectation bound.
Prove That Lasso Can Be Written as a Quadratic Program	Practice an assumption-led convexity proof without confusing convexity with uniqueness.
Compare Losses and Explain LoRA	Identify the variable in which a loss is convex before making an averaging claim.

Sources and Further Reading

• MIT 18.440 Lecture 32: expectation-form Jensen and concavity
• MIT 6.079 Lecture 3: convex functions and weighted combinations
• MIT RES.6-012 S18.2 transcript: Jensen's inequality and the variance example