Articles

Maximum Likelihood Interview Questions: Support, Boundary Solutions, and Failed Derivatives

Practice maximum likelihood interview questions with Bernoulli boundary estimates, Uniform support, and cases where derivatives miss the answer.

“Set the derivative of the log likelihood to zero” is a useful middle step, not a definition of maximum likelihood estimation. In an interview, the hardest examples often place the optimum at a boundary, make the parameter determine the data's support, or have no finite maximizer. A reliable answer writes the full likelihood, states the parameter space, checks its edges, and only then uses calculus where it applies.

Evidence boundary: MIT OpenCourseWare's likelihood lecture, Penn State's Bernoulli likelihood notes, and MIT's uniform-endpoint example support the method and model examples. The numeric samples and answer rubric below are original exercises.

If you want a first derivation to attempt aloud, open Derive MLE and Bayesian posterior for Bernoulli. Then revisit it with the all-successes boundary case that a quick interior derivative can miss.

State the Model and Parameter Space First

For observations $x_1,\ldots,x_n$ and a model with parameter $\theta$ , the likelihood is the joint probability mass or density evaluated at the observed data, viewed as a function of $\theta$ . Under an i.i.d. assumption it is $L(\theta;x)=\prod_i f(x_i\mid\theta)$ . Taking logs turns the product into a sum wherever $L>0$ ; because log is increasing, positive-likelihood maximizers are unchanged.

Before differentiating, ask four questions: What are the legal values of $\theta$ ? Does the support of $f$ depend on it? Is the maximum at an interior point, a boundary, or only approached as a limit? Does a solution actually exist in the stated parameter space? A stationary point can be a minimum, a saddle point, or an illegal parameter. An optimizer returning a large finite number does not prove a finite MLE exists.

This matters even when the routine interior result is familiar. In a Bernoulli model, the open interval $0 < p < 1$ and the closed interval $0\le p\le1$ lead to different existence statements for a sample of all successes. In a uniform-endpoint model, an indicator for admissible data is part of the likelihood; dropping it changes the problem.

Work a Bernoulli Boundary All the Way Through

Suppose $n$ independent Bernoulli trials yield $k$ successes and $n-k$ failures. For $0\le p\le1$ , $L(p)=p^k(1-p)^{n-k}$ , with endpoint values interpreted from the original likelihood. For $0<k<n$ , the log likelihood is $\ell(p)=k\log p+(n-k)\log(1-p)$ . Its derivative is $k/p-(n-k)/(1-p)$ , giving the interior candidate $\hat p=k/n$ . The second derivative is negative, so it is the unique interior maximum.

Now let all four observations be successes: $(1,1,1,1)$ . Then $L(p)=p^4$ . It increases throughout $[0,1]$ , so the MLE is $\hat p=1$ . The derivative $4/p$ never equals zero in $(0,1)$ . If the model forbids $p=1$ and instead uses $0 < p < 1$ , the supremum is one as $p\uparrow1$ but no parameter in the space attains it. That is a nonexistence result, not a numerical failure. Four failures similarly put the closed-space MLE at $p=0$ .

Sample and parameter space	Interior score equation	Correct likelihood conclusion
$k=2,n=4$ , $0\le p\le1$	root $p=1/2$	maximum at $1/2$
$k=4,n=4$ , $0\le p\le1$	no root in $(0,1)$	maximum at $1$
$k=4,n=4$ , $0 < p < 1$	no root in $(0,1)$	supremum as $p\uparrow1$ ; no MLE

A boundary estimate of one is not proof the true success probability is one. Four successes still leave uncertainty. A naive Wald standard error formula that plugs in $\hat p=1$ yields zero, a warning that the usual symmetric approximation behaves badly at this edge. Use an interval method appropriate to a binomial boundary and report the sample size. Also keep MLE and Bayesian posterior mean separate: a prior changes the inferential object.

Keep the Support Indicator in a Uniform Model

Let $X_1,\ldots,X_n$ be independent uniform observations on $[0,\theta]$ , where $\theta>0$ . Choose the endpoint-inclusive density convention $f(x\mid\theta)=1/\theta$ for $0\le x\le\theta$ and zero otherwise. If all observations are nonnegative and $m=\max_i x_i>0$ , then

$L(\theta)=\theta^{-n}\,\mathbf 1\{\theta\ge m\}.$

For $\theta<m$ , the likelihood is zero because at least one observation lies outside the model's support. Within the admissible region $\theta\ge m$ , $\theta^{-n}$ decreases as $\theta$ increases. Therefore the MLE under this stated convention is $\hat\theta=m$ . There is no interior score root: $d\ell/d\theta=-n/\theta$ on the allowed positive region. The boundary, not the failed derivative, supplies the answer.

For an original sample $(0.2,0.6,0.9)$ , $m=0.9$ . Under the inclusive convention, $L(0.9)=0.9^{-3}\approx1.372$ , while $L(1.0)=1$ and $L(0.8)=0$ . A continuous density may exceed one; it is not a probability assigned to one exact point. The likelihood comparison is valid because we evaluate the joint density consistently under the model.

There is a subtle support convention. A continuous uniform distribution can equally be written with $x<\theta$ at the upper endpoint, because changing the density at one point does not alter its probability law. Under that strict convention the observed maximum is excluded, the supremum occurs as $\theta\downarrow m$ from above, and no maximizer is attained. Some lecture treatments use this exact convention. In an interview, declare the endpoint convention before saying “the MLE is the sample maximum.” The substantive lesson remains: the support indicator governs the boundary.

Explain Why the Usual Calculus Recipe Fails

The derivative shortcut commonly assumes a differentiable log likelihood on an open parameter space with a maximum in its interior. It can fail for three different reasons, and naming which one applies is more useful than saying “the math is tricky.”

First, a legal boundary may be optimal. Bernoulli all-successes with $p\in[0,1]$ is the clean example. Check the boundary values against all stationary candidates. Second, the support changes with the parameter. The uniform example has a discontinuous indicator in $\theta$ at the observed maximum, so differentiation of $-n\log\theta$ alone ignores where likelihood is zero. Third, the objective may be unbounded or have only a supremum. Then there is no finite MLE despite a software optimizer returning its current iterate.

Normal variance illustrates the third issue. For $x_i\sim N(\mu,\sigma^2)$ with $\sigma^2>0$ , routine maximization gives $\hat\mu=\bar x$ and $\widehat{\sigma^2}=n^{-1}\sum_i(x_i-\bar x)^2$ when the sample is nonconstant. This MLE variance divides by $n$ , not the unbiased sample-variance denominator $n-1$ . But if all observations equal the same $a$ , set $\mu=a$ and let $\sigma^2\downarrow0$ . The joint normal density grows without bound. Since zero variance is excluded from the nondegenerate normal parameter space, the likelihood has no finite maximum there.

This is not fixed by reporting $\widehat{\sigma^2}=0$ as if it were an ordinary interior normal model. You can impose a positive variance floor as a modeling or numerical constraint, or choose a different model, but the constrained solution answers a changed problem. State that change rather than hiding it behind an optimizer's lower bound.

Use a Four-Case Diagnostic in the Interview

Given a new model, draw a small decision table on paper before deriving anything:

Check	What to inspect	Typical outcome
Parameter domain	Open, closed, or constrained?	Boundary candidate may exist or only be approached
Data support	Does it depend on the parameter?	Include indicator before logging
Interior shape	Score root and curvature?	Root may be maximum, minimum, or irrelevant
Limits	What happens at edges and infinity?	Finite maximum, supremum, or unbounded likelihood

This also keeps numerical optimization honest. A parameter transform such as $p=\operatorname{sigmoid}(z)$ restricts optimization to $0 < p < 1$ ; it cannot represent a closed-space solution $p=1$ with a finite $z$ . If a solver keeps pushing $z$ upward on all-success data, that matches the mathematical limit. Conversely, a fixed box bound may return the box edge even if the unconstrained mathematical problem has no maximizer. Report the chosen domain and constraint.

For multidimensional problems, you may need constrained optimization conditions instead of a one-variable derivative. The same reasoning applies: test feasible boundaries and directions of escape. A positive-definite covariance constraint, for example, can matter as much as the gradient equation. Do not claim a global maximum from one local stationary solution.

Separate Estimation From Its Interpretation

An MLE is a function of data under a specified family, not a certificate that the family is true. The uniform endpoint estimator $m$ tends to underestimate the real upper endpoint in a finite sample; under the simple uniform model its expected value is $n\theta/(n+1)$ . Bias does not make it cease to be the MLE. In the Bernoulli example, a boundary estimate may be mathematically exact while predictive certainty would be unwarranted.

An interview follow-up might ask about uncertainty. Usual asymptotic normal approximations can be unreliable at boundaries or when regularity conditions such as parameter-independent support fail. Avoid applying an interior Fisher-information formula without checking its assumptions. For the uniform example, derive the distribution of the maximum directly: $P(M\le t)=(t/\theta)^n$ for $0\le t\le\theta$ . That gives a route to finite-sample reasoning suited to this model.

The answer should therefore end with both an estimate and a diagnostic: “Under these assumptions and endpoint convention the maximizer is here; these boundary or support conditions limit the usual derivative and interval shortcuts.” That sentence demonstrates more understanding than a bare formula.