Articles

Monte Carlo Interview Questions: Standard Error, Control Variates, and Simulation Checks

Practice Monte Carlo interview questions with a reproducible simulation, standard errors, a control variate, and checks against an exact answer.

Monte Carlo interview questions are usually asking for more than a loop that draws random numbers. A complete answer names the quantity being estimated, explains why the samples target it, reports simulation uncertainty, and checks the implementation against a case with a known answer. If you propose a control variate, show the known control mean and compare precision at an equal computational budget.

Evidence boundary: the standard-error and control-variate relationships below follow university Monte Carlo variance-reduction notes. The numerical integral and seeded run are original reproducible exercises, not a report of any employer's interview.

Define the Estimand Before the Sampler

Let $U$ be uniform on $[0,1]$ and suppose the target is $\mu=E[U^2]=\int_0^1u^2\,du=1/3$ . A crude Monte Carlo estimator draws independent $U_1,\ldots,U_n$ and averages $Y_i=U_i^2$ . Since each draw has the intended distribution, $E[\bar Y]=\mu$ . The exact integral gives us an unusually strong check: we know what a correct simulator should approach.

In a less convenient problem, you might estimate a rare-event probability, an expected payoff, or a patient-level mean. State that target precisely. “Run 10,000 simulations” says how much computation you plan to do; it does not say what the number estimates, whether each trial is independent, or whether a trial matches the real process.

For this worked exercise, the sampling unit is one independent uniform draw. For a daily business simulation, the unit might instead be an entire customer path or a correlated time block. Reporting a standard error that treats repeated observations from one path as independent would make a narrow interval look more convincing than the design warrants.

Derive the Standard Error, Not Just the Mean

For independent, identically distributed draws with finite variance, the standard error of $\bar Y$ is $\sigma_Y/\sqrt n$ , estimated in practice by $s_Y/\sqrt n$ . Here $E[U^4]=1/5$ , so $\operatorname{Var}(U^2)=1/5-(1/3)^2=4/45$ . At $n=1,000$ , the exact theoretical standard error is $\sqrt{(4/45)/1000}\approx0.009428$ .

That number describes variability of the estimator across independent runs under this model. It is not the standard deviation of individual $U_i^2$ values, and it does not measure model misspecification or software bugs. Doubling $n$ cuts this standard error by only $1/\sqrt2$ ; achieving half the standard error requires four times as many independent draws if all else stays fixed.

A rough normal interval is estimate ± 1.96 standard errors when a suitable central limit approximation is credible. For a small sample or a very rare event with zero observed successes, that shortcut can behave poorly. An interviewer should hear what conditions make the interval plausible, not just the formula. A bootstrap is not automatically a repair for an incorrect sampling unit.

A Seeded Run You Can Reproduce

The following original Python code uses the standard library. It draws 1,000 uniforms from a fixed seed, computes the crude estimate and its empirical standard error, and records the known analytical target. The seed supports debugging and review; it does not make one realization equal the truth.

import math
import random
import statistics

n = 1_000
rng = random.Random(20260927)
u = [rng.random() for _ in range(n)]
y = [x * x for x in u]

crude = statistics.mean(y)
crude_se = statistics.stdev(y) / math.sqrt(n)
truth = 1 / 3
print(crude, crude_se, truth)

On this exact seed and Python's standard random.Random sequence, the result is approximately 0.322139 with an empirical standard error of 0.009409. The error from $1/3$ is about 0.01119, or 1.19 estimated standard errors. That is not evidence of a bug by itself. A useful debugging question is whether the discrepancy is much larger than the estimated Monte Carlo error repeatedly, especially at increasing $n$ .

Keep the raw simulated outputs or summary counts and the seed in a reproducible exercise. Record the sampling code and parameter values, too: the same seed with a different sampler or changed trial definition is not the same experiment. Do not claim bit-for-bit cross-language reproducibility merely because the seed is equal.

Build a Control Variate With a Known Mean

A control variate is a variable computed on each simulated draw whose expectation is known and whose fluctuation is correlated with the target output. Here choose $X=U$ , with known $E[X]=1/2$ . For any fixed coefficient $c$ , define $Z=Y-c(X-1/2)$ . The correction has mean zero, so $E[Z]=E[Y]=1/3$ .

The variance is $\operatorname{Var}(Z)=\operatorname{Var}(Y)-2c\operatorname{Cov}(Y,X)+c^2\operatorname{Var}(X)$ . Its minimum occurs at $c^*=\operatorname{Cov}(Y,X)/\operatorname{Var}(X)$ . In this exercise, $\operatorname{Cov}(U^2,U)=1/4-(1/3)(1/2)=1/12$ and $\operatorname{Var}(U)=1/12$ , so the exact optimum is $c^*=1$ .

Use the same draws to compute $Y_i$ and $X_i$ . The adjusted output is z_i = u_i*u_i - (u_i - 0.5). It preserves the target mean while canceling part of the common fluctuation. Its exact variance is $1/180$ , compared with $4/45$ for the crude output: a factor of 16 less variance and a factor of four less standard error at the same $n$ . That is a property of this deliberately favorable one-dimensional example, not a universal control-variate gain.

z = [x * x - (x - 0.5) for x in u]
adjusted = statistics.mean(z)
adjusted_se = statistics.stdev(z) / math.sqrt(n)
print(adjusted, adjusted_se)

For the seeded 1,000-draw run, this prints approximately 0.334725 and 0.002375. Both estimators target $1/3$ ; this particular adjusted estimate happens to be closer. The stronger claim is theoretical and repeatable: its sampling variance is smaller for the fixed coefficient, regardless of which estimator wins on one seed.

Compare Equal Work, Not Just Equal Draw Counts

In this toy integral, evaluating $U$ adds essentially no work beyond evaluating $U^2$ , so equal $n$ is an appropriate comparison. In a real simulation, obtaining a control may require extra model runs, storage, or synchronization. Compare error against wall-clock time or computational cost, not merely against the number of final estimator rows.

The theoretical table makes the intended comparison explicit. Values are under independent uniforms and a fixed exact coefficient of one.

Estimator, 1,000 draws	Exact output variance	Theoretical standard error	Seeded estimate
Crude mean of $U^2$	$4/45\approx0.088889$	approximately 0.009428	0.322139
Control mean of $U^2-U+1/2$	$1/180\approx0.005556$	approximately 0.002357	0.334725

If the control is weakly correlated, its benefit may be small. If its mean is incorrectly supplied, the adjusted estimator is biased even if its sample variance is tiny. If you estimate $c$ from the same finite sample, the simple fixed- $c$ unbiasedness argument no longer directly describes that random fitted coefficient; use a separate pilot or a carefully justified inference procedure when exact finite-sample properties matter. The interview answer should identify this distinction without pretending every estimated coefficient invalidates practical control variates.

Checks That Catch Simulation Failures

First, test the random input: uniforms should lie in $[0,1)$ , and a large sample should have a mean near 1/2 within sampling error. Second, test invariants: each $Y_i=U_i^2$ is between zero and one, and the same sampled $U_i$ must feed both $Y_i$ and $X_i$ . Third, compare the simulated estimate to the analytical $1/3$ using its reported standard error rather than an arbitrary absolute tolerance.

Next, scale $n$ by a factor such as 100 and check whether empirical standard errors decrease roughly tenfold. For this fixed seed, crude estimates for 100, 1,000, and 10,000 draws are approximately 0.328720, 0.322139, and 0.331389; their empirical standard errors are approximately 0.030580, 0.009409, and 0.002973. The estimates are not required to approach the truth monotonically. The error scale should be consistent with the $1/\sqrt n$ pattern over repeated experiments.

Finally, repeat the entire run under several independent seeds. Count how often a nominal interval covers the known target. One result outside a 95% interval is possible; a systematic failure suggests incorrect variance accounting, biased sampling, a wrong analytic target, or a code defect. A checksum of a sample or unit test of a formula is useful, but neither replaces a statistical calibration check.

Five Interview Follow-Ups to Answer Aloud

What if the samples are correlated? For Markov-chain output or repeated paths, s/sqrt(n) based on individual observations may understate uncertainty. Explain the dependence structure and use an appropriate effective-sample-size, batch-means, or independent-replication analysis. Do not apply the i.i.d. formula by reflex.

What if no rare event occurs? Zero observed successes does not prove zero probability. Report the number of trials, distinguish a point estimate from an upper bound, and consider importance sampling with a verified weighting scheme if the desired tail is too rare for crude simulation. The control-variate example here is not a universal rare-event solution.

Why use paired draws for method comparison? Using the same $U_i$ makes the crude and adjusted estimates directly comparable for each trial and reduces noise in their difference. But method performance should be judged across runs or from variance theory, not from which single estimate is nearer the answer on one seed.

When is a confidence interval misleading? When the sampling law is wrong, the unit is misidentified, a control mean is wrong, or the normal approximation is poor. A narrow interval quantifies uncertainty conditional on its assumptions, not all possible error.

How many simulations are enough? Start from a precision requirement: if estimated standard error is $s/\sqrt n$ , approximate the needed $n$ for a desired half-width, then validate with a pilot and a stopping rule. More simulations are not the only answer when a valid control variate cuts variance at lower cost.