Articles

SVD Interview Questions: Matrix Shapes, Rank, and Low-Rank Reconstruction

Practice SVD interview questions with rectangular factor shapes, numerical rank, null spaces, truncation error, and checked NumPy reconstruction examples.

An SVD interview answer can fail before the first singular value is calculated. For a rectangular matrix, U @ diag(s) @ Vh may have incompatible dimensions when you request full factors. Switching to reduced factors repairs the shape, but it does not automatically produce a low-rank approximation.

Keep three questions separate: How large are the factors? What is the matrix's rank? Which directions are you choosing to retain? This article follows a small rank-deficient matrix through those decisions and checks the reconstruction numerically.

Evidence boundary: official NumPy, SciPy and LAPACK documentation supports API behavior; university teaching notes support the mathematics. All worked examples are original, locally checked with NumPy 2.5.3. They are not candidate reports or a prediction of employer questions. For related dimensional reasoning, practice Implement matrix transforms and discuss eigenvalues.

SVD interview questions: define the factorization and its shapes

For a real m × n matrix, a full singular value decomposition is A = U Σ Vᵀ. The columns of U and V are orthonormal bases, and the rectangular Σ has nonnegative singular values on its diagonal. For complex matrices, use the conjugate transpose Vᴴ. Singular values are ordered from largest to smallest in a conventional dense decomposition. LAPACK SVD guide

Let p = min(m, n) and let r denote the rank. Those quantities can differ. Full factors use square U and V, while reduced factors retain p singular directions. A compact rank-r representation keeps only nonzero directions in exact mathematics; a truncated representation deliberately keeps a chosen subset.

Use this original matrix:

import numpy as np

A = np.array([
    [3., 0., 0.],
    [0., 2., 0.],
    [0., 0., 0.],
    [0., 0., 0.],
])

A has shape (4, 3), p = 3, and rank r = 2. Its singular values are 3, 2 and 0. The following shape ledger describes mathematical diagonal factors; NumPy returns the singular values as a one-dimensional array instead of returning Σ.

Representation of this A	U shape	Diagonal factor shape	Vᵀ shape	Reconstructs A exactly?
Full	4×4	4×3	3×3	Yes.
Reduced	4×3	3×3	3×3	Yes, including the zero direction.
Compact, rank 2	4×2	2×2	2×3	Yes in exact arithmetic.
Truncated, retain 1	4×1	1×1	1×3	No; the value2 direction is omitted.

The distinction is operational. “Economy SVD” usually describes reduced storage, but terminology varies. State the shapes rather than relying on the name alone. SciPy documents the full-versus-reduced dimensions explicitly. SciPy dense SVD

Reconstruct the matrix without a shape shortcut

For NumPy's reduced decomposition, U is (4, 3), s is (3,), and Vh is (3, 3). Vh already represents the transpose, or conjugate transpose, of the right factor. Transposing it again changes the formula. NumPy SVD

U, s, Vh = np.linalg.svd(A, full_matrices=False)
reconstructed = (U * s) @ Vh
assert reconstructed.shape == A.shape
assert np.allclose(reconstructed, A)

Here broadcasting scales each column of U by its corresponding singular value. With full factors, U has four columns but s has three entries, so neither the same broadcast nor a three-by-three diag(s) can directly multiply the full U.

Two deliberate repairs are possible. Use the first p columns of full U with the p singular values, or construct a rectangular four-by-three Σ and evaluate the full product. Both express the same matrix. The wrong repair is to drop a row of A so that multiplication happens to run.

Check dimensions before values. Then verify reconstruction residual, orthogonality of retained columns, and the relevant rank decision. Printing plausible factors is not enough: singular-vector signs can differ between valid implementations, so comparing individual entries to a memorized answer is fragile.

For the transpose Aᵀ, redo the ledger. Its full U is three-by-three and full Vᵀ is four-by-four; reduced Vᵀ is three-by-four. This simple follow-up catches candidates who memorize “U is tall” without asking which matrix is being decomposed.

Rank and null spaces: dimensions tell you what is missing

The two nonzero singular directions span the useful action of our matrix. A maps a three-dimensional input to a four-dimensional output, but its image has dimension two. Its right null space has dimension n-r = 1; its left null space has dimension m-r = 2.

For this diagonal-shaped fixture, the right null vector can be the third coordinate vector. The left null space is spanned by the third and fourth output coordinate vectors. Multiplying A by the right null vector gives zero; multiplying Aᵀ by either left null vector also gives zero. These are direct checks of the original example, not an appeal to the factor display.

Full U contains enough vectors to represent the entire two-dimensional left null space. Reduced U has only three columns: two active directions and one zero direction. It therefore omits one left-null direction. This matters if the task asks for a complete null-space basis rather than reconstruction alone.

Singular vectors also have sign ambiguity. Flipping both members of a corresponding left/right pair leaves their outer product unchanged. When singular values are repeated, even the basis within the repeated subspace can vary. Compare reconstructed action or subspaces when that is the invariant you need. MIT SVD lecture

A concise interview answer states both dimensions and purpose: “Reduced factors are sufficient to reconstruct A here, but they do not contain every left-null direction.” That is more precise than saying reduced SVD “removes unnecessary information” without naming the task.

Numerical rank is a tolerance decision

In exact mathematics, rank counts nonzero singular values. Floating-point computations cannot use an exact equality-to-zero test as a general numerical-rank policy. Tiny singular values may reflect finite precision, measurement noise, or real weak directions.

NumPy's matrix_rank documentation describes an SVD-based threshold and a default that depends on scale, dimensions and precision. A custom tolerance should reflect the problem's uncertainty rather than being copied blindly from another matrix. NumPy matrix_rank

Consider the original counterexample:

B = np.diag([1., 1e-12])
assert np.linalg.matrix_rank(B) == 2
assert np.linalg.matrix_rank(B, tol=1e-10) == 1

Neither result changes B's stored entries. The second result declares the smaller direction negligible under that tolerance. Explain why that declaration is appropriate for the application's units and error scale. If measurements have noise larger than 1e-12, treating that direction as reliably invertible may be unjustified.

For a full-rank matrix, the two-norm condition number is the largest singular value divided by the smallest. B therefore has a condition number of 10¹². Inverting its small direction can amplify a tiny perturbation into a large output change. Rank-deficient matrices have an infinite condition number in this sense; do not silently divide only by the smallest retained nonzero value and call that the original matrix's condition number. NumPy condition number

The pseudoinverse offers a thresholded inverse construction. NumPy documents a relative cutoff for small singular values; with rcond=1e-10, this B's small direction is discarded. Default pseudoinverse and default numerical-rank thresholds are not interchangeable policies. NumPy pinv

Low-rank reconstruction: compute the error you claim

To retain k directions, use the first k singular values with their corresponding vectors:

k = 1
A1 = (U[:, :k] * s[:k]) @ Vh[:k, :]
error = np.linalg.norm(A - A1, ord='fro')
# 2.0

For our fixture, A1 keeps the entry3 and discards the entry2. The squared Frobenius norm of A is 3² + 2² = 13. Retained squared energy is 9/13, approximately 69.2%; the residual Frobenius norm is 2. The relative Frobenius error is 2/sqrt(13), approximately 55.5%.

These percentages describe different quantities. A 30.8% discarded squared-energy fraction does not mean a 30.8% relative norm error. Taking a square root connects them. Name the denominator and whether the norm is squared whenever you report an “error percentage.”

The truncated SVD minimizes error among matrices of rank at most k under the Frobenius and spectral norms. Frobenius error is the square root of the sum of discarded squared singular values; spectral error is the next discarded singular value. Our rank-one example happens to give 2 for both because only one nonzero direction is discarded. Cornell matrix-nearness notes

To distinguish the norms, use singular values 5, 3, 1 and retain one direction: spectral error is 3, while Frobenius error is sqrt(3²+1²) = sqrt(10). Do not generalize the equality from the first fixture.

The optimality statement has boundaries. It concerns unconstrained matrix approximation under those norms. It does not guarantee the best classifier, the fairest retrieval result, preservation of every distance, or the best reconstruction under a differently weighted loss. If a downstream task values a small but meaningful feature, evaluate that task directly rather than equating high retained matrix energy with high accuracy.

Storage and solver choices: reduced is not automatically cheap

For an original storage exercise, a 30,000-by-300 matrix has a full U with 900 million entries. At eight bytes per entry, U alone uses about 7.2 GB in decimal units. Reduced U has 9 million entries, about 72 MB. This arithmetic excludes workspace, the input, other factors and object overhead; it is not a runtime benchmark.

Keeping only k directions can save more storage, but a dense solver that first computes all reduced factors may still incur substantial work. State whether the task needs all singular values, a complete basis, or only a small leading subspace before selecting a solver.

SciPy's sparse svds computes selected singular components, with solver-specific constraints. Its documentation does not guarantee sorted output. If you reorder singular values, reorder U columns and Vh rows together; sorting s alone breaks correspondence. Specify whether you want largest or smallest directions. SciPy svds

Finally, SVD is a factorization; PCA is a statistical use of a centered data matrix. Scikit-learn's TruncatedSVD does not center its input, which supports sparse-data applications but changes interpretation relative to centered PCA. For centering, scaling and leakage, use the separate PCA interview guide. TruncatedSVD documentation

Sources and Further Reading

Research checked October 4, 2026. Original numerical fixtures use NumPy 2.5.3; implementation options and thresholds should be checked against the installed version.