TF-IDF Interview Questions: Fit the Vocabulary, Compute Weights, and Avoid Leakage

The strongest TF-IDF interview answers specify the tokenizer, vocabulary, term-frequency rule, IDF formula, and normalization before calculating a score. They also explain which documents may influence those choices. A correct formula can still produce an invalid evaluation when the vectorizer learns from withheld text.

Official scikit-learn behavior: the default smoothed weighting uses idf(t) = log((1 + N) / (1 + df(t))) + 1, followed by L2 row normalization. Here, df(t) counts documents containing a term, not its total occurrences. The TfidfTransformer documentation defines those conventions. Other interview prompts may specify different formulas, so repeat the contract you are implementing.

The corpus and calculations below are original practice material, not reported employer scoring rules.

What does TF-IDF represent?

TF-IDF means term frequency multiplied by inverse document frequency. It represents a document as weights over a vocabulary: repetition within that document affects TF, while prevalence across the fitted corpus affects IDF. The representation records lexical occurrence rather than word order, meaning, or factual correctness.

In an interview, separate the representation from the task using it. A classifier consumes document features; a retrieval system compares a query with indexed documents; a duplicate detector may threshold similarity. None of those downstream decisions is determined by the abbreviation TF-IDF alone.

Ask whether TF means raw count, relative frequency, binary occurrence, or logarithmic scaling. Confirm the logarithm base and smoothing rule. Then ask whether the output must be sparse and whether rows are normalized. These details turn a vague request into a testable computation.

Can you compute a three-document matrix by hand?

Use this original training corpus: D1 is red red blue, D2 is blue green, and D3 is blue. Lowercase tokens are already separated by spaces. Fix column order to blue, green, red, use raw term counts, and apply the smoothed natural-log formula above.

Blue occurs in three documents, so its IDF is log(4/4) + 1 = 1. Green and red each occur in one document, so their IDF is log(4/2) + 1, approximately 1.693147. Red's two occurrences in D1 contribute a TF of two but only one document to its DF.

Document	Counts: blue, green, red	TF-IDF before normalization	Approximate L2-normalized vector
D1: red red blue	1, 0, 2	1, 0, 3.386294	0.283217, 0, 0.959056
D2: blue green	1, 1, 0	1, 1.693147, 0	0.508542, 0.861037, 0
D3: blue	1, 0, 0	1, 0, 0	1, 0, 0

For D1, divide each nonzero weight by sqrt(1² + 3.386294²). Do not normalize each column across documents; this exercise normalizes each document row. The zero coordinates remain zero, and the vector's squared entries sum to approximately one after rounding.

Notice that blue survives even though it occurs everywhere. The outer +1 gives it weight one. Under a different formula, such as unsmoothed log(N/df), an everywhere-present term would receive zero IDF. That is why a remembered textbook formula cannot silently replace the specified convention.

How do you reproduce the weights in code?

This original check makes normalization explicit, verifies feature order, and then applies L2 normalization separately. It was run locally with scikit-learn 1.7.2; the linked stable documentation describes 1.9.1. The fixture checks these stated operations, not every behavior of either release.

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.preprocessing import normalize

docs = ["red red blue", "blue green", "blue"]
vec = TfidfVectorizer(norm=None)
raw = vec.fit_transform(docs)
assert vec.get_feature_names_out().tolist() == [
    "blue", "green", "red"
]
rare = np.log(2.0) + 1.0
expected = np.array([
    [1.0, 0.0, 2.0 * rare],
    [1.0, rare, 0.0],
    [1.0, 0.0, 0.0],
])
np.testing.assert_allclose(raw.toarray(), expected)
unit = normalize(raw, norm="l2")
np.testing.assert_allclose(
    np.asarray(unit.multiply(unit).sum(axis=1)).ravel(),
    np.ones(3),
)

The conversion to a dense array is only for this tiny assertion. Production-sized text matrices can have hundreds of thousands of columns; densifying them changes the memory requirement substantially. Keep the working representation sparse unless a justified downstream operation requires otherwise.

If asked to implement the vectorizer from scratch, accumulate per-document counts and increment DF once for each distinct token in that document. Establish a deterministic vocabulary-to-column mapping before constructing sparse entries. Your transform method must reuse that mapping and the learned IDF values.

What state does fitting learn?

Official behavior: TfidfVectorizer combines text counting and TF-IDF transformation; its fitted vocabulary and IDF array define subsequent transforms. A supplied vocabulary fixes the columns, but fitting can still estimate IDF. See the TfidfVectorizer API.

For the toy corpus, transforming purple red produces weights only in the red column because purple was not learned. Without normalization, that red weight remains 1.693147. With L2 normalization, the resulting nonzero row becomes 0, 0, 1. Transformation must not silently create a fourth column.

Vocabulary order matters even when both training and inference matrices have the same shape. Suppose a newly fitted vectorizer orders three different tokens into three columns. A classifier trained against the original columns will interpret those coordinates incorrectly. Shape agreement cannot prove feature identity.

Save the vectorizer with the classifier as one inference artifact, and preserve the tokenizer configuration and training-data identity. An interview answer should identify the object supplying the columns, not merely say that the model was saved.

Why can fitting without labels still leak?

Official evaluation guidance: scikit-learn recommends splitting before fitted preprocessing and keeping transformations inside a pipeline during cross-validation. See its data-leakage guidance. Unsupervised fitting still learns properties of the evaluation population.

Here is an original leakage counterexample. Reserve purple red as a withheld document. Correct train-only fitting uses N equal to three and ignores purple at transform time. If you instead fit on all four documents, purple enters the vocabulary and the IDF values of existing terms change too.

Red now appears in two of four documents, giving log(5/3) + 1, approximately 1.510826. Blue appears in three of four, giving log(5/4) + 1, approximately 1.223144. Green and purple each receive log(5/2) + 1, approximately 1.916291. The withheld document has changed the training representation before a label is consulted.

This example does not prove that leakage always raises accuracy. It proves the evaluation boundary was crossed. The effect can vary with data and estimator; a favorable metric does not restore independence.

How should cross-validation fit the vectorizer?

Within each fold, fit the vectorizer on that fold's training text and transform its validation text with the resulting state. Fitting once on the full development corpus before splitting folds exposes each fold to its withheld vocabulary statistics. The same rule applies to vocabulary filters such as document-frequency thresholds.

An original classifier setup can use Pipeline([("tfidf", TfidfVectorizer()), ("clf", LogisticRegression(max_iter=1000))]). Pass the pipeline and raw training texts into the search procedure. Tune named parameters such as tfidf__ngram_range and clf__C using training folds, then evaluate the chosen system on a separate untouched test set.

A pipeline does not repair the split itself. Near-duplicate messages from one conversation can appear in both partitions, or a later message can reveal an earlier prediction's outcome. When the deployment task involves new users or future text, choose groups or time boundaries consistent with that task.

If an interviewer proposes fitting the vocabulary on all unlabeled documents, ask whether a transductive setup is explicitly allowed. Some retrieval settings intentionally index a known collection. That is a different evidence boundary from claiming generalization to unseen classification examples; name the setting rather than calling every corpus-wide fit safe.

Which tokenization choices cause surprising results?

Official default: the word analyzer uses a token pattern selecting tokens of at least two word characters. Single-character words and punctuation-sensitive identifiers therefore deserve inspection. The text feature-extraction guide also explains sparse text representations and preprocessing choices.

Consider technical text containing C++, C#, R, or product codes separated by punctuation. A default tokenizer may lose the feature you expected. Run the analyzer on representative strings and compare its actual tokens with the intended vocabulary. Do not diagnose an absent column by changing the classifier first.

Removing stop words can also remove useful distinctions. In sentiment text, discarding negation can make not useful resemble useful. Word bigrams or a deliberate stop-word policy can preserve some distinctions, but the change must be evaluated on the same valid split.

Character n-grams can help with misspellings or surface variants, at the cost of more features and different interpretation. They do not automatically solve domain shift. Explain what information your analyzer retains and what it discards before presenting a larger vocabulary as an improvement.

How do unknown tokens and empty rows behave?

For the fitted toy vocabulary, a document containing only purple has no represented terms and becomes an all-zero row. The ordinary nonzero-vector cosine formula would divide by zero, so a from-scratch similarity implementation needs an explicit policy. A classifier may still produce a prediction from its intercept or priors; that is not evidence that it understood the unseen text.

An empty document and an out-of-vocabulary document can share the same numerical row while representing different input conditions. Track them separately when debugging coverage. A corpus whose every token is removed can fail during vocabulary fitting rather than produce a usable fitted model.

Useful checks include repeated terms, a term appearing in every document, a filtered rare term, a completely unseen document, an empty string, and consistent column names across transforms. Compare sparse indices and values as well as shape. For numerical assertions, use tolerance for floating-point weights instead of demanding identical decimal strings.

An original deployment regression check should compare the representation of the same document before and after packaging. Save the fitted vocabulary, IDF state, analyzer configuration, and classifier together; equal matrix widths alone do not prove that column meanings match. Suppose a replacement vectorizer swaps the positions of red and green while the classifier keeps its old coefficients. The prediction can change without any shape error. Inspect feature names and a known nonzero row to expose that mismatch. Also transform the document alone and alongside unrelated documents using the same fitted vectorizer. Its coordinates should stay consistent under this fixed transformation, rather than inherit new batch-level vocabulary statistics.

How do you finish a defensible answer?

Give a small calculation, identify the fitted state, and name the evaluation boundary. If the interviewer changes the formula, update the expected matrix first. If they change the deployment population, reconsider the split and vocabulary coverage before changing the estimator.

Rebuild the sparse TF–IDF exercise using the corpus above, then add purple red only at transform time. Explain both its resulting coordinates and what would change if you fitted on it. That distinction connects mathematical fluency with reliable ML implementation.

Sources and Further Reading

• scikit-learn TfidfTransformer — weighting and normalization conventions.
• scikit-learn TfidfVectorizer — fitted vocabulary and API parameters.
• scikit-learn text feature extraction — sparse features and analyzers.
• scikit-learn common pitfalls — preprocessing and evaluation boundaries.