Articles
Feature Hashing Interview Questions: Collisions, Signed Hashes, and Unknown Categories
Practice feature hashing interview questions with signed collision calculations, unknown categories, bucket sizing, sparse inputs, and train–serve consistency.
A new category arrives in production. Feature hashing accepts it without an unknown-category error, but the model gives it a surprisingly strong score. Why?
The category may share a bucket with an old feature whose coefficient is already large. Hashing solves a representation problem—mapping feature names into a fixed width—while deliberately giving up a unique column for every name. It does not create independent evidence for each new category.
Feature hashing interview questions usually become clearer when you can calculate one collision, explain what a signed projection changes, and define the contract between training and serving.
Evidence boundary: Library behavior comes from official scikit-learn documentation; the research result is identified separately. Bucket assignments and numerical examples below are original exercises, not actual MurmurHash outputs or employer interview reports. No candidate reports are used to claim interview frequency.
How Does Feature Hashing Build a Vector?
Let h(i) select a bucket for feature i, and let s(i) be its sign, either +1 or −1. For input values x_i, the hashed coordinate j is:
z_j = sum of s(i) × x_i over features with h(i) = j
The accumulation matters. Colliding features contribute to the same coordinate; the transformer does not resolve the collision by opening another slot. Repeated occurrences also need a defined policy: counts, binary presence, or numeric values.
Official FeatureHasher documentation describes a stateless transformer using signed MurmurHash3, numeric feature values, and a configurable output width. Its string input mode implies unit values. That behavior differs from a vocabulary builder that assigns a remembered column to each discovered feature.
In an original toy mapping, device=mobile and country=US both select bucket 2. With values 1 and 1, unsigned hashing produces 2 in that bucket. With opposite signs, it produces 0. Neither representation preserves the two feature identities.
A zero bucket can therefore mean no contribution, or cancellation between nonzero contributions. If your debugging tool treats every zero as absence, it may explain a prediction incorrectly. Keep a small diagnostic trace of feature construction and projected contributions when testing the pipeline.
Feature values also have semantics before hashing. Hashing the name price and attaching a numeric amount represents a continuous feature. Hashing a string such as price=19.99 creates a category. These are different models even when the output width and hash algorithm match.
What Do Signed Hashes Actually Improve?
Signed hashing helps control collision-induced inner-product error under suitable random-hash assumptions. The original feature-hashing paper provides the research basis. An expectation over randomized mappings is not a promise that one fixed deployed mapping preserves every pair of vectors.
Here is an original calculation with two features forced into one bucket. Let x = [1, 0] and y = [0, 1]. Their true dot product is 0. Unsigned projection makes both hashed values 1, so the hashed dot product is 1.
With independent equally likely signs s1 and s2, the projected dot product is s1×s2:
| s1 | s2 | Hashed dot product |
|---|---|---|
| +1 | +1 | +1 |
| +1 | −1 | −1 |
| −1 | +1 | −1 |
| −1 | −1 | +1 |
Averaging these four equally likely assignments gives 0, matching the original dot product. Yet each individual assignment is wrong by magnitude 1. The example separates unbiasedness in this simplified setting from low error for a particular mapping.
A second calculation uses x = [1, 1]. Its true squared norm is 2. With the same collision, the signed hashed value is s1+s2, giving squared norms 4, 0, 0, and 4. Their average is 2. Cancellation can be severe even though the average is correct.
Do not extend this argument automatically to cosine similarity after normalization. Cosine divides by random norms, and an expectation of a ratio does not follow from an unbiased numerator. A zero vector also needs an explicit similarity policy.
Why Is a Feature-Hash Collision Different from a Hash-Map Collision?
A normal hash map retains keys and uses a collision-handling strategy to distinguish them. Feature hashing deliberately merges their numerical contributions. It trades inspectability and some predictive fidelity for a bounded representation.
Official scikit-learn feature-extraction guidance describes this loss of inspectability and the absence of an inverse transformation. A learned coefficient for a bucket cannot identify which original feature caused its value.
Suppose a linear model has bucket coefficient w2 = 3. An unseen feature with h(new)=2 and sign +1 contributes 3 when its value is 1; sign −1 contributes −3. It shares a parameter learned from other features. That is not a separately estimated effect for the new category.
Explainability therefore needs more than printing the top bucket coefficients. You can log which features contributed to a particular prediction, but you cannot uniquely reconstruct all original names from the hashed vector. Avoid claiming that a bucket is equivalent to one semantic category.
For comparison, official DictVectorizer documentation describes a learned feature mapping and ignores features absent from its fitted vocabulary during transformation. Neither approach is universally preferable: decide whether bounded width, stable names, or explicit unknown handling matters most.
How Should You Choose the Number of Buckets?
Start with model quality and memory constraints, then sweep candidate widths. Collision counting can guide intuition, but the cost of a collision depends on feature frequencies, values, labels, and which features co-occur.
Under an original idealized model where k distinct names choose m buckets independently and uniformly, the expected number of colliding unordered pairs is k(k−1)/(2m). With k=100 and m=1,024, that is approximately 4.83. Increasing m to 4,096 reduces it to approximately 1.21.
These are expected pair counts, not the probability that any collision occurs. They also are not the expected number of occupied buckets. That quantity is m×[1−(1−1/m)^k]. Three names in one bucket produce three colliding pairs, not just one.
The practical follow-up is to measure downstream performance at several widths using a valid held-out split. Inspect rare and new categories separately. A small average metric change may hide a substantial loss on the segment the product cares about.
Sparse rows do not eliminate model-width costs. A linear learner may maintain one coefficient per bucket, and optimizer state can add more arrays. Conversely, the sample matrix may store only nonzero contributions. Account for both the feature representation and the estimator.
Do not increase width at serving time while leaving the trained coefficients unchanged. Changing the modulus changes feature coordinates. Width is part of the model artifact contract and generally requires retraining or a deliberately supported migration.
How Do You Handle Unknown Categories and Field Names?
Hashing an unseen name is mechanically possible because no fitted vocabulary lookup is required. Its statistical behavior still depends on the bucket it receives and the model's existing coefficients. Representation availability is not evidence of reliable cold-start prediction.
Field names should prevent accidental aliases before hashing. If both country and merchant contain the value US, passing only US intentionally makes them the same feature. Encoding country=US and merchant=US separates their identities before any random collision can occur.
Namespacing does not eliminate collisions. It prevents a construction bug in which unrelated fields are guaranteed to share a name. An unambiguous serialization should also handle delimiters: a raw value containing = or a separator should not create the same string as a different field-value pair.
An original input contract distinguishes missing, empty, explicit unknown, and unseen-but-present values. For example, country=<missing> need not mean the same thing as an empty submitted field. Decide these semantics before choosing the hash function, and make training and serving apply them identically.
Unicode and case handling deserve similar attention. A product may consider two visually equivalent labels identical while byte-level feature construction does not. Canonicalize only according to the domain's intended semantics; indiscriminate lowercasing can merge meaningful identifiers.
For text, distinguish feature construction from projection. Official HashingVectorizer documentation describes a text pipeline with analysis and normalization options. Matching bucket width alone does not match tokenizer, n-grams, case rules, or output normalization.
When Can Signed Values Break a Downstream Model?
An estimator that requires nonnegative inputs cannot accept negative projected contributions merely because the source counts were nonnegative. The MultinomialNB documentation describes a multinomial feature model; scikit-learn's feature-extraction guide explicitly discusses disabling sign alternation for estimators requiring nonnegative inputs.
Disabling signs avoids that particular incompatibility when the original values are themselves nonnegative. It does not remove collisions. It also changes the transform, so training and inference must use the same setting.
A normalization choice can create another mismatch. Raw counts, binary indicators, and unit-length vectors have different magnitudes and interpretations. Compare candidate transforms with an estimator that supports the resulting representation, and record the transform alongside the trained model.
The interview answer should include an inexpensive compatibility check: inspect output width, sparse format, value range, finite values, and whether empty rows occur. Then validate a few constructed examples with known expected contributions. This catches contract failures before a large training job hides them behind aggregate metrics.
How Would You Debug Train–Serve Hashing Drift?
Use a golden record whose exact feature names and values are saved. Run it through both environments and compare the sparse coordinates and values before comparing predictions. If projection differs, model evaluation is premature.
This original symptom table narrows the next inspection:
| Symptom | Likely layer to inspect | Useful check |
|---|---|---|
| Same names map to different columns | Transform configuration | Hash implementation, width, and version |
| Columns match but signs differ | Sign configuration | Signed/unsigned setting and golden contributions |
| Text rows differ only online | Feature construction | Tokenizer, n-grams, case, and missing-value rules |
| New category has a large score | Shared coefficient | Bucket assignment and existing feature contributions |
| Larger width fails to load | Model contract | Coefficient dimensions and retraining requirement |
| Nonnegative model rejects input | Estimator compatibility | Source values and projected sign range |
Version the complete feature-construction specification, not just a label saying “hashing enabled.” A stable projection is useful only when the names reaching it are stable. Any upstream change should have a reviewed migration and quality comparison.
A final comparison should include an explicit vocabulary baseline when feasible. If hashing loses quality, investigate whether the difference comes from collisions, preprocessing, regularization, or an estimator mismatch. Changing several of those together makes the result difficult to explain.
For a temporal deployment, evaluate new names from a later period rather than randomly splitting repeated occurrences of the same categories. Report how often new features appear and whether their errors differ from established categories. That experiment tests the operational question—what happens when the vocabulary evolves—more directly than a collision count alone.