Articles

Cold-Start Recommendation Interview Questions: New Users, New Items, and Evaluation

Practice cold-start recommendation interviews with new-user and new-item strategies, retrieval eligibility, exploration, and time-correct evaluation.

Cold-start recommendation is a missing-information problem, and the missing information differs for a new user and a new item. A new user may have no interaction history but still provide session intent. A new item may have no clicks but already have a description, category, availability, and other usable attributes. Your interview answer should identify which information exists before choosing a model.

For machine learning candidates, the central question is whether the system can recommend a useful, eligible item while learning enough to improve later. An unknown-ID embedding alone does not establish either capability. Start with Design a Cold-Start-Aware Recommender and separate representation, candidate availability, and evaluation.

Evidence boundary: Official TensorFlow documentation supports feature-processing and retrieval behavior. Cited research supports evaluation and exploration concepts.

Which entity is cold, and relative to what cutoff?

Define “cold” before measuring it. A user can be new to the platform, unknown to the current model, or known but lacking interactions in the relevant product category. An item can be newly published, previously unexposed, or old but absent from the training snapshot. These conditions should not silently share one cohort label.

For a concrete interview design, define a cold user as having zero eligible interactions before the prediction timestamp. Define a cold item similarly, and separately track its publication age. Then distinguish zero-history cases from sparse-history cases such as one or two interactions. Learning from an initial click is a different problem from producing the first recommendation.

Cohort	Available evidence and decision
Warm user, warm item	Both histories exist. Test ordinary personalization.
Cold user, warm item	Item history exists. Use context or stated interests.
Warm user, cold item	User history exists. Test item attributes.
Cold user, cold item	Neither history exists. Combine content, context and exploration.

This matrix prevents an aggregate metric from hiding the most difficult quadrant. Also specify identity handling. A returning person on a new device may look new because the system cannot link identities. Treat that as an observable serving condition; do not assume access to personal data that the product does not legitimately collect.

What can you recommend to a user with no history?

Begin with a reliable contextual baseline. For a hypothetical learning-content product, language, selected subject, and the current page can narrow eligible material. A short optional onboarding choice may reveal more relevant information than a complicated model built around an empty user ID.

A popularity baseline can work as an initial comparison, but define popularity within an appropriate context and time window. A globally popular English beginner lesson may be irrelevant to an advanced Spanish-speaking visitor. Apply eligibility and availability rules before ranking.

The official TensorFlow feature-preprocessing tutorial explains how user IDs, item IDs, text, and numerical features can become model inputs. Its unknown-token handling prevents unseen categorical values from breaking the lookup. It does not mean that a previously unseen user automatically receives a distinctive, learned preference vector.

As session events arrive, use only information available before the next request. A subject click can update a session representation even before the durable user embedding is retrained. State how quickly that update reaches serving, and define a fallback if the feature service is unavailable.

Avoid describing onboarding as free information. Questions add friction, users may skip them, and stated interests can differ from behavior. In an interview, propose testing the trade-off between initial relevance and abandonment, rather than requiring a long preference survey for everyone.

How can a new item receive a useful representation?

An ID-only collaborative model has little basis for learning a unique embedding for an item with no interactions. Content features provide another route: encode a title, description, image, category, or other attributes available at publication time. Which features are valid depends on the product and its data rights.

Official TensorFlow material on two-tower retrieval describes separately encoding queries and candidates, including candidate features that permit representations for items not observed during training. This is architectural support for generalization, not a guarantee that every new item will be relevant.

The candidate encoder must actually use those generalizable features. Concatenating a description with a dominant ID embedding can leave the system dependent on historical IDs. Evaluate an ablation that removes item-history signals, then inspect the genuinely unseen-item cohort rather than assuming the metadata branch learned enough.

Content quality also sets a limit. A sparse description, misleading category, or unfamiliar vocabulary can produce a weak representation. Record missing attributes and route incomplete items through an appropriate fallback or review process. An embedding exists even when it carries little useful information.

As interaction evidence accumulates, combine it with content evidence. A hypothetical policy might initially rely more on attributes, then gradually increase the influence of behavior. The transition should reflect evidence quality and uncertainty, not an arbitrary statement that an item becomes “warm” exactly after its tenth click.

Why can a good ranker still fail every new item?

Because ranking only sees retrieved candidates. If a new item has not entered the retrieval index, no ranking improvement can place it in the result list.

Consider an illustrative publishing timeline. A lesson becomes available at 09:00. Metadata validation finishes at 09:01, its embedding is ready at 09:03, and the candidate index refresh completes at 09:20. Requests between 09:00 and 09:20 cannot discover it through that index, regardless of how accurate the ranker would be if given the item.

Publication: confirm eligibility and availability; count invalid items.
Feature preparation: validate attributes and schema; track missing fields.
Encoding: use a compatible model; count failures and stale versions.
Indexing: confirm searchability; measure delay since publication.
Display: apply downstream constraints; count exclusions at each stage.

A freshness lane is one possible design: a small, separately maintained candidate source for recently eligible items. It still needs quality checks, deduplication, availability filters, and a controlled share of impressions. Mixing in new items without those constraints can turn a retrieval fix into a poor user experience.

Explain the latency boundary precisely. “Real-time updates” could mean rapid embedding calculation while index refresh still takes minutes. Measure the end-to-end delay from eligibility to actual retrievability, not just the speed of the component that is easiest to benchmark.

How much exploration should the system allow?

A new item needs exposure to collect feedback, but exposure has a cost. Exploration is a decision about gathering information under constraints. It should not mean randomly placing any untested item in the most valuable position.

Specify a traffic budget, relevant candidate pool, quality floor, and outcome window. A contextual bandit may allocate exposure adaptively, but naming the algorithm does not define those product constraints.

Google's primary research on attribute-to-feature mapping for cold-start recommendations studies using attributes to guide exploration. Treat its reported experimental results as findings in that research setting, not as expected gains for an unrelated product.

Log what was eligible, what was displayed, its position, and the selection probability when the policy supplies one. Without this information, later evaluation can confuse lack of exposure with lack of relevance. Also record policy versions so that changes in feedback are interpretable.

An unclicked item is not automatically a negative preference. The user may never have seen it, may have abandoned the page, or may have found a useful alternative first. Define the observation window and distinguish displayed-but-unselected examples from catalog items that received no opportunity.

What does an honest offline cold-start test look like?

Use a prediction-time boundary that matches the intended deployment. Train on interactions available before a cutoff, construct user and item history features as of each evaluation request, and admit only candidates that were eligible at that time. Choose validation and final test periods before tuning.

A random interaction split can expose an item's future popularity to a model supposedly handling its launch. Even a per-user chronological split can create inconsistent global time boundaries across users. Research on recommender offline-evaluation leakage analyzes these temporal issues. The practical consequence is to audit timestamps across features, labels, and candidate availability together.

For the 09:00 lesson example, a description written before publication may be valid. Its seven-day click-through rate is not available for a prediction made at 09:05. Nor should a test request at 09:05 receive a candidate set built from tomorrow's catalog snapshot.

Separate two useful experiments. Holding out item identities tests generalization to unseen IDs under a controlled split. A chronological launch cohort tests what the deployed system could know when actual new items arrive. They answer related but different questions; label the protocol rather than reporting both as interchangeable “cold-start accuracy.”

If interaction counts define the cold cohort, compute them before each request. Counting an item's entire test-period interactions first can reclassify successful launches as warm and remove them from the cohort being measured. That silently changes the question after outcomes are known.

Which metrics reveal a real cold-start improvement?

Report results by the four cohorts, history-size buckets, and time since publication or signup. Retrieval recall at K asks whether the candidate stage includes relevant items. Ranking metrics such as NDCG at K evaluate their ordering under a defined relevance label. Neither alone proves a business outcome.

Here is a synthetic aggregate trap. Suppose 90% of evaluation requests are warm and 10% are cold. Model A scores 0.40 on warm requests and 0.20 on cold requests for the same per-request ranking metric. Model B scores 0.43 and 0.15 respectively. The weighted aggregate rises from 0.380 to 0.402 while the cold cohort gets worse.

This calculation assumes the same cohort proportions and metric definition. It illustrates why a broad recommender improvement is insufficient evidence for a cold-start claim. Report cohort sample sizes and uncertainty; small launch cohorts can produce unstable differences.

For new items, add exposure coverage, time to first qualified impression, and performance conditional on opportunity. For new users, evaluate initial-session relevance and an appropriate longer-term outcome. More clicks can reflect curiosity or clickbait; select a label that fits the product's actual value.

Offline logs come from an existing exposure policy. Li and colleagues' work on unbiased offline bandit evaluation shows why the data-collection policy matters. Do not claim arbitrary historical logs can fairly evaluate any new exploration policy: action support, logging probabilities, and estimator assumptions constrain the comparison.

How do you turn the design into a launch decision?

State a testable claim, such as improving first-session relevance without harming completion quality or exposing unavailable items. Define eligible traffic and keep the cold-cohort rule stable. An online experiment should record whether treatment changes the rate at which users or items leave that cohort.

Monitor the path from item publication to retrieval, exposure, and mature feedback. A low cold-item click rate can originate in bad content, poor ranking, insufficient exposure, or an index delay. Stage-specific evidence tells you which intervention is justified.

Finish the interview answer with one fallback and one limitation. For example, a context-aware popularity list can serve when personalization features are missing, but it does not discover every niche preference. Content embeddings help a new lesson enter retrieval, but cannot manufacture missing or misleading metadata. Those boundaries make the proposal credible.