Articles
Vector Index Interview Questions: Updates, Deletes, and Freshness Guarantees
Answer vector index interview questions on updates, deletes, CDC, stale retrieval, tombstones, reconciliation, and freshness guarantees.
A document is corrected at 10:00. Your vector index acknowledges an update at 10:01, yet a user still retrieves the old paragraph at 10:02. What exactly did the acknowledgment mean? A good vector index interview answer separates the source document, the indexed representation, and the results the application is willing to serve. Those are different states, even when a vendor exposes one convenient upsert call.
These vector index interview questions use two synthetic records: doc-42 moves from version 6 to version 7; doc-43 is deleted. The exercise asks how to prevent stale answers before background updates settle. Try Integrate Chroma and design semantic search API.
Evidence boundary: Pinecone and Qdrant documentation supports the provider-specific operations and consistency options identified below. The version trace, serving gate, and service-level objective are proposed application designs, not claims about a particular vendor's default behavior or a company's actual interview questions.
The short answer: three clocks, not one write
Begin with a precise contract. The source-of-truth store says what content and access policy are current. An ingestion worker transforms that content into chunks and embeddings. A vector index stores a searchable representation, possibly on a different schedule. The serving layer decides whether a retrieved hit can be shown or placed into a model prompt. An acknowledged write says only what the relevant API documents promise; it does not automatically prove that every query replica sees the new record or that stale content cannot reach an answer.
For doc-42, imagine source version 7 is committed while an index still contains version 6. For doc-43, the source has a deletion marker while an old vector may still be returned during propagation. A strong design can refuse both results at serving time, then clean up the index asynchronously. A weak answer says “upsert and wait a few seconds,” without specifying which operation was acknowledged, how long to wait, or how to test the claim.
State the product requirement first. A public catalog may tolerate a brief stale title; an access-revoked internal document may require immediate exclusion. Those are different freshness and safety contracts. Do not treat an arbitrary vendor's eventual-consistency window as the application's authorization policy.
What should an update identify?
The key needs to represent the object being replaced. A single source document may produce several chunks, each with its own vector. One useful scheme is tenant, document ID, document version, chunk ID, and embedding-model version. A mapping table or manifest connects the document to its current chunk IDs. This is an application design, not a requirement of every vector database.
Why keep the manifest? If version 6 produced five chunks and version 7 produces three, simply upserting three new vectors does not tell you which two old IDs to remove. Stable chunk slots can help when segmentation is stable, but a paragraph insertion may shift all boundaries. A versioned manifest makes the replace operation auditable: add the new set, mark version 7 current, then remove version 6 IDs. During transition, a serving gate can accept only the manifest's current version.
Pinecone documents both upsert and update operations; its current documentation distinguishes a vector index from an index with a document schema. Do not assume a metadata-only update regenerates embeddings for changed text. State which component computes the new vector, which fields change, and whether the old vector ID is overwritten or a new versioned ID is created. If the embedding model itself changes, re-embedding is a migration, not a simple text edit.
In an interview, give an invariant: every served hit must belong to the requesting tenant, have current permission, and match the current accepted document version. If that check is too expensive per hit, describe the cache or materialized-version table that supports it and how its invalidation works. A fast vector query is not a complete freshness solution.
Walk through doc-42: an out-of-order update
Suppose an event stream emits doc-42 version 7, then a delayed retry of version 6 arrives. If a worker blindly upserts both using the same ID, version 6 can overwrite version 7. “At least once” delivery makes idempotence necessary, but idempotence alone does not prevent an older, distinct event from winning. The worker needs a monotonic version check against a durable current-version record, or it can use versioned vector IDs and switch the serving pointer only after the new set is ready.
Write the trace on the whiteboard. At time A, source current version becomes 7. At B, the version-7 event reaches the worker. At C, its vectors are accepted or made searchable under the provider's documented semantics. At D, the application changes its accepted-version pointer. At E, old version-6 vector IDs are removed. An out-of-order version-6 event at any later time must be rejected as stale rather than reviving old content. Distinguish the external index operation from the authoritative pointer change.
There is a failure window between B and D. If version-7 indexing fails, the application must decide whether to keep serving version 6 as stale-but-available, suppress doc-42, or retrieve source text through another path. The decision depends on the promised freshness and safety target. For a harmless correction, bounded stale service may be acceptable; for redacted secrets, suppressing the record is safer. Name the policy rather than smuggling it into a retry loop.
Do not claim that versioned IDs alone guarantee atomic replacement. A search can see both versions during overlap. Filter or post-validate results against the accepted-version pointer. Retrieve more than top k if stale candidates will be discarded, or recall may collapse. That over-fetch is a trade-off: it raises vector-query work while preserving answer validity.
How should deletion work when old vectors remain?
First mark doc-43 deleted in the authoritative source or a strongly available denial record. Next prevent doc-43 from reaching users or a model prompt at the query boundary. Then issue deletes for its known vector IDs, retry failures, and reconcile the index against the source. A deletion request can be acknowledged before all application caches, replicas, and answer paths have converged; only a verified end-to-end test supports a stronger claim.
Pinecone's delete documentation describes deletion by ID, metadata filter, or an entire namespace for vector indexes; it calls ID deletion the most efficient way to remove specific records. That is a provider-specific fact, not a reason to use an unscoped delete-all operation. A tenant/document manifest lets the worker target exactly the relevant chunk IDs. Qdrant's data-synchronization guidance likewise discusses keeping an external source and vector collection aligned through event-driven or batch approaches.
The tombstone or deny gate must itself have a clear failure behavior. If it cannot be read, should the app fail closed for sensitive content, or accept a known stale snapshot? This is a product and security decision. If the gate is updated asynchronously from the same delayed event stream as the index, it does not provide immediate protection. The source transaction or a synchronous deletion registry must establish the deny state before the app claims exclusion.
Test more than “ID not returned once.” Run repeated queries that used to retrieve doc-43, include cache and reranking paths, and inspect whether a generated response could still use a previously cached excerpt. Verify every tenant and namespace boundary. Physical deletion and safe serving are separate milestones; record both.
Which freshness guarantee can you honestly make?
Define an observable event and an observable outcome. “Fresh within two minutes” is incomplete until you say whether the clock starts at source commit, event publication, worker pickup, or vector API acknowledgment, and whether it ends at fetch-by-ID, approximate search visibility, or an end-to-end answer. The strongest useful measure is often source-commit-to-first-correct-query-visibility, but it is more expensive to test than request latency.
For an example service-level objective, choose a synthetic target such as 99% of benign edits searchable within five minutes, measured from source commit to a query that retrieves the new version. Track deletion safety separately: a deleted or revoked item must be excluded from serving immediately after the authoritative deny marker is committed. An asynchronous deletion-completion target can be looser, provided the safety gate holds.
Instrument each stage: source event age, queue lag, embedding time, index-write retries, visibility probe latency, version-gate rejections, and stale-result rate. A low mean ingest delay can hide a long tail. Segment by tenant, document size, chunk count, and model version. If probes use only fetch-by-ID but users search approximately, they can falsely certify the experience; test the operation users actually perform.
Qdrant documents selectable write ordering and read consistency options for replicated setups. Those controls affect replica behavior, but they are not a universal source-to-answer freshness promise. Pinecone's API details differ. Name the deployment, API mode, and query path before asserting a guarantee. If you have not verified the provider's exact semantics for that path, label the claim as a proposed application contract and test it.
What if CDC, retries, or backfills race?
Change data capture is a transport for source changes, not a complete correctness proof. Events may be duplicated, delayed, reordered across partitions, or replayed during recovery. Key the worker by tenant and document; preserve a source version or sequence; make a repeated event safe; and reject older versions. Persist enough state to resume after a crash without assuming the index and checkpoint advance atomically.
A useful recovery question is: what if the worker upserts vectors but crashes before committing its event offset? On replay, a deterministic ID and identical version should make the second write harmless. If the worker commits the offset before the index write, an outage can skip the document indefinitely; reconciliation must catch the gap. Exactly-once delivery slogans do not remove the need for source-index comparison.
Reconciliation can compare a manifest of current source documents, versions, and chunk counts with index-visible IDs or provider-supported inventory. A full scan may be costly; use incremental sampling, partitioned audits, and periodic deeper sweeps. Report “missing,” “stale version,” “orphaned after delete,” and “wrong model version” separately, because the repair action differs. A fresh index can still be wrong if an ACL change was not propagated.
During a backfill or embedding-model migration, dual-write or build a separate index generation. Do not switch all traffic merely because the last bulk job completed. Validate recall, metadata filters, tenant separation, deletion state, and end-to-end freshness; then route a controlled share and retain a rollback path. A model migration can change nearest neighbors even when documents are unchanged, so relevance testing belongs beside operational correctness.
How do you explain the design in an interview?
Start with the user-visible invariant, not a vendor feature list. Ask what updates and deletes mean to the product, what latency target matters, and whether stale content is merely inconvenient or unsafe. Draw source, ingest, vector index, and serving gate. For each arrow, mark a failure or delay and the signal that proves recovery. Then narrate doc-42 version 7 and doc-43 deletion through that diagram.
Avoid three common shortcuts. First, “upsert replaces the document” ignores multi-chunk manifests and old IDs. Second, “delete the vector” ignores propagation, caches, and prompt construction. Third, “use strong consistency” conflates a replicated index read setting with source-to-index transformation and application permission checks. Strong answers distinguish each boundary, state what is assumed, and propose a test.
Finish with a measurable failure drill. Pause the embedding worker, edit doc-42, and delete doc-43. Confirm the app either serves the last allowed doc-42 version under its stated policy or suppresses it; doc-43 must never be exposed after the deny marker. Resume the worker, inject an old retry, and verify the accepted version remains 7. This is a sharper answer than promising a generic “eventually consistent” system.