Articles
Elasticsearch Write-Path Interview Questions: Refresh, Flush, and Segment Merges
Trace Elasticsearch writes through acknowledgment, refresh, flush, and segment merges, with search-visibility timelines, crash reasoning, and safer testing.
Your service indexes a document and receives a successful response. A GET by ID returns it, but the search endpoint does not. A teammate proposes flushing the index; another suggests force merging it. Which boundary is missing, and which operation provides it?
For Elasticsearch write-path interview questions, separate four boundaries: acknowledgment, search visibility, a Lucene commit, and segment maintenance. They answer different questions. A strong explanation traces a specific write across those boundaries instead of treating every background operation as “saving data.”
Here, the goal is to explain an apparently missing write, reason about a crash, and choose a visibility policy without creating unnecessary indexing work.
Evidence boundary: Product guarantees are attributed to Elastic’s official documentation. Timelines, diagnostic choices, and interview exercises are original reasoning examples. No independently verified recent candidate reports establish that an employer asks these exact prompts. Examples assume a conventional Elasticsearch index; check deployment-specific settings before transferring them to Serverless or a different release.
Why can GET find a document that search misses?
Official fact: The document GET API is real-time by default. Its behavior differs from a search request over the index’s searchable view. Disabling real-time GET is a separate option. Elastic’s GET API reference.
Official fact: Elasticsearch provides near-real-time search. A refresh makes recent indexing changes available to search by opening a searchable view of new segments, without requiring a Lucene commit for every write. Near-real-time search.
That explains a possible sequence: document 42 is accepted, GET returns the new source, and a search issued before the relevant refresh still misses it. This observation alone does not prove data loss, a failed replica, or an incorrect mapping.
Start the interview answer by checking that both requests use the intended index or alias, document ID, routing, and query. A search with the wrong filter will remain wrong after refresh. A write to one backing index and a read from another is a destination problem. Establish those facts before diagnosing a timing problem.
Also avoid claiming every immediate search must miss. An automatic refresh may happen between requests. The correct statement is that ordinary write acknowledgment does not, by itself, establish the search-visibility boundary your application wants.
Trace one write through acknowledgment and visibility
Original exercise: Assume an isolated test index, no concurrent writes, the default request translog durability, and an ordinary indexing request with refresh=false. The table describes one possible ordering, not a universal execution schedule.
| Event | Observation | What you can conclude |
|---|---|---|
| Index document 42 | Write succeeds | Evaluate the response and configured acknowledgment contract |
| GET document 42 | New source is returned | Real-time retrieval can observe the write |
| Search before refresh | No matching hit yet | The searchable view can still be older |
| Refresh completes | Search returns document 42 | The new change is now visible to search |
| Process crashes before a later Lucene commit | Recovery is required | Durable translog operations can bridge the commit gap |
The last row deliberately separates “searchable” from “included in a Lucene commit.” A document can already be searchable while its recovery still depends on the translog.
Official fact: With the default index.translog.durability=request, success waits for the translog to be synchronized and committed on the primary and allocated replicas. Recent acknowledged operations not included in the last Lucene commit can be recovered from that log. Setting durability to async introduces a loss window for operations since the previous synchronization. Translog settings.
Inference for this exercise: If the process stops but the required durable storage remains available, “no flush yet” is insufficient evidence that the acknowledged document disappeared. Conversely, neither refresh nor journaling is a backup against losing every durable copy. State the failure model before promising recovery.
What do refresh, flush, and merge each do?
Use this distinction when the interviewer offers several plausible maintenance operations.
| Operation | Boundary it addresses | Poor reason to invoke it |
|---|---|---|
| Refresh | Recent changes become searchable | “I need a backup of this write” |
| Flush | Lucene commit and a new translog generation | “My application needs a precise search-after-write contract” |
| Segment merge | Combine segments and reclaim eligible deleted-document space | “The latest document has not appeared in search” |
Official fact: Flush persists operations into the Lucene index so recovery need not replay them from the translog. Elasticsearch automatically triggers flushes, balancing recovery work against commit cost; direct manual calls are rarely necessary. Old translog generations can be removed when no longer needed. Flush API.
The distinction is about which contract to use, not a claim that these operations have no internal interactions. If your application requires its indexed changes to be searchable before continuing, express that requirement through the refresh controls. Do not build the application around incidental side effects of another maintenance API.
Original interview follow-up: “Would flushing after every request be safer?” Ask what failure remains unaddressed by the existing durability settings. If the complaint is search lag, you have changed the wrong policy. If the complaint is recovery duration, investigate translog size and recovery behavior rather than making every request pay for a manual commit.
When should you choose refresh=wait_for?
Official fact: refresh=false takes no refresh action. refresh=true forces a refresh of the relevant shards. refresh=wait_for waits until a refresh exposes the request’s changes, normally without forcing one; exhausted refresh-listener capacity is an exception. With automatic refresh disabled, another action must trigger refresh for the waiting request to finish. Refresh parameter reference.
Original product decision: A user saves an article and immediately navigates to a search-backed management page. If the page must include the new article, request the visibility contract deliberately, or show the successful write directly while search catches up. Which choice fits depends on the product requirement; the database setting should follow that decision.
For a background log pipeline, waiting for every document may add latency with little benefit. Define the acceptable ingestion-to-search delay, batch writes, and measure whether the configured refresh policy meets that objective. “As fast as possible” is not a useful service target.
As Elastic explains in its refresh reference, forcing refresh repeatedly can create small segments and extra indexing, search, and merge work. Waiting also has a cost: the caller holds resources until the visibility condition is met. Compare throughput and end-to-end latency under the intended concurrency rather than assuming either option is free.
A useful interview answer names the observer. Does the user need GET by ID, a filtered search, an aggregation, or a background export? Those are different read paths. Requiring search visibility for an operation that only needs a returned document may spend resources without changing the user experience.
How do you make a search-after-write test deterministic?
A fixed sleep is a weak synchronization mechanism. It can pass on a quiet laptop and fail under load, or waste time long after visibility is established.
Original test recipe: In a disposable conventional index, first disable automatic refresh, create one explicitly mapped document, and verify successful indexing. Do not use this setting change as a production troubleshooting shortcut. The following requests illustrate the visibility boundary; run them against your target version to validate the behavior there.
PUT interview_write_path_demo
{"settings":{"refresh_interval":"-1"},"mappings":{"properties":{"tag":{"type":"keyword"}}}}
PUT interview_write_path_demo/_doc/42?refresh=false
{"tag":"write-path-demo"}
GET interview_write_path_demo/_doc/42
GET interview_write_path_demo/_search
{"query":{"term":{"tag":"write-path-demo"}}}
POST interview_write_path_demo/_refresh
GET interview_write_path_demo/_search
{"query":{"term":{"tag":"write-path-demo"}}}
With no intervening refresh, the intended contrast is GET finding 42 before search does, followed by search finding it after refresh. Use a fresh index name so old matching documents cannot make the assertion pass accidentally. Assert the returned ID and source, not merely a positive hit count.
For application tests that need only the final searchable state, a successful write with refresh=wait_for is often simpler, provided automatic refresh is enabled. The disabled-refresh fixture above is specifically for demonstrating the intermediate state. Combining that fixture with a waiting write and no independent refresh would leave the test waiting for an event it never schedules.
These snippets are an executable exercise design, not a report of a benchmark or a running Elasticsearch cluster used to produce this article.
What changes when writes arrive in bulk?
Official fact: Bulk responses contain individual operation results and an overall errors indicator. A successful HTTP exchange does not establish that every submitted item succeeded. The bulk API also supports refresh controls for the affected shards. Bulk API.
Original debugging case: A batch contains 1,000 documents; one item fails because its field cannot be parsed. Search returns 999 matching IDs. Refreshing again cannot recover the rejected item. Inspect the item response, correct the data or mapping mismatch, and decide how to replay that item safely.
Maintain the distinction between an accepted-but-not-visible item and a rejected item throughout your answer. Record IDs, per-item status, and an application correlation identifier so a partial failure can be reconciled. Retrying an entire batch without considering update semantics may repeat work or apply an update twice.
Batching also changes the waiting pattern. If one downstream step needs all successful writes to be searchable, prefer a deliberate batch boundary over a loop that waits after every item. Then handle failed items explicitly; a visibility barrier does not transform partial ingestion into complete ingestion.
Why is force merge usually the wrong rescue operation?
Official fact: Merges normally run automatically, combining segments and reclaiming deleted-document space. Elastic recommends force merging indices that no longer receive writes. Forcing large segments on an actively changing index can increase storage and maintenance costs; the operation also needs temporary disk space. Force merge API.
Recommended diagnosis: If a write-heavy index becomes slow, inspect indexing rate, refresh policy, segment counts, merge activity, disk headroom, and search latency together. More segments can be a symptom of the workload or an aggressive visibility policy. Launching a heavy maintenance operation without understanding the bottleneck can compete with the work already struggling.
For an older rolled-over index that has stopped receiving writes, a planned force merge may fit the lifecycle. That is a separate decision from making a new document searchable. Explain why the index is effectively read-only and how you will assess temporary resource use before suggesting it.
Practice the boundary, not just the vocabulary
| Practice question | What to rehearse |
|---|---|
| Design a Scalable Post, Feed, and Search Service | Define the delay allowed between publishing a post and finding it. |
| Design a scalable logging system | Choose a batching and refresh policy for newly ingested logs. |
| Build a Crash-Consistent Single-Server Key-Value Store | Separate a durable acknowledgment from a searchable reader. |
| Design distributed log storage service | Trace recovery from a log separately from background compaction. |
| Discuss Distributed-System Experience with Go, Java, Spark, and Elasticsearch | Explain Elasticsearch visibility guarantees without generalizing other systems. |
Then explain which failures that operation still does not cover.
Sources and Further Reading
- Elastic: Near-real-time search
- Elastic: Translog settings
- Elastic: Refresh parameter
- Elastic: Get a document
- Elastic: Flush API
- Elastic: Bulk API
- Elastic: Force merge API
Research checked September 21, 2026. Product behavior and supported settings can vary by version and deployment.