Articles
Filesystem Crash Consistency Interview Questions: fsync, Rename, and Durable Writes
Explain Linux durable writes with fsync, atomic rename, directory synchronization, crash-point reasoning, concurrent writers, and uncertain retry outcomes.
A configuration file is replaced successfully. The service returns “saved.” After a power failure, the replacement is missing. Nothing about that sequence contradicts the claim that rename was atomic: visibility during execution and durability after a crash answer different questions.
Filesystem crash consistency interview questions become easier when you state exactly what the application acknowledges, which objects it synchronizes, and what recovery is allowed to observe. This guide develops a single-file replacement protocol, then challenges it with short writes, failed synchronization, concurrent writers, and ambiguous retries.
Evidence boundary: Official facts below come from the Linux manual pages. Worked cases and suggested interview answers are original practice exercises and engineering inference. They are not candidate reports or evidence that a particular employer asks these questions. The scope is a Linux local filesystem and storage stack that supports and correctly honors the relevant synchronization operations; network filesystems, other operating systems, and faulty hardware need separate contracts.
For a related implementation exercise, use Design a Durable Key-Value Cache with File-System Persistence; the practice table below explains its connection to this topic.
What does a successful write actually promise?
Official fact: Linux write can transfer fewer bytes than requested. A successful call does not establish that the data has reached persistent storage; errors can surface later during another write, synchronization, or close. Read the write contract before equating a return value with a durable record.
Consider an original exercise: a 12 KB configuration is written through a buffered language stream. There are three distinct boundaries to explain. First, the language may hold bytes in its own buffer. Second, the kernel may have accepted bytes into its cache. Third, the required data and metadata may have been synchronized to storage. A method named flush does not automatically cover all three.
A useful interview answer is: “I check for complete transfer, flush any userspace buffer before synchronizing the descriptor, and acknowledge only after the synchronization operations required by my recovery contract succeed.” That sentence gives the interviewer something concrete to challenge.
For an unbuffered descriptor, sketch a loop that advances by the actual positive byte count. Handle interruption according to the API contract, reject a no-progress result, and propagate other errors. Do not retry an entire buffer after a partial transfer without adjusting its position: that can duplicate bytes. Buffered libraries may implement this loop internally, but you still need to understand their error and flush behavior.
The follow-up is often, “Why not just close it?” Closing releases a resource; it is not the durability boundary this design promises. Keep resource lifetime, complete transfer, and durable acknowledgment separate throughout the answer.
Why is fsync on the file insufficient for a replacement?
Official fact: Linux fsync synchronizes file data and associated metadata, but does not necessarily synchronize the directory entry naming that file. The manual explicitly calls for synchronizing the directory too. fdatasync can omit metadata unnecessary for later data retrieval; file-size changes remain relevant. See the fsync and fdatasync documentation.
Official fact: Replacing an existing destination with rename is atomic with respect to that destination name: readers do not encounter a moment when the destination is absent solely because of the replacement. Existing open descriptors remain usable. Cross-filesystem renames can fail with EXDEV. These are rename semantics, not a power-loss persistence guarantee.
Engineering inference: For the bounded Linux design here, create a unique temporary file in the destination directory, finish its contents, synchronize that file, replace the destination name, then synchronize the parent directory before reporting durable success. Same-directory placement avoids a cross-filesystem move and keeps the namespace change within the directory being synchronized.
An atomic replacement is also different from updating in place. Truncating the destination and then writing exposes readers to an incomplete document. A temporary file keeps preparation away from the current name until replacement. That benefit still holds even before you have discussed crash recovery.
If a reader opened the old destination before replacement, it can continue reading the old file through its descriptor. A newly opened destination can refer to the replacement. Neither behavior proves a lost update or broken rename. Ask whether the application needs snapshot-like reader behavior, and define which generation each read is expected to use.
Walk through a durable replacement protocol
Use this original prompt: “Persist generation 42 of settings.json. The caller must not receive success unless that generation can be recovered after a system crash.” Assume the previous generation is already durable and the containing directory already exists.
The protocol is an ordered sequence, not four interchangeable calls:
- Open the existing parent directory and create an exclusive, uniquely named temporary file inside it.
- Write the complete replacement, including any generation identifier and integrity information the recovery format requires.
- Flush language buffers if applicable, apply required file metadata, and synchronize the temporary file.
- Rename the temporary name over
settings.jsonwithin that directory. - Synchronize the parent directory, then acknowledge durable success.
A checksum detects corruption; it does not make data durable. A generation identifier helps identify what was recovered; it does not serialize competing writers. State why each field exists instead of presenting it as a magic reliability feature.
Treat file ownership, mode, and access requirements as part of the intended replacement. A newly created temporary file does not inherit every attribute of the overwritten file automatically. If those attributes matter to the recovered state, establish them before the relevant synchronization boundary.
This is a protocol outline, not a universal library implementation. A production implementation must also address path handling, symlinks, cleanup, descriptor ownership, permissions, and the target filesystem's documented behavior. In an interview, name these boundaries without letting them obscure the durability argument.
What happens if the system crashes between those steps?
The following original ledger separates what a running observer can see from what the application has established for recovery. “Not established” deliberately avoids guessing a universal post-crash state for every filesystem.
| Last completed operation | Running destination name | Durable success established? | Recovery reasoning |
|---|---|---|---|
| Temporary write only | Previous generation | No | New bytes may remain only in buffers; discard or inspect leftovers |
| Temporary file synchronization | Previous generation | No | Prepared contents do not establish the destination namespace change |
| Rename returns | Replacement generation for new opens | No | Visibility is established; the directory change has not been synchronized |
| Parent directory synchronization succeeds | Replacement generation | Yes, under stated assumptions | Contents and the required naming change have crossed the acknowledgment boundary |
| Reply transmission completes | Replacement generation | Already established | Caller knowledge can still differ if the reply is lost |
A strong answer explains why the third row is dangerous: the process can observe the new file and still lack the durability evidence required to promise it after a crash. Reading the file back immediately does not replace the missing synchronization.
Do not claim that every unsynchronized crash yields exactly the old or new version. That statement requires a stronger filesystem-specific recovery contract than atomic runtime replacement alone supplies. Instead, identify the desired invariant and the barriers on which it depends.
Another follow-up changes the setup: the application creates a new directory before writing the file. Now the durability of that directory's own entry in its parent matters too. The earlier protocol assumed an existing durable directory. Calling that assumption out is more useful than memorizing “always fsync one directory.”
For a move between two directories, both namespaces change. Revisit which directory entries need persistence and whether the application can tolerate the intermediate recovery outcomes. Do not casually extend a same-directory replacement proof to an arbitrary multi-directory operation.
What should the application do when synchronization fails?
Suppose the rename succeeds, but the parent-directory synchronization returns an I/O error. The new generation may be visible. The application cannot honestly return the durable-success result it promised. It also cannot safely infer that nothing changed.
Engineering inference: Model this as an uncertain durability outcome and preserve enough operation identity to reconcile it. A failed synchronization is not an automatic rollback. Renaming the old contents back is another write protocol with its own failure window, and may overwrite a concurrent update.
| Failure point | Safe statement to the caller | Useful recovery action |
|---|---|---|
| Complete-write check fails before rename | Replacement has not been committed by this protocol | Keep the existing destination; remove temporary data when safe |
| File synchronization fails | Prepared replacement is not acknowledged as durable | Record the error and avoid publishing the temporary file |
| Rename fails | No successful replacement has been established | Inspect the actual error; do not substitute copy-and-delete silently |
| Directory synchronization fails after rename | Replacement may be visible; durable commit is uncertain | Reconcile generation and operation identity under the writer's coordination rule |
| Durable commit succeeds but response is lost | Commit can exist without caller confirmation | A retry should recognize the already committed operation |
If the API supports retries, include an operation identifier and a defined rule for checking whether generation 42 belongs to that request. A blind retry that writes generation 42 over generation 43 is not idempotent merely because the bytes are unchanged.
Also distinguish operational cleanup from correctness. A leftover temporary file can be harmless if recovery ignores uncommitted names. Removing all temporary-looking files indiscriminately can harm a live writer. Choose a naming and ownership convention that recovery can interpret safely.
How do concurrent writers change the answer?
Two writers can each prepare and synchronize a complete file, then rename over the same destination. Every replacement may be atomic, yet one logical update can be lost. Atomic naming does not implement compare-and-swap on a generation number.
Original exercise: writer A reads generation 41 and changes a timeout. Writer B reads generation 41 and changes a retry limit. B publishes generation 42, then A publishes its own generation 42 without B's change. The filesystem protocol can be crash-safe while the application update is incorrect.
Engineering inference: Define the concurrency contract separately. A cooperating single-writer service can serialize the read-modify-publish sequence. A lock can protect cooperating writers if held across the relevant checks and publication. An optimistic version check needs an actual atomic coordination mechanism; checking a generation and then renaming leaves a race.
Unique temporary names prevent preparation collisions but do not prevent destination overwrites. Similarly, O_APPEND is relevant to append behavior, not a complete multi-record transaction or generation update scheme. Ask whether the question is about replacing snapshots, appending a log, or coordinating logical updates.
For group commit, explain which callers share a synchronization barrier and when each is acknowledged. Batching can reduce synchronization overhead, but an early success response changes the durability contract. A performance answer must preserve the promised acknowledgment boundary.
How would you test the crash-consistency claim?
Start with the invariant: after recovery, every acknowledged generation must be recoverable according to the declared contract. Unacknowledged operations may have committed, so the test cannot assume their absence. Validate generation identifiers, lengths, checksums, and application-level consistency, not just file existence.
Use deterministic error injection around complete transfer, file synchronization, rename, directory synchronization, and reply delivery. Include partial writes and errors after namespace replacement. These tests exercise control flow and caller-visible outcomes.
Then use an appropriate Linux crash-testing environment to test persistence behavior on the target filesystem and storage configuration. Killing a process leaves the kernel running and able to flush data; it does not simulate every kernel crash or power-loss condition. A unit test on another operating system cannot certify the Linux durability protocol either.
For a multi-file update, ask what recovery uses as the commit record. Renaming two independent files sequentially does not make their pair atomic. You may need a durable manifest, log, or transactional storage engine with a documented recovery protocol. The key is to define the recovered unit before selecting synchronization calls.