Linux File Descriptor Interview Questions: Leaks, Limits, and Deleted Files Using Disk

A Linux process can fail with “Too many open files” while the machine still has memory and disk space. A filesystem can report that a large file was deleted while df shows no space recovered. Both incidents make sense once you separate file descriptors, open file descriptions, directory entries, and inodes.

This guide builds that evidence chain for interviews. Start with Linux Host and Filesystem Fundamentals if you want a broader warm-up.

Evidence boundary: Linux man-pages and kernel proc documentation are the official technical sources below. No employer-specific interview pattern is claimed because candidate reports were insufficiently consistent.

Build the object model first

open() returns the lowest available file descriptor number in the process. That integer indexes the process's descriptor table. The entry refers to an open file description, which carries file offset and status flags and ultimately references a file, pipe, socket, device, or other kernel object.

After fork, processes may share references to the same open file description. After dup, two descriptor numbers in one process can refer to the same description. Closing one descriptor removes one reference; the underlying object remains while other references exist.

/proc/<pid>/fd/ exposes one symbolic link per open descriptor. Descriptors 0, 1, and 2 conventionally represent standard input, output, and error, but services can redirect or close them. Socket and pipe links use kernel identifiers rather than ordinary paths.

This model prevents the weak statement “an FD is a file.” An FD is a per-process handle and may represent many object types.

Distinguish the two limit failures

Linux open(2) documents:

• EMFILE: the process has reached its per-process open-file-descriptor limit.
• ENFILE: the system-wide limit on open files has been reached.

RLIMIT_NOFILE is one greater than the largest descriptor number the process may open. A shell's ulimit -n displays or sets a limit for that shell and descendants, but service managers, containers, and runtime configuration can establish different effective limits.

Collect evidence from the affected process:

pid=<service-pid>
cat /proc/$pid/limits
ls /proc/$pid/fd | wc -l
cat /proc/sys/fs/file-nr
cat /proc/sys/fs/file-max

Do not raise the limit before identifying growth. A higher limit may delay failure while the leak consumes more kernel memory or exhausts a downstream service.

Diagnose descriptor growth by type

Take repeated snapshots rather than one count:

ls -l /proc/$pid/fd > /tmp/fd.$(date +%s)
lsof -nP -p "$pid"

Group targets into sockets, pipes, regular files, anonymous inodes, and devices. Then correlate growth with request rate, connection state, worker count, deployments, and error paths.

For sockets, use ss -tanp and inspect states. Many CLOSE_WAIT sockets suggest the peer closed but the local application has not closed its side. Many TIME_WAIT sockets reflect closed connections retained by the TCP stack and are not, by themselves, proof of an application FD leak. Verify whether the process still owns them.

For files, compare paths and flags. Log rotation, temporary files, watchers, and repeated configuration loads are common suspects. For pipes or anon_inode targets, inspect the runtime: eventfd, epoll, inotify, and asynchronous libraries have distinct ownership paths.

Explain why deleted files still use space

unlink() removes a name from a directory. Linux man-pages state that if it was the last link and a process still has the file open, the file remains until the last descriptor is closed. The pathname can disappear while the inode and data blocks remain allocated.

That explains this incident:

1. A service opens /var/log/app.log and keeps writing.
2. A rotation or operator unlinks the path.
3. The process continues writing through its open descriptor.
4. ls cannot find the path, but df still counts allocated blocks.
5. lsof +L1 or a /proc/<pid>/fd symlink marks the target as (deleted).

Use:

lsof +L1
ls -l /proc/$pid/fd | grep '(deleted)'
du -sh /var/log
df -h /var/log

du walks reachable directory entries, while df reports filesystem block allocation. Deleted-but-open files are one reason they differ. Reserved blocks, sparse files, snapshots, mounts, and permissions are other reasons, so confirm the open reference.

The safest recovery is usually to make the owning process close and reopen the file through its supported rotation or reload path, or restart it with understood impact. Truncating via /proc/<pid>/fd/<n> can free blocks but may violate application assumptions, race with writes, and destroy needed logs. Present it as an emergency operation requiring ownership approval, not the default fix.

Separate descriptor leaks from connection leaks

A database or HTTP connection leak is often an FD leak because sockets use descriptors. Yet pool exhaustion and FD exhaustion are different limits. A pool of 20 leaked database connections may block application requests while the process has thousands of free descriptors. Conversely, repeated file opens can hit RLIMIT_NOFILE while the database pool is healthy.

Build a layered timeline: process FD count, socket states, connection-pool active count, request rate, error types, and limit headroom. If FD count rises monotonically across steady load and never falls after requests finish, ownership is suspect. If it scales to a stable plateau with concurrency, it may be intended pooling.

Find the code path that owns the leak

System symptoms identify type and growth, not the line of code. Add application counters for resource creation and close, tagged by call site or component. Use tracing or sampling carefully; logging every open at high volume can create a second incident.

In code review, inspect all exits:

• success and ordinary close;
• exceptions after open but before registration;
• cancellation and timeout;
• partial initialization of several resources;
• retry loops that replace a handle without closing the old one;
• worker shutdown and hot reload;
• ownership transfer between layers.

Language constructs such as Python with, Java try-with-resources, Go defer placed immediately after a successful open, and Rust RAII reduce risk, but asynchronous lifetime and ownership transfer still need tests.

Design a reproducible test

Run a bounded local test, not an uncontrolled production experiment. Record baseline FD count. Execute the target operation hundreds of times with successes, injected failures, timeouts, and cancellations. Allow legitimate pools to settle, then compare the final count and target distribution.

For a deleted-open-file demonstration, create a temporary file, open it, unlink it, and inspect /proc/self/fd/<n>. The descriptor remains readable until close. Keep the fixture small and inside a temporary directory. That proves the kernel lifetime behavior without consuming meaningful disk.

An invariant can be stronger than “count did not increase”: after each completed request, all request-scoped descriptors are closed; only a bounded named pool may remain; every cancellation path reaches cleanup; deleted-open bytes are zero after rotation settles.

Reason about inherited and duplicated descriptors

A child process can inherit descriptors across fork, and descriptors remain across exec unless marked close-on-exec. Linux provides O_CLOEXEC so a program can request the flag atomically when opening. Setting it later with another call can race in multithreaded programs: one thread may fork and exec between the open and flag update, leaking the descriptor into a child.

This matters in servers that launch helper processes. The parent may close its handle correctly, yet an unexpected child keeps another reference alive. A deleted file then continues consuming blocks even after the original service rotates it. Inspect all owning PIDs reported by lsof, not only the process that created the file.

Duplicated descriptors also affect counts. dup, dup2, and library abstractions can create several descriptor numbers for one open file description. Closing one number does not release the object while another reference remains. When debugging, group by inode or socket identity as well as descriptor number.

Explain limits in containers and services

The value printed by your interactive shell may differ from the production process. systemd LimitNOFILE, container runtime settings, parent-process limits, and privilege rules determine the effective soft and hard limits. Read /proc/<pid>/limits for the affected PID.

Containers share the host kernel even when process and mount views are isolated. A container can hit its process limit, while many containers together pressure system-wide file tables or a downstream service's connection cap. The remediation therefore needs both container and host evidence.

Increasing a soft limit within the hard limit may be possible for the process; raising the hard limit requires appropriate privilege. Even with a high FD limit, applications can encounter ephemeral-port, conntrack, database, or memory limits first. Capacity planning should name the smallest bounding resource.

Make containment explicit

If the leak is gradual, drain one instance, capture its descriptor distribution, restart it, and compare the slope on the fixed canary. Preserve enough evidence before restart because the descriptor table disappears with the process. If the system-wide table is near exhaustion, prioritize restoring headroom while avoiding a fleet-wide simultaneous restart.

Rate limiting or disabling the leaking feature can slow growth. Closing arbitrary FDs from a debugger is risky because descriptor numbers can be reused and libraries maintain their own state. Prefer the application's supported shutdown or resource lifecycle.

Interview prompts and strong answers

“The service says too many open files. What do you do?” Identify EMFILE versus ENFILE, read the affected process's actual limits, snapshot FD types over time, correlate growth with traffic and code paths, then contain and fix ownership.

“Why didn't deleting a 50 GB log free space?” The pathname was unlinked, but a process still holds the inode open. Confirm with lsof +L1 or /proc/<pid>/fd, then cause a supported close/reopen.

“Can you fix it with ulimit?” Raising the soft limit can provide temporary headroom when the workload legitimately needs more descriptors, but it does not repair a monotonic leak and may move failure to a system or downstream limit.

“What does CLOSE_WAIT mean?” The peer's FIN was received; the local application has not closed its socket. A growing set owned by the process supports a missing-close hypothesis, but capture stack or component evidence.

“How do you prove the fix?” Repeat a workload with injected errors and cancellation, assert descriptor count returns to a stable bound, inspect target types, and monitor the same production signal after canary deployment.

Avoid common diagnostic mistakes

Counting /proc/<pid>/fd once does not prove a leak. lsof is a snapshot and can race with rapid opens and closes. TIME_WAIT does not equal leaked application descriptors. Deleted files are not the only explanation for du/df mismatch. A process restart proves that releasing descriptors helps, not which code path leaked.

A concise answer is: “I identify the failing limit, observe descriptor count and type over time, and connect targets to application ownership. For missing disk after delete, I prove the inode is still open. I contain by closing through a supported lifecycle, then test every success, exception, timeout, and cancellation path until resources return to a stable bound.”

Sources and Further Reading

• Linux open(2)
• Linux getrlimit(2)
• Linux unlink(2)
• Linux proc_pid_fd(5)