Articles

strace Interview Exercises: Diagnose Hung Processes, Missing Files, and Failed Connections

Practice strace interview exercises with three syscall traces: a waiting thread, failed file lookups, and refused connections, plus the next test.

In a strace interview exercise, the best answer is rarely “this syscall is the bug.” A trace shows a process's calls across the user/kernel boundary, their arguments, return values, and sometimes time spent in the calls. That can rule out several stories quickly. It cannot, by itself, tell you why application code chose an address, expected a pathname, or waited for a particular event.

Here are three original, synthetic traces: a service that appears hung, a missing configuration file, and a failed connection. For each, state what is observed, what remains unknown, and the next measurement. The linked prompts are practice, not a claim that an employer will give these traces.

What strace can and cannot establish

The strace manual describes tracing system calls, signals, arguments, and return values. A successful openat proves the kernel returned a descriptor for that call. An ENOENT proves that pathname resolution failed in the traced process's context at that moment. Neither proves the application used the right configuration path.

Likewise, a long wait call can show that a thread waited for an event. It does not prove the entire multi-threaded service was idle, nor that the awaited event ought to have arrived. User-space CPU loops might produce little syscall activity. A trace is therefore one layer of evidence; a profiler, thread dump, application logs, and network observations may be needed.

The interviewing skill is a disciplined sentence: “This line proves X, it does not prove Y, so I would check Z.” Use it for every trace rather than storytelling past the available facts.

Before attaching: bound the diagnostic

Ask which process and namespace you are investigating, whether the symptom is reproducible in a safe environment, and whether tracing is approved. Attaching can require appropriate permissions, alter timing, and expose arguments, paths, or data. Do not casually trace a production process containing credentials or customer payloads. Prefer a disposable reproduction or an approved, narrowly scoped session.

The manual documents -p for attaching to a process and -f for following relevant process creation. -tt adds high-resolution wall-clock timestamps, -T shows time spent in each syscall, and -yy decodes additional descriptor information where available. Filters such as -e trace=file or -e trace=network can reduce noise; selecting only one class may hide the call needed to understand the symptom. The exact flag combination should match the question and the installed strace version.

When following child processes, keep process IDs in the output or separate trace files so a call from a helper is not attributed to the parent. Establish start and end time, capture enough context around the failure, and avoid dumping an unbounded trace into a shared log. These operational choices often matter more than memorizing every syscall name.

Exercise 1: “the process is hung”

Consider this synthetic excerpt from one thread:

12:00:00.000001 epoll_wait(7, [], 64, 30000) = 0 <30.000121>
12:00:30.000190 epoll_wait(7, [], 64, 30000) = 0 <30.000087>

The trace shows that this thread waited twice on descriptor 7 and each wait timed out without a reported event. A wait call returning zero after its timeout is not itself an error. The analogous poll manual distinguishes descriptor readiness, signal interruption, and timeout, and notes that a negative timeout can wait indefinitely. The excerpt does not say what fd 7 represents, what should wake it, or what other threads are doing.

A strong first answer is: “One traced thread is idle in an event wait, not proven deadlocked. I would identify the descriptor and the expected producer, then inspect all relevant threads or a thread dump.” If the service accepts requests but never completes them, check the request worker rather than only the event-loop thread. If no requests arrive, inspect listener state and upstream routing.

Now change the excerpt to an unfinished blocking read on a pipe. The hypothesis changes: perhaps this thread waits for a writer that never sends data or closes its end. Follow descriptor ownership, process relationships, and the producer's behavior. Do not label every blocking call a deadlock; a deliberate server loop is supposed to block while idle.

Exercise 2: a missing configuration file

Here is a second synthetic excerpt:

openat(AT_FDCWD, "config/service.yaml", O_RDONLY) = -1 ENOENT
openat(AT_FDCWD, "/etc/service/service.yaml", O_RDONLY) = -1 EACCES

The first path is relative to the traced process's current working directory because AT_FDCWD was supplied. ENOENT means a pathname component was not found in that context; it does not mean a similarly named file does not exist somewhere on the host. The second call reached a different absolute path but failed an access check. The Linux open manual documents ENOENT and EACCES behavior, including directory-search permissions.

Ask which path the application intended to use. Inspect the process's working directory, its container or mount namespace, symlink targets, file existence, owner and mode on every path component, and the identity under which the process runs. If you can safely reproduce in the same runtime, compare the intended configuration location with the actual call. A developer showing you the file in their shell does not establish that the service sees it in its namespace.

Do not prescribe chmod 777 based on EACCES. That is both overbroad and possibly ineffective: the denial may be on a parent directory or caused by other controls. First establish the denied path and the least-privilege permission the service requires. Similarly, do not “fix” ENOENT by copying arbitrary configuration files into every candidate directory.

Exercise 3: a failed connection

Consider this synthetic network trace:

connect(9, {sa_family=AF_INET, sin_port=htons(5432),
            sin_addr=inet_addr("10.0.0.8")}, 16) = -1 ECONNREFUSED

The process attempted to connect to 10.0.0.8:5432 and the call failed with ECONNREFUSED. That is stronger evidence than “the database connection is broken,” but weaker than “the database server crashed.” The address may be wrong, the service may not be listening there, a local or remote policy may reject it, or the target may be reachable only through a different network route. The connect manual documents its return and error behavior.

Distinguish other outcomes. A timed-out connection points toward a different set of network and policy checks from an immediate refusal. In nonblocking code, EINPROGRESS is not final failure; the application must wait for completion and inspect the socket status. An error on a later read or write is not interchangeable with a connect error. Interviewers value that temporal separation.

Your follow-up should include the configured hostname, resolved addresses, destination port, listener state on the intended server, and the relevant network path. Verify from the same host or container context as the process. If the client retries, record whether each attempt uses the same address and error; a single failed call does not describe the whole retry policy.

A three-case evidence matrix

Use this matrix as the second exercise asset. For each row, practise saying the “does not prove” column aloud before proposing a repair.

Trace observation	Proves	Does not prove	Next discriminating check
epoll_wait times out twice	One thread saw no reported event in two waits.	The whole process is deadlocked.	Descriptor identity, other threads, expected event producer.
openat relative path returns ENOENT	Lookup failed from this process context.	No file exists anywhere with that name.	Working directory, namespace, symlink and path components.
openat returns EACCES	Access was denied for that operation.	World-writable permissions are needed.	Process identity and every path component's access policy.
connect returns ECONNREFUSED	That connection attempt was refused.	The application's entire dependency is down.	Configured endpoint, listener, routing and retry sequence.

This format prevents a classic interview error: describing an application root cause as though the syscall line contained it. It also keeps the next action testable.

Read the timing fields carefully

With -tt, each reported trace line carries a wall-clock timestamp; with -T, it shows time spent in the syscall. Those are useful but not a complete latency profile. Time outside syscalls, scheduling, child-process work, buffering, and clock behavior can affect end-to-end latency. Summing a few -T values does not necessarily equal a user request's duration.

For an apparently hung process, compare timestamps across threads or child processes before saying where time went. A thread might wait while another uses CPU. If the program appears CPU-bound but the trace is sparse, a CPU profiler is a more direct next tool. If the problem is lock contention, a thread dump or language profiler may expose synchronization that a syscall filter misses.

Descriptor decoding with -yy can make an fd more legible, but the displayed association is still a snapshot of that process's descriptor table. File descriptors can be closed and reused. Connect an fd to nearby open, socket, accept, or close events and process ID; do not assume “fd 9” means the same socket forever.

How would you run a safe interview reproduction?

The interview does not require a production attachment. Build three tiny disposable programs or ask for a supplied trace: one sleeps in an event wait, one attempts a deliberately absent path and a restricted path, and one connects to a test port with no listener. Run each with a bounded trace and compare the observed output to the synthetic excerpt. This article does not claim those Linux experiments were executed on this macOS host.

Record the command line, Linux and strace versions, process ID, container context, filters, and exact symptom. Redact private paths and addresses before sharing a trace. If tracing changes the timing of a race, say so; retry under a lower-overhead tool or use application instrumentation. A good diagnostic is reproducible and appropriately scoped, not simply verbose.

An answer that earns trust includes a stop point: once the trace shows a clear boundary, gather the one other data source that can validate the cause. Do not keep expanding a potentially sensitive trace because it is easy to collect.

The interview response to rehearse

“I would scope a short trace to the affected process and the relevant syscall family, then read arguments, return values, timing, and process identity. The wait line tells me one thread waited; the file error tells me resolution or access failed in that process's context; the connect error tells me what happened to one endpoint attempt. None alone proves the application root cause. I would choose one follow-up check that distinguishes my leading hypotheses, then verify the symptom changed after the fix.”

Sources and Further Reading

Research note: Checked September 28, 2026. The excerpts above are synthetic teaching fixtures; actual syscall names and output vary by program, kernel, architecture, and strace version.