Articles
TCP Troubleshooting Interview Questions: Retransmissions, Timeouts, and Packet Evidence
Prepare for TCP troubleshooting interviews with sequence-number evidence, retransmissions, layered timeouts, and capture limitations.
A client times out while sending an order. The packet capture contains three retransmissions, so someone concludes that the server never received the request. Another engineer sees a TCP acknowledgment and concludes that the order succeeded. Both conclusions go beyond the evidence.
Strong TCP troubleshooting interview answers identify the last transition the available observations actually establish. They connect sequence numbers, acknowledgments, timers, and capture locations to a specific hypothesis. They also explain what remains unknown about the application.
Evidence boundary: Protocol facts below come from IETF standards, while tool behavior comes from official Wireshark documentation and the Linux socket utility manual. The packet ledger and troubleshooting exercises are original practice scenarios, not candidate reports or records of a particular employer's interview. Start with Explain the TCP Three-Way Handshake and Its Guarantees, then explain how the same evidence discipline applies after establishment.
What does a TCP retransmission actually prove?
Official protocol fact: TCP tracks a byte stream using sequence numbers and acknowledges received sequence space. Its transport acknowledgment does not describe a database transaction or business response. The core specification also treats connection establishment, data transfer, and connection termination as distinct state transitions. RFC 9293
Interview reasoning: A retransmission tells you that the sender repeated sequence space. It does not, by itself, locate loss. The original data might have disappeared before reaching the receiver, the acknowledgment might have disappeared on the return path, or the sender might have retransmitted before a delayed acknowledgment arrived.
Begin by describing your observation precisely: “At the client capture point, bytes in this range appeared twice, and I have not yet seen an acknowledgment advancing beyond that range.” That sentence leaves room for incomplete capture and avoids blaming a firewall prematurely.
Then ask where the capture was collected. A host capture, a switch mirror, and a load balancer capture observe different boundaries. If a proxy terminates TCP, the client-to-proxy connection and proxy-to-backend connection have different sequence spaces. Correlate them with timestamps and application identifiers; do not match their sequence numbers directly.
Can you read this sequence-and-acknowledgment ledger?
Original exercise: Assume a connection is already established, no sequence-number wraparound occurs, and the client is sending a 400-byte request. The numbers are simplified absolute values, not a captured production trace.
| Observation | Meaning supported by the observation | What remains unresolved |
|---|---|---|
| Client sends SEQ 1001, LEN 400 | The segment carries bytes 1001 through 1400 | Whether the receiver saw it |
| Server-side capture sees that segment | Those bytes reached that observation point | Whether the application read them |
| Server sends ACK 1401 | Its cumulative acknowledgment advances through byte 1400 | Whether an order committed |
| Client capture lacks ACK 1401 | The capture does not show that acknowledgment | Return-path loss versus capture incompleteness |
| Client repeats SEQ 1001, LEN 400 | The same sequence range is transmitted again | Why the sender still considered recovery necessary |
Explain the arithmetic aloud: 1001 plus 400 equals 1401, the next expected byte. Do not say “packet 1401.” TCP acknowledgment values refer to sequence space, not an application message counter.
Now change one observation: the server capture never sees the first segment but sees the retransmission. Your leading hypothesis moves toward a forward-path or observation problem. Change another: both captures show the original data and acknowledgment, but the application still times out. Investigate response production, subsequent bytes, and the timeout's actual owner before changing transport settings.
A useful interviewer follow-up is whether retrying the order is safe. The transport ledger cannot answer that. You need an application request ID, a durable result lookup, or an idempotency contract. A TCP acknowledgment is insufficient evidence for either “retry freely” or “the order definitely succeeded.”
Where did connection establishment stop?
Avoid the single category “cannot connect.” Break the attempt into observable transitions and record both directions.
| Evidence at a known observation point | Working hypothesis | Next distinguishing observation |
|---|---|---|
| Client emits SYN repeatedly, no reply visible | Forward path, reverse path, listener, or capture problem | Compare arrival and response at the destination boundary |
| Destination sees SYN and emits SYN-ACK | A listener responded at that point | Look for SYN-ACK arrival at the client |
| Client sees SYN-ACK and emits final ACK | Client progressed through its handshake | Confirm destination receives the final ACK |
| Immediate RST appears | A reset was generated somewhere on the path | Identify its source boundary and correlate listener state |
| Connection establishes, then request stalls | Failure is later than initial establishment | Trace request bytes, receiver window, and response timing |
Original diagnostic method: Write the endpoint addresses and ports beside every observation. Account for address translation and whether the connection crosses a proxy. Otherwise, a capture of the wrong backend can look like proof of missing traffic.
A successful test from your laptop is a comparison, not a reproduction, when the failing process runs in a different network namespace. Run the narrow test from the workload's actual environment. Preserve the original source, destination, protocol, and timing when possible.
Which timeout expired: TCP, the socket, or the request?
Official timer fact: RFC 6298 derives the retransmission timeout from smoothed round-trip time and variation, and specifies backoff when its retransmission timer expires. The document's timer algorithm is not a universal application deadline or a guarantee that every deployed stack uses identical observable retry intervals. RFC 6298
Interview reasoning: Build a timeline with separate clocks. A client might enforce a two-second total request deadline while a transport continues its recovery attempt. A proxy might have a different idle timeout. A connection-pool wait might consume most of the budget before any SYN is emitted.
For each timeout, ask four things: who starts it, which event resets it, what action expiration triggers, and whether cancellation reaches downstream work. A message saying “request timed out” does not answer any of those questions.
Suppose the client deadline fires at two seconds and the server commits at three seconds. Extending the deadline may reduce visible errors, but it does not resolve ambiguous outcomes or repeated submissions. The diagnosis should distinguish a genuinely slow operation from a lost response and should preserve request identity across a controlled retry.
Avoid calculating a production timeout budget solely from a memorized retransmission sequence. Observe the operating system and application configuration. State which values are measured, which are configured, and which are hypothetical numbers introduced for the exercise.
Is the sender blocked by congestion or by the receiver?
Official distinction: Congestion control constrains how much the sender puts into the network, while receiver flow control reflects the receiver's available window. RFC 5681 describes congestion-window behavior and its relationship to the receiver-advertised window. These controls address different constraints. RFC 5681
Original scenario: An upload is fast for several seconds and then nearly stops. The receiver advertises a zero window while its application is paused. Calling this packet loss misses the strongest evidence: the receiving side is currently advertising no space for ordinary new data.
Investigate whether that application is reading its socket and what prevents progress. A saturated processing queue, blocked worker, or downstream stall can be more relevant than network capacity. Treat those as hypotheses until application observations support one of them.
For a contrasting case, the receiver window remains open but delivery becomes irregular during path contention. Congestion or loss recovery becomes more plausible, but the capture alone may not expose the sender's complete congestion state. Combine packet evidence with host measurements instead of assigning a cause from throughput shape.
The Linux ss utility can expose TCP socket information, including internal details with -i; use its installed-version documentation to interpret available fields. ss manual
An illustrative read-only starting point is ss -tin. Narrow the output to the relevant socket before interpreting it. A host-wide snapshot with hundreds of connections is a poor substitute for identifying the failing connection.
Can Wireshark's labels be misleading?
Official tool behavior: Wireshark's TCP analysis uses the packet sequence it observed and applies defined conditions for retransmission, out-of-order, and related labels. Those labels describe its interpretation of the capture, not an independently verified network root cause. Wireshark TCP analysis
The documentation also explains that checksum offloading can make locally captured outbound packets appear to have invalid checksums before hardware completes them. A local checksum warning is therefore not enough to conclude that corrupt packets crossed the wire. Wireshark checksum documentation
Interview reasoning: Check capture completeness before trusting an alarming color. Did capture start before establishment? Were packets dropped by the capture process? Was the filter too narrow? Did a mirror observe both directions? Are timestamps comparable across machines?
In a practice trace, an acknowledgment covering bytes you never saw should trigger a question about missing observations. Do not immediately invent a protocol violation. Similarly, a giant locally captured segment may require investigation of segmentation offload before you conclude that the network transmitted an oversized frame.
Preserve the original capture and document any display filters. Analysis should be reproducible: another engineer should be able to find the same byte range, direction, and timestamp without relying on your screenshot alone.
How would you present a defensible incident answer?
Use a short evidence chain: symptom, boundary, competing explanations, discriminating test, and remaining uncertainty.
Original model answer: “The client repeats bytes 1001–1400. The server-side capture sees the original request and an ACK advancing to 1401, but the client capture does not show that ACK. I would compare return-path observations and capture-drop counters before assigning network loss. Independently, I would query the order by request ID because transport delivery cannot establish its business result.”
A safe test changes one variable at a time. Compare one failing connection with one successful connection from the same workload environment. Capture a bounded interval around an agreed reproduction. Define success as the expected response and business result, not merely a lower retransmission count.
The stop condition matters too. If capture overhead affects the workload or the reproduction risks duplicate transactions, stop and use a test endpoint or existing telemetry. More packets are not automatically better evidence.