Articles
IPsec VPN Interview Questions: IKE Negotiation and Tunnels With No Traffic
Prepare IPsec VPN interviews with IKEv2 and Child SAs, route and selector checks, directional packet counters, NAT traversal, and MTU failure scenarios.
An IPsec VPN can show “connected” while the application behind it still times out. In an interview, the useful answer starts by separating three questions: did the peers negotiate security, did this particular packet enter the protected path, and did the remote service return a response? A green tunnel indicator answers only part of that investigation.
This guide focuses on IKEv2 site-to-site debugging. Official facts come from protocol specifications and identified vendor documentation. Candidate-origin practice records appear in the five-question table; they do not establish a universal IPsec interview format. The counter traces and troubleshooting recommendations are original exercises and engineering inference, not captures from a production incident.
What does successful IKE negotiation actually prove?
Official protocol fact: IKEv2 establishes an IKE security association for authenticated control exchanges and negotiates Child SAs for protected traffic. The usual initial exchange consists of IKE_SA_INIT followed by IKE_AUTH. The first negotiates cryptographic parameters and exchanges key material; the second authenticates peers and normally creates the first Child SA. IKE authentication can succeed while Child-SA creation fails. Additional Child SAs and rekeys use CREATE_CHILD_SA. These are IKEv2 exchange names, rather than a direct renaming of IKEv1 “phase 1” and “phase 2.” RFC 7296
For an interview explanation, attach a proof boundary to each event:
| Observed result | Useful conclusion | Question still open |
|---|---|---|
| IKE_SA_INIT response | A peer responded to this negotiation attempt | Can identities authenticate and establish usable traffic protection? |
| IKE_AUTH succeeds | An authenticated IKE control relationship exists | Did the intended Child SA also succeed? |
| Intended Child SA exists | Negotiated traffic-protection state is installed | Does the actual business flow select and traverse it? |
| Remote application responds | This tested flow completed at that moment | Will other addresses, ports, sizes, or later rekeys work? |
This timeline prevents a common answer failure: diagnosing the application from the control-plane status alone. Ask the interviewer what “up” means on the device: an IKE session, a Child SA, a tunnel interface, or a health probe. Those indicators need different follow-up evidence.
Which addresses and policies should you inspect?
Official architecture fact: the Security Policy Database selects traffic for protection, bypass, or discard. Traffic selectors include relevant packet-header fields. ESP/AH security associations are directional; ordinary two-way communication requires corresponding protection in both directions. Tunnel mode protects an inner packet carried between tunnel endpoints. RFC 4301
Now draw two address spaces. In our original example, application host 10.1.0.10 connects to 10.2.0.20:443. Gateway A and gateway B have separate outer addresses used to carry encrypted traffic. A route to B’s outer address does not establish a route to the remote application subnet. A firewall allowance for the outer connection does not establish an allowance for decrypted TCP traffic.
Write down the test flow before choosing commands:
- Inner source:
10.1.0.10, including its actual interface or network namespace. - Inner destination:
10.2.0.20, TCP destination port443. - Selected routing table or VRF, next hop, and tunnel/interface policy.
- Negotiated local and remote selectors, including protocol or port restrictions.
- Translation and filtering points before encryption and after decryption.
A gateway-generated ping can use a different source from a workload-generated TCP connection. It can therefore succeed through a selector that excludes the real workload. Conversely, a selector allowing TCP 443 need not allow ICMP. Explain why you choose a test whose source, destination, and protocol match the complaint.
For route-based products, a tunnel interface changes how forwarding is expressed; it does not make negotiated traffic restrictions irrelevant. Confirm the device’s configuration model before applying a policy-based checklist unchanged.
How do you locate “tunnel up, no traffic” with counters?
Vendor-specific fact: Cisco’s troubleshooting guide uses IPsec state, selectors, and encapsulation/decapsulation counters to inspect traffic protection. Its commands and examples belong to the documented Cisco platforms; they are not a universal CLI or a current cryptographic configuration recommendation. Cisco IPsec troubleshooting
The following ledger is an original controlled exercise. Assume an isolated test flow, established matching Child SAs, stable counter definitions, and no retransmissions. Send ten requests from the application host. Record before-and-after deltas for the same SAs at both gateways rather than reading a lifetime total.
| Test round | A encrypts | B decrypts | B encrypts replies | A decrypts replies | Next evidence to request |
|---|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 | Did packets reach A and select the intended protected path? |
| 2 | 10 | 10 | 0 | 0 | Did B forward the inner packets, and did the destination generate replies? |
| 3 | 10 | 10 | 10 | 0 | Where did B’s protected return packets disappear or get rejected? |
In round 1, inspect A’s inside-facing observation point, route lookup, selector match, and translation result. If the packet never reached A, the VPN cannot explain that absence. If it reached A but entered a different route, focus on forwarding. If routing is correct but the source became an address outside the protected selector, inspect the applicable NAT policy.
Round 2 narrows the search beyond the forward encryption boundary. Inspect B’s decrypted forwarding path and the destination host. Is TCP 443 listening? Does the host firewall accept the source? Does the host have a return route? If a reply exists on B’s inside interface but the reply-encryption counter remains unchanged, investigate B’s return route, selector, and policy instead of blaming the remote application.
Round 3 calls for observation on the outer return path and A’s inbound processing. Check whether packets arrive at A, whether they identify the expected SA, and whether rejection counters rise. The table identifies the next boundary; it does not uniquely identify a fault. A failed integrity check and a missing outer packet can produce the same zero successful-decryption delta.
Real environments also have background traffic, hardware offload, multiple SAs, retransmissions, and counters with different semantics. Establish which counters are authoritative, avoid resetting shared counters casually, and correlate a bounded time window with flow-specific observations. A plausible total from an unrelated tunnel is weaker evidence than a matching before-and-after delta.
Why is NAT traversal different from NAT exemption?
Official protocol fact: UDP encapsulation lets ESP traffic traverse NAT devices; RFC 3948 specifies the packet format, including distinguishing encapsulated ESP from IKE traffic sharing the UDP port. RFC 3948
In engineering terms, NAT traversal concerns delivery of protected packets across the outer network. A NAT exemption or identity-NAT rule concerns whether a device changes an inner flow before it selects or processes traffic protection. They solve different problems.
Consider the original flow from 10.1.0.10. Suppose a general outbound rule changes its source to an address not covered by the negotiated local selector. The peers can maintain a working control relationship while that application packet takes the wrong policy path. Enabling UDP encapsulation does not restore the original inner source.
Ask for the platform’s packet-processing order and an observation of the translated tuple. Do not claim NAT always runs before selector matching on every implementation. Likewise, do not assume that opening IKE traffic proves the outer ESP path works. Determine whether this tunnel actually uses native ESP or UDP encapsulation, then inspect that path and its return direction.
A concise answer is: “I would verify outer transport separately from inner-address translation. For the failed application flow, I need its tuple at the policy-selection boundary and the relevant counter deltas.” That specifies evidence without inventing a configuration change.
What if small probes work but large transfers stall?
Official standards fact: ESP adds headers, trailer fields, and cryptographic overhead. IPv4 Path MTU Discovery uses feedback about packets that cannot be forwarded with the Don’t Fragment constraint. Those facts make packet size and feedback delivery relevant, but they do not prescribe one fixed IPsec overhead for every algorithm and encapsulation. RFC 4303, RFC 1191
For an original follow-up, the interviewer says that a small application request succeeds but a larger response repeatedly stalls. Keep the same endpoints and service; vary size deliberately. Compare the largest successful payload, observed outer packet sizes, fragmentation behavior, retransmissions, and relevant ICMP feedback. State whether a tool reports payload size or total IP length.
An MTU hypothesis becomes stronger when the failure follows packet size and the captures or drop evidence agree. It remains a hypothesis until then: an application timeout, congestion, or a blocked response path can also interrupt a transfer. TCP MSS adjustment may help a TCP-specific issue when configured appropriately; it is not evidence that every UDP application or every path has been fixed.
Avoid prescribing “subtract 50 bytes” from memory. Explain which outer IP version, UDP encapsulation, ESP algorithm, padding, and path constraints you would measure. The useful interview skill is deriving the effective limit for the tested path.
How should you investigate intermittent failures around rekey?
Treat a failure that coincides with rekey as a timing hypothesis. Record the old and new SA identifiers, negotiation events, direction-specific counters, and the first failed application timestamp. Determine whether the issue affects both directions or only one. A change in status at roughly the same time is a lead, not proof of causation.
Ask whether the peer installed and used the new state, whether protected packets still identify old state, and whether failures continue after the transition. Compare a healthy rekey if evidence is available. Preserve logs before suggesting a restart, because clearing the tunnel can erase the state needed to distinguish the alternatives.
An interview answer should also address change scope. If a proposed selector correction affects several application subnets, define the intended flows and validate an unaffected flow as a control. Keep the change reversible and avoid widening a security policy just to make a probe succeed. That makes the diagnosis and the recovery plan reviewable.
How to give a strong two-minute answer
Name the business flow and clarify what “up” means. Check the intended IKE and Child-SA state. Follow that flow through its route, selectors, translation, and paired counter deltas. Choose the next observation from the last boundary with positive evidence. Then describe one narrow repair and the same-flow validation that would establish recovery.
For example: “A encrypted ten test packets and B decrypted ten, so I would next check decrypted forwarding and replies at B. If the host replied but B encrypted none, I would examine the return route and protection selection. I cannot infer a bad shared secret from this trace.” The answer is useful because each conclusion is limited to what was observed.