TLS Handshake Failure in PCAP: ClientHello, ServerHello, Certificate, Alert, and Reset Evidence

TLS handshake failures are often reported as "SSL error," "certificate problem," "handshake failed," or "connection reset." Those messages are useful, but a PCAP can show where the handshake stopped. That location matters.

A TLS failure before ServerHello is different from a certificate validation alert. A TCP reset after ClientHello is different from a fatal TLS alert after certificate exchange. The packet capture timeline can identify the boundary.

Start with the TCP Connection

Before debugging TLS, confirm TCP:

• SYN
• SYN/ACK
• ACK
• data from client

If TCP never establishes, this is not a TLS handshake failure. It is routing, firewall, port, server reachability, or TCP policy.

If TCP establishes and the client sends ClientHello, TLS begins.

ClientHello Shows What the Client Offered

ClientHello can reveal:

• TLS versions supported
• cipher suites offered
• SNI hostname
• ALPN protocols
• supported groups
• signature algorithms

If SNI is missing, the server may return a default certificate or reject the handshake. If the client offers only old protocols or ciphers, the server may respond with handshake failure or reset.

This is why captures are useful for old embedded clients, proxies, and custom integrations.

ServerHello or No ServerHello

If the client sends ClientHello and no ServerHello ever arrives, inspect:

• firewall or middlebox drop
• server policy silently closing
• MTU/path issue around large handshake messages
• TCP reset from server
• load balancer behavior

If ServerHello arrives, inspect the chosen version and cipher. The server's choice may explain later failure.

Alerts Are Evidence, Not Noise

TLS alerts can be very informative:

• unknown CA
• bad certificate
• handshake failure
• protocol version
• illegal parameter
• close notify

A fatal alert from the client after certificate delivery often points to trust chain, hostname mismatch, expired certificate, or unsupported certificate properties. A fatal alert from the server after ClientHello may point to cipher, protocol, SNI, client certificate, or policy.

Do not discard alerts when trimming a capture.

Where PCAP Surgery Fits

PCAP Surgery should help engineers isolate the TLS conversation and preserve handshake evidence. A useful derived capture for TLS support includes:

• TCP handshake
• ClientHello
• ServerHello if present
• certificate messages if present
• TLS alerts
• TCP resets
• timing between messages

If the capture must be sanitized, be careful. Removing certificate details, SNI, or payload lengths may also remove the reason for the failure. The sanitization decision should match the troubleshooting goal.

For search queries like "TLS handshake failure pcap," "ClientHello no ServerHello," or "SSL alert unknown CA Wireshark," the answer is in the handshake boundary. A good packet surgery workflow preserves that boundary instead of hiding it.