Debug SSL Handshake Failures (F5, *nix)

This article primarily applies to debugging SSL handshake failures on F5 LTM, but it can be used on any device with tcpdump. 

handshake

It can be tricky to truly understand who is affected when you change settings on your F5 SSL profiles. F5 has a handy little counter under the Statistics tab for your virtual-server, but it doesn’t tell you anything about who is failing.

sslshake1

They also log SSL handshake errors (01260009), but again, that doesn’t tell you who is failing.

Let’s say our security team asked us to change the F5’s ciphers, TLS or some other setting. Who did I break? Unless you have a way to talk to the customer, look at a DB for less data, etc. this can be tricky.

TCPDUMP and Wireshark can give us some insight into this, with the right capture.

The foundation for this was a response found here. However, I wanted to take a look at only the handshake failures in Wireshark to get an idea of the customer IP’s that are affected.

If I run a basic capture on the interface where SSL traffic terminates, I can see messages like this:

sslshake2

The actual content we are looking for always starts with 0x15 in hex.

sslshake3

Using the foundation article above, we can craft a tcpdump command to look for these messages.

tcpdump -ni public -C 100 -W 5 -w /var/tmp/ssl_traffic.pcap "port 443 and (tcp[((tcp[12] & 0xf0) >> 2)] = 0x15)"

This command will create 5 100MB files that will cyclically rotate and overwrite each other for you to analyze. They will mostly contain only the handshake failure messages we are looking for.

Filter Only Handshake Failure Packets

If you want to view statistics only for the ‘Handshake Failures’, take a look at the highlighted hex above. We can apply that as a filter so we only see those packets, and view the statistics on those (described below).

Use the following filter to view only the Handshake Failure packets.

frame contains 15:03:01:00:02:02:28

Now the IP’s that are failing to establish an SSL handshake can be analyzed. In Wireshark, using the Statistics tab, click Endpoints. Sort by Packets to see who the top offenders are. This can be used by others to determine if they are legit or not.

sslshake4

 

Extra

Maybe you want to see what ciphers/protocol the client proposed before they failed to analyze further?

Well–add an or statement to our tcpdump statement, you will see both that info. Expand the Client Hello in wireshark, and check what they are proposing. Perhaps that will help you to determine what ciphers you minimally need.

tcpdump -ni public -w /var/tmp/ssl_traffic.pcap "port 443 and ((tcp[((tcp[12] & 0xf0) >> 2)] = 0x15) or (tcp[((tcp[12] & 0xf0) >> 2)] = 0x16))"
Advertisements
Debug SSL Handshake Failures (F5, *nix)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s