Debugging ARP on Cisco ASA

broadcast

The packet capture wizard in ASDM is a great feature of the ASA platform. It allows a network administrator to easily debug an issue and export the capture right to Wireshark from the wizard.

However, as you use this you may notice something. Where are my arp packets? Any time Wireshark is ran from a layer-2 network, arp packets will inevitably be captured. Something I didn’t know is that the ASDM wizard does not capture broadcast packets (at least at the time this was written ASA version 9.4(2) and ASDM 7.6).

Unfortunately Cisco doesn’t really describe this in any of their capture documentation, so if you don’t typically capture through the command line, you’ll never see broadcasts and may wonder what’s wrong.

How can I capture arp broadcasts on my ASA for troubleshooting layer-2 issues?

You have to do this through the ASA command line.

  1. Log in to the ASA you want to capture/see ARP packets.
  2. Use the ‘capture’ command with the ethernet-type arp

An example would be:

ASA# capture arp-cap ethernet-type arp interface inside

Where arp-cap is the name of your capture, the ethernet-type filters the capture to only arp packets and the interface picks the interface where you want to see the broadcasts.

You can define a ‘buffer’ flag if you want, but don’t worry about overloading your ASA, the default is 512kb. The above command is typically what you want.

Now we can execute a show command to see the capture buffer:

ASA# sh cap arp

81 packets captured

1: 13:21:17.283554 arp who-has 192.168.10.1 (cc:3:ca:f8:34:50) tell 192.168.10.21
 2: 13:21:17.283630 arp reply 192.168.10.1 is-at cc:3:ca:f8:34:50
 3: 13:21:18.600005 arp who-has 10.4.49.190 tell 192.168.10.1
 4: 13:21:20.053692 arp who-has 192.168.10.1 (cc:3:ca:f8:34:50) tell 192.168.10.167
 5: 13:21:20.053784 arp reply 192.168.10.1 is-at cc:3:ca:f8:34:50
 6: 13:21:21.069271 arp who-has 10.4.49.182 tell 10.4.48.25
 7: 13:21:21.998391 arp who-has 10.4.49.182 tell 10.4.48.25

We now  see who is broadcasting for what, and what hardware address they reside on. Use the detail flag to see more information.

Clean Up

We’ve got what we need, so its time to clean up. It’s very simple:

ASA# no cap arp-cap
Advertisements
Debugging ARP on Cisco ASA

ACL Debugging on ASA with ASDM

I know–that title is a mouthful, but it’s actually pretty simple. A lot of environments out there have ACL’s that have implicit permits because it was simply too much work to get a list of what needs to talk to what before the firewall(s) went into place.

So, you’ve got an ACL applied to an interface that is set to permit all. The goal is to not permit everything and break out individual permit rules so we can change that to a deny rule. In ASDM it should look something like this.

1

So what goes through that ACL that I can start breaking out and writing specific rules for?

Right now if you right click the ACL in ASDM you get this nice little function called “Show Log…” The problem is this doesn’t work right out of the gate. If you click on this option you wont see anything yet.

2

The first thing I like to do is enable Debug logging on ASDM. That way when I open the log viewer or try to see hits on an ACL–I don’t have to modify anything.

Under Configuration –> Logging –> Logging Filters –> ASDM change “Filter on severity” to Debugging

The strange thing is now when you go back to the ACL’s and click “Show Log…” you still don’t see anything. Why? The problem is that permit statements will not log by default. This needs to be enabled either in ASDM or through the command line.

Command Line:

logging message 106100

ASDM:

Configuration > Device Management > Logging > Syslog Setup > 106100 “Disabled = No”

Now when you right click a rule and select “Show Log…” you will start seeing hits on the ACL. Monitor these hits for a while during work hours and you will at least get a huge chunk of the rules written. This is really nice for breaking out permit rules so you can eventually have an implicit Deny All there instead of a Permit All.

ACL Debugging on ASA with ASDM

ASA Appliance to Module Headaches

asa-smheadache

I recently migrated one of our environments from an ASA 55xx appliance to an ASA service module running in a 6509-e chassis. There were a few gotchas that I was not aware of that made the transition date quite an interesting experience… There were three pain-points for me on migration day. I’ll cover what they were and the configuration that was actually needed to make things work.

I use ASDM for most of my day-to-day work on ASA, so I’ll post the configuration with ASDM with the command line equivalent after.

NAT Exempt rules for VPN

I had a heck of a time finding a definitive document on the changes made on ASA NAT Exempt Rules for VPN tunnels between ASA version 8.2 and 8.3 (9.1 in my case). I tried to put whatever I could find on Cisco’s support site and on Google into my config prior to migration day, but of course what I had in there was wrong.

On ASA version 8.2 NAT exempt rules were simple; click the Add button under NAT Rules, followed by Add NAT Exempt Rule. Simple!

1

Enter your source interface, source host(s) and destination VPN network. Boom done!

2

If you were NAT’ing through the VPN tunnel you created a Static Policy NAT rule. Defined the source server, destination VPN network, and the translated address. A little more work, but still simple.

3

Then comes the new ASA version!! Ok I know ASA, let me go to my NAT Rules, Add… wait a minute…?

4

The new version doesn’t contain Policy NAT rules, NAT exempt rules or Dynamic NAT rules!! With a little Googling I couldn’t find any definitive answer as to how to create my NAT exempt rules for VPN. In both cases I exempt using the servers Real IP and I NAT to another IP over the tunnel.

Real IP NAT exempt

Here is how to do it through ASDM, I’ll post the command line results at the end.

  1. Click Add NAT Rule Before “Network Object” NAT Rules…
  2. 5
  3. Pick your inside interface where the server sits.
  4. Pick the interface (most likely outside) where the VPN traffic traverses through.
  5. Add a network object for Source Address; either subnet, range or host IP.
  6. Pick the destination IP/subnet that the VPN interested traffic would match.
  7. Check Disable Proxy ARP
  8. Leave the rest

6

A statement will show up above your network object NAT rules like this:

7

The command line equivalent would be this:

object network MY-RSERVER
 host 10.100.1.99
 object network VPN-NETWORK
 subnet 172.44.0.0 255.255.0.0
 nat (DEVPUB,OUTSIDE) 7 source static MY-RSERVER MY-RSERVER destination static VPN-NETWORK VPN-NETWORK no-proxy-arp

NAT Exempt with Translation

  1. Click Add NAT Rule Before “Network Object” NAT Rules…
  2. 5
  3. Pick your inside interface where the server sits.
  4. Pick the interface (most likely outside) where the VPN traffic traverses through.
  5. Add a network object for Source Address; either subnet, range or host IP.
  6. Pick the destination IP/subnet that the VPN interested traffic would match.
  7. To NAT the traffic, enter a network object with the translated IP in the Source Address field.
  8. Check Disable Proxy ARP

8

Once you enter the config, you will get a line in ASDM like this:

9

The resulting command line results will be the following.

object network MY-RSERVER
 host 10.100.1.99
object network MY-RSERVER_VPNNAT
 host 192.168.10.99
object network VPN-NETWORK
 subnet 172.44.0.0 255.255.0.0
nat (DEVPUB,OUTSIDE) 7 source static MY-RSERVER MY-RSERVER_VPNNAT destination static VPN-NETWORK VPN-NETWORK no-proxy-arp

Without these NAT statements your VPN traffic will not properly go over the tunnel.

Network object NAT Rule Changes

The second part that I was not used to was the way that NAT statements were configured for public servers. It was really confusing to me at first because there are so many options with the new version. Simply stated, this is how to do it.

  1. Go to the Network Objects pane
  2. 10
  3. Click Add
  4. Enter the name for your object and IP address
  5. Expand the NAT section and click Add Automatic Addresss Translation Rules
  6. Add a network object with the Public IP of the host11
  7. Expand the Advanced… section
  8. Check Disable Proxy ARP
  9. Pick the source interface the server sits on
  10. Pick the destination interface for your public NAT(probably OUTSIDE)12
  11. Click OK

Once the configuration is complete you will get something like this:

13

Command line equivalent:

object network MY-RSERVER
 host 10.100.1.99
object network MY-RSERVER_PUB
 host 4.2.2.1
object network MY-RSERVER
 nat (DEVPUB,OUTSIDE) static MY-RSERVER_PUB no-proxy-arp

OUTSIDE ACL Changes

The other issue that I ran into that was EXTREMELY frustrating was the ACL statements needed for the outside interface. In ASA version 8.2 it made sense: allow anyone, to my public IP, on the port I specify. I had all my ACL’s created on migration day, but nothing worked!! Why? Because on the new version your OUTSIDE ACL’s match against the real-IP of the server, not the NAT IP.

This is pretty self explanatory, once you figure it out. All you have to do is make sure your outside interface rules match against the REAL-IP of the server, not the public IP!

14

Command line equivalent:

access-list OUTSIDE_access_in line 1 extended permit tcp any object MY-RSERVER eq https
ASA Appliance to Module Headaches