ASA5525 CPU= 95%. "High probability of traffic loop on an interface."

[jmaxwellUSAF]

On an ASA 5525, it is true that same-security-permit intra-interface is configured on Inside interface.

The CPU is currently at 95%. The Cisco CLI Analyzer diagnostic tool states...

"High probability of traffic loop on an interface. This loop may account for up to 98.74 percent of overall traffic of this device.

The 1 minute output rate on "Inside" is 216,977 pkts/sec. This is 215,534 pkts/sec greater than the combined input rate on all other interfaces (which is only 1443 pkts/sec). As a result there is a significant probability that there is a traffic loop ocurring on "Inside" interface which may be causing elevated CPU usage or other network instability. This loop accounts for up to 98.74 percent of overall traffic of this device.

You can diagnose this issue further by checking the output of "show conn all". This output will show you all the current connections traversing this device. Typically these looped packets are entering and leaving the same interface, so filtering the "show conn" output for that interface can easily identify that looping connection. For example if the looping interface is "Inside" you can issue the command "show conn all | in Inside.*Inside"
The bytes counter in successive outputs will indicate how much traffic this connection has passed. You can also use the "capture" command to get a quick packet capture of the traffic.

Some common packet loop causes:

• A misconfigured route or NAT rule on ASA or adjacent routers can trigger this kind of loop if same-security-permit intra-interface is configured.
• If one of the IPs in the loop is from a VPN pool on ASA, it could be the issue described in this document
• The loop could also be due to to/from-the-box traffic. A packet capture can help you identify the traffic.

Inside:

received (in 321401.922 secs):

1473179127860 packets 209359639740234 bytes

4583002 pkts/sec 651395001 bytes/sec

transmitted (in 321401.922 secs):

1955821110067 packets 193563127184244 bytes

6085000 pkts/sec 602246006 bytes/sec

1 minute input rate 66039 pkts/sec, 4054586 bytes/sec

1 minute output rate 216977 pkts/sec, 32020726 bytes/sec

1 minute drop rate, 2 pkts/sec

5 minute input rate 66940 pkts/sec, 4425035 bytes/sec

5 minute output rate 216598 pkts/sec, 31981230 bytes/sec

5 minute drop rate, 3 pkts/sec

-----

OK then, see output below from show conn all | in Inside.*Inside

(obfuscated)

Hawthorne-FW/pri/act# show conn all | in Inside.*Inside
TCP Inside 10.15.50.14:21 Inside 172.16.8.9:12103, idle 0:00:00, bytes 0, flags SaAXB
TCP Inside 10.15.50.14:25 Inside 172.16.8.9:34252, idle 0:00:00, bytes 0, flags SaAXB
UDP Inside 10.15.10.93:137 Inside 172.16.8.9:57285, idle 0:00:00, bytes 8827350, flags X
TCP Inside 10.15.10.93:21 Inside 172.16.8.9:12103, idle 0:00:00, bytes 0, flags SaAXB
TCP Inside 10.15.10.93:25 Inside 172.16.8.9:34252, idle 0:00:00, bytes 0, flags SaAXB
UDP Inside 10.15.10.23:111 Inside 172.16.8.9:57285, idle 0:00:00, bytes 7046560, flags RX
TCP Inside 10.15.10.22:21 Inside 172.16.8.9:12103, idle 0:00:00, bytes 0, flags SaAXB
UDP Inside 10.15.10.16:161 Inside 172.16.8.9:34463, idle 0:00:00, bytes 7747300, flags X

UDP Inside 172.16.77.255:138 Inside 172.16.77.2:138, idle 0:00:26, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.3:138, idle 0:00:32, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.33:138, idle 0:00:40, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.6:138, idle 0:00:46, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.35:138, idle 0:01:22, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.49:138, idle 0:01:33, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.22:138, idle 0:01:38, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.21:138, idle 0:01:38, bytes 25527, flags X
UDP Inside 172.16.77.255:138 Inside 172.16.77.11:138, idle 0:01:49, bytes 25527, flags X

---

QUESTIONS:

1. "TCP Inside 10.15.50.14:21 Inside 172.16.8.9:12103, idle 0:00:00, bytes 0, flags SaAXB"... Does this mean "from interface Inside 10.15.50.14 to interface Inside 172.16.8.9? What means "flags SaAXB"?

2. Second block (172.16.77.x/24)  is Anyconnect subnet. Is this UDP broadcast traffic normal?

3. Does this data suggest evidence of symptom ASA Has High CPU Usage Due to a Traffic Loop When VPN Clients Disconnect - Cisco  ?

4. Can you suggest source of this 95% cpu usage? Is there any evidence here of routing loops? Can you suggest next troubleshoot steps or remediation?

Thank you!

[MHM Cisco World · Accepted solution]

UDP Inside 172.16.77.255:138 Inside 172.16.77.2:138, idle 0:00:26, bytes 25527, flags X

This udp conn is repeated many times !!

What is these IP and why same subnet connect via fw?

[jmaxwellUSAF]

I agree this is curious.

172.16.77.255/24 is broadcast address of Anyconnect subnet. 

Am I correct in believing that broadcast address should only be a destination, not a source?

Do you have theory as to what is going on here? Is this normal, or should there be a remediation?

[MHM Cisco World]

Show vpn sessiondb anyconnect 

Check if the vpn pool assign broadcast IP to one of anyconnect active user 

[jmaxwellUSAF]

I conclude VPN pool is NOT assigning 172.16.77.255 to any active users.

# sh run | b SSLVPN-SUBNET
ip local pool SSLVPN-SUBNET 172.16.77.1-172.16.77.254 mask 255.255.255.0

--

Also, I run #Show vpn sessiondb anyconnect. I see all users IP addresses. There is no 172.16.77.255 user address.

Then I check #show conn all | in Inside.*Inside, I see two 172.16.77.255 source addresses going to two active users.

Then I again run #Show vpn sessiondb anyconnect. I see same users. There is no 172.16.77.255 user address.