ISE 3.2 slowness

[ammahend]

Hi members, I am seeing following issues on ISE after upgrading to 3.2 patch 4 :

• sometimes the guest page won't load with "error loading page" message, sometimes it will works great.
• Lot of authentication request is going to secondary node which was not the case before, seems like lot of radius timeouts on primary
• There is a general sluggishness in GUI and CLI both

This is a 2 node deployment with 3715, we are closing around only 7-8K active sessions, Only additional feature enabled in addition to upgrade is log analysis but my understating was it should not be resource intensive. I am seeing 90% plus ram and 100% CPU spike from time to time almost every hour or so. I understand under 90% RAM is expected but CPU spike and 90% plus memory usage doesn't make much sense, as test I have disabled log analysis to monitor behavior since this was the only few feature addition, I also have opened a case open with Cisco. I wanted to pick your brain and see if anyone has any advice or input.

I will upgrade to patch 5 probably next weekend.

[Arne Bier]

Hi @ammahend 

I don't have an SNS to compare this to. I have one customer with a few SNS-3615 servers currently running ISE 3.1 that I was planning to upgrade to ISE 3.2 soon.

Have you logged into the CIMC to have a look around for any hardware events that might be causing slowness?  It's worth checking (although unlikely that a patch would affect the hardware).

Do you see any clues in the Dashboard Alarm panel?

I think you've done the right thing to get TAC involved. They should be able to point to the cause (checking the process table to see what's hogging the CPU).

[ammahend]

Hi Arne, No hardware events, I am waiting for Cisco to review the Support bundle, but we did make some changes after which we have not high load average

disabled log analysis, deactivated any unused probes, disabled Profiler Forwarder Persistence queue. I will post when I hear back from Cisco.

[alexhilton]

Hi - I have exactly the same problem on ISE 3.2 Patch 4. I have two Admin Nodes and 3 PSNs across multiple sites. A safe re-boot of the Admin Nodes does help it for a short time and then the Slowness returns. Getting Tacacs/Radius Logs takes an age. It was great initially as Patch 4 did resolve a lot of our issues. I was looking to upgrade to Patch 5 so let me know how you get on with it and I will probably attempt the same so long as it does not break anything else. 

[omehmetoglu]

I too am having the same issue, I upgraded from a perfectly working fine v3.1 and now am on 3.2 patch 5 and my admin nodes are getting slower in responding and taking a while. A safe reboot does fix it for a short period but then will eventually get slow again.

Will be raising a TAC case for this tomorrow. Does anyone else have an update on this at all?