Buy or Renew
Buy or Renew
Duo Security forums now LIVE! Get answers to all your Duo Security questions. Learn more
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
592
Views
15
Helpful
14
Replies

Logging and Monitoring Dashboard - Troubleshooting

bryan.hepding1
Level 1
Level 1

‎02-02-2016 04:04 AM - edited ‎03-10-2019 11:26 PM

We've recently upgraded to ACS 5.8.0.32 with two ACS servers. One is the primary and the secondary is for log collecting. When we click on troubleshooting we are unable to see any live authentications. I've tried to stop and start the logprocessor as well as doing a replace-cleandb command which erased all logs. Any help would be greatly appreciated.

Thanks,

Bryan

Labels:
0 Helpful
14 Replies 14

Jatin Katyal
Cisco Employee
Cisco Employee

‎02-02-2016 06:48 AM

From the primary please get the output of this command: sh acs-logs  filename acsLogForward.log  | last 80

From the secondary ( log collector) - get me the o/p of acsview show-dbsize

~ Jatin

~Jatin

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-02-2016 06:56 AM

Thanks for your quick reply, here is the output.


--- Primary ---


Daemon,24/01/2016,15:57:22:%Q,INFO ,0x7ffef6630760,Daemon ctor: process ID=4781,Daemon.cpp:22
Daemon,24/01/2016,15:57:22:%Q,INFO ,0x7ffef6630760,Daemon:init: daemon process ID=4784,Daemon.cpp:98
ConfigManager,24/01/2016,15:57:22:%Q,INFO ,0x7ffef6630760,ConfigManager init success,ConfigManager.cpp:110
ConfigManager,24/01/2016,15:57:24:%Q,INFO ,0x7ffef6630760,ConfigManager::createVersion version id 1,ConfigManager.cpp:917
ConfigManager,24/01/2016,15:57:24:%Q,INFO ,0x7ffef6630760,ConfigManager::activateVersion version id 1 activated,ConfigManager.cpp:944
Daemon,24/01/2016,15:57:24:%Q,INFO ,0x7ffef6630760,Log forwarding is suspended,LogForwardDaemon.cpp:308
HttpWorkerThread,24/01/2016,15:57:24:%Q,INFO ,0x7ffef6630760,HttpWorkerThread started,HttpWorkerThread.cpp:35
HttpHandler,24/01/2016,15:57:24:%Q,INFO ,0x7ffee3770700,HttpHandler::onHttpStartEvent: opened HTTP port 8087; RT Control is available,HttpHandler.cpp:94
Tailor,24/01/2016,15:57:24:%Q,WARN ,0x7ffef6630760,cannot open watermark file /opt/CSCOacs/runtime/config/logforward/watermark: No such file or directory,Tailor.cpp:4
38
Daemon,24/01/2016,15:57:24:%Q,ERROR,0x7ffef6630760,Daemon::createInitFile: failed to create init file. Error: Permission denied,Daemon.cpp:119
Daemon,24/01/2016,15:57:24:%Q,INFO ,0x7ffef6630760,initialized,LogForwardDaemon.cpp:111
Daemon,24/01/2016,15:58:44:%Q,INFO ,0x7ffef6630760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,24/01/2016,15:58:44:%Q,INFO ,0x7ffef6630760,Log forwarding is active,LogForwardDaemon.cpp:312
Daemon,24/01/2016,16:16:18:%Q,INFO ,0x7ffef6630760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,25/01/2016,08:26:37:%Q,INFO ,0x7f29f14c6760,Daemon ctor: process ID=2709,Daemon.cpp:22
Daemon,25/01/2016,08:26:37:%Q,INFO ,0x7f29f14c6760,Daemon:init: daemon process ID=2722,Daemon.cpp:98
ConfigManager,25/01/2016,08:26:37:%Q,INFO ,0x7f29f14c6760,ConfigManager init success,ConfigManager.cpp:110
ConfigManager,25/01/2016,08:26:39:%Q,INFO ,0x7f29f14c6760,ConfigManager::createVersion version id 1,ConfigManager.cpp:917
ConfigManager,25/01/2016,08:26:39:%Q,INFO ,0x7f29f14c6760,ConfigManager::activateVersion version id 1 activated,ConfigManager.cpp:944
Daemon,25/01/2016,08:26:39:%Q,INFO ,0x7f29f14c6760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,25/01/2016,08:26:39:%Q,INFO ,0x7f29f14c6760,Log forwarding is active,LogForwardDaemon.cpp:312
HttpWorkerThread,25/01/2016,08:26:39:%Q,INFO ,0x7f29f14c6760,HttpWorkerThread started,HttpWorkerThread.cpp:35
HttpHandler,25/01/2016,08:26:39:%Q,INFO ,0x7f29de602700,HttpHandler::onHttpStartEvent: opened HTTP port 8087; RT Control is available,HttpHandler.cpp:94
Daemon,25/01/2016,08:26:39:%Q,ERROR,0x7f29f14c6760,Daemon::createInitFile: failed to create init file. Error: Permission denied,Daemon.cpp:119
Daemon,25/01/2016,08:26:39:%Q,INFO ,0x7f29f14c6760,initialized,LogForwardDaemon.cpp:111
Daemon,25/01/2016,08:28:56:%Q,INFO ,0x7ff40d978760,Daemon ctor: process ID=2724,Daemon.cpp:22
Daemon,25/01/2016,08:28:56:%Q,INFO ,0x7ff40d978760,Daemon:init: daemon process ID=2736,Daemon.cpp:98
ConfigManager,25/01/2016,08:28:56:%Q,INFO ,0x7ff40d978760,ConfigManager init success,ConfigManager.cpp:110
ConfigManager,25/01/2016,08:28:58:%Q,INFO ,0x7ff40d978760,ConfigManager::createVersion version id 1,ConfigManager.cpp:917
ConfigManager,25/01/2016,08:28:58:%Q,INFO ,0x7ff40d978760,ConfigManager::activateVersion version id 1 activated,ConfigManager.cpp:944
Daemon,25/01/2016,08:28:58:%Q,INFO ,0x7ff40d978760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,25/01/2016,08:28:58:%Q,INFO ,0x7ff40d978760,Log forwarding is active,LogForwardDaemon.cpp:312
HttpWorkerThread,25/01/2016,08:28:58:%Q,INFO ,0x7ff40d978760,HttpWorkerThread started,HttpWorkerThread.cpp:35
HttpHandler,25/01/2016,08:28:58:%Q,INFO ,0x7ff3fabb8700,HttpHandler::onHttpStartEvent: opened HTTP port 8087; RT Control is available,HttpHandler.cpp:94
Daemon,25/01/2016,08:28:58:%Q,ERROR,0x7ff40d978760,Daemon::createInitFile: failed to create init file. Error: Permission denied,Daemon.cpp:119
Daemon,25/01/2016,08:28:58:%Q,INFO ,0x7ff40d978760,initialized,LogForwardDaemon.cpp:111
Daemon,25/01/2016,08:33:17:%Q,INFO ,0x7fca89ed8760,Daemon ctor: process ID=2658,Daemon.cpp:22
Daemon,25/01/2016,08:33:17:%Q,INFO ,0x7fca89ed8760,Daemon:init: daemon process ID=2661,Daemon.cpp:98
ConfigManager,25/01/2016,08:33:17:%Q,INFO ,0x7fca89ed8760,ConfigManager init success,ConfigManager.cpp:110
ConfigManager,25/01/2016,08:33:19:%Q,INFO ,0x7fca89ed8760,ConfigManager::createVersion version id 1,ConfigManager.cpp:917
ConfigManager,25/01/2016,08:33:19:%Q,INFO ,0x7fca89ed8760,ConfigManager::activateVersion version id 1 activated,ConfigManager.cpp:944
Daemon,25/01/2016,08:33:19:%Q,INFO ,0x7fca89ed8760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,25/01/2016,08:33:19:%Q,INFO ,0x7fca89ed8760,Log forwarding is active,LogForwardDaemon.cpp:312
HttpWorkerThread,25/01/2016,08:33:19:%Q,INFO ,0x7fca89ed8760,HttpWorkerThread started,HttpWorkerThread.cpp:35
HttpHandler,25/01/2016,08:33:19:%Q,INFO ,0x7fca5301d700,HttpHandler::onHttpStartEvent: opened HTTP port 8087; RT Control is available,HttpHandler.cpp:94
Daemon,25/01/2016,08:33:19:%Q,ERROR,0x7fca89ed8760,Daemon::createInitFile: failed to create init file. Error: Permission denied,Daemon.cpp:119
Daemon,25/01/2016,08:33:19:%Q,INFO ,0x7fca89ed8760,initialized,LogForwardDaemon.cpp:111
Daemon,25/01/2016,08:46:06:%Q,INFO ,0x7f0426f27760,Daemon ctor: process ID=2811,Daemon.cpp:22
Daemon,25/01/2016,08:46:06:%Q,INFO ,0x7f0426f27760,Daemon:init: daemon process ID=2825,Daemon.cpp:98
ConfigManager,25/01/2016,08:46:06:%Q,INFO ,0x7f0426f27760,ConfigManager init success,ConfigManager.cpp:110
ConfigManager,25/01/2016,08:46:08:%Q,INFO ,0x7f0426f27760,ConfigManager::createVersion version id 1,ConfigManager.cpp:917
ConfigManager,25/01/2016,08:46:08:%Q,INFO ,0x7f0426f27760,ConfigManager::activateVersion version id 1 activated,ConfigManager.cpp:944
Daemon,25/01/2016,08:46:08:%Q,INFO ,0x7f0426f27760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,25/01/2016,08:46:08:%Q,INFO ,0x7f0426f27760,Log forwarding is active,LogForwardDaemon.cpp:312
HttpWorkerThread,25/01/2016,08:46:08:%Q,INFO ,0x7f0426f27760,HttpWorkerThread started,HttpWorkerThread.cpp:35
HttpHandler,25/01/2016,08:46:08:%Q,INFO ,0x7f0418285700,HttpHandler::onHttpStartEvent: opened HTTP port 8087; RT Control is available,HttpHandler.cpp:94
Daemon,25/01/2016,08:46:08:%Q,ERROR,0x7f0426f27760,Daemon::createInitFile: failed to create init file. Error: Permission denied,Daemon.cpp:119
Daemon,25/01/2016,08:46:08:%Q,INFO ,0x7f0426f27760,initialized,LogForwardDaemon.cpp:111
Daemon,01/02/2016,13:59:13:%Q,INFO ,0x7f0426f27760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,01/02/2016,14:25:39:%Q,INFO ,0x7f0426f27760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,01/02/2016,14:48:36:%Q,INFO ,0x7fc15fc82760,Daemon ctor: process ID=2740,Daemon.cpp:22
Daemon,01/02/2016,14:48:36:%Q,INFO ,0x7fc15fc82760,Daemon:init: daemon process ID=2743,Daemon.cpp:98
ConfigManager,01/02/2016,14:48:36:%Q,INFO ,0x7fc15fc82760,ConfigManager init success,ConfigManager.cpp:110
ConfigManager,01/02/2016,14:48:38:%Q,INFO ,0x7fc15fc82760,ConfigManager::createVersion version id 1,ConfigManager.cpp:917
ConfigManager,01/02/2016,14:48:38:%Q,INFO ,0x7fc15fc82760,ConfigManager::activateVersion version id 1 activated,ConfigManager.cpp:944
Daemon,01/02/2016,14:48:38:%Q,INFO ,0x7fc15fc82760,Configured syslog: *.*.*.*:20514,LogForwardDaemon.cpp:380
Daemon,01/02/2016,14:48:38:%Q,INFO ,0x7fc15fc82760,Log forwarding is active,LogForwardDaemon.cpp:312
HttpWorkerThread,01/02/2016,14:48:38:%Q,INFO ,0x7fc15fc82760,HttpWorkerThread started,HttpWorkerThread.cpp:35
HttpHandler,01/02/2016,14:48:38:%Q,INFO ,0x7fc150dde700,HttpHandler::onHttpStartEvent: opened HTTP port 8087; RT Control is available,HttpHandler.cpp:94
Daemon,01/02/2016,14:48:38:%Q,ERROR,0x7fc15fc82760,Daemon::createInitFile: failed to create init file. Error: Permission denied,Daemon.cpp:119
Daemon,01/02/2016,14:48:38:%Q,INFO ,0x7fc15fc82760,initialized,LogForwardDaemon.cpp:111


--- Secondary ---


     Actual DB Size (bytes) : 69181440
     Actual DB Size (GBs) :0.06
     Physical DB Size (bytes):69230592
     Physical DB Size (GBs) :0.06
     Physical ACSviewlog file Size (GBs) :0

0 Helpful

Jatin Katyal
Cisco Employee
Cisco Employee
In response to bryan.hepding1

‎02-02-2016 07:07 AM

1. I could see an error log forwarding is suspended and then active again. I would still suggest you to check the current status of log forwarding on primary ACS under the log collector configuration.

Daemon,24/01/2016,15:57:24:%Q,INFO ,0x7ffef6630760,Log forwarding is suspended,LogForwardDaemon.cpp:308

Daemon,24/01/2016,15:58:44:%Q,INFO ,0x7ffef6630760,Log forwarding is active

2. Do you see logs in live authentication of primary ACS node if we switch the log collector from secondary to primary ACS? If you see then the next step is to ensure UDP 20514 is open between both the ACS.

~ Jatin

~Jatin

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-02-2016 07:27 AM

I moved the log collector to point to the primary and nothing is showing up still. Should I still look to see if the UDP port 20514 is being blocked?

0 Helpful

Jatin Katyal
Cisco Employee
Cisco Employee
In response to bryan.hepding1

‎02-02-2016 09:55 AM

If the primary is also showing the same behavior you don't need to check the port 20514. What do you see monitoring and reports > ACS reports > AAA protocol.

~ Jatin

~Jatin

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-03-2016 12:41 AM

AAA Diagnostics - Invalid or unexpected EAP payload received. Code - 11500

RADIUS Accounting - No data

RADIUS Authentication - No data

TACACS Accounting - Able to see users logging into switches.

TACACS Authentication - No data

TACACS Authorization - No data

0 Helpful

Jatin Katyal
Cisco Employee
Cisco Employee
In response to bryan.hepding1

‎02-03-2016 07:02 AM

It seems few logging components are showing logs and other shows nothing. Normally this happens due to issue with acs-viewdb and you've already replaced it. Can you reload the primary ACS and also check opt size again.

~ Jatin

~Jatin

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-03-2016 07:09 AM

So you would like me to restart the primary ACS? I did this yesterday and it fixed nothing. I can try again though if you recommend. If this is the case I need to wait for people to go home, so that I do not disrupt their work.

0 Helpful

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-03-2016 10:28 AM

Still not working.

     acsview show-dbsize
     Actual DB Size (bytes) : 118243328
     Actual DB Size (GBs) :0.11
     Physical DB Size (bytes):118259712
     Physical DB Size (GBs) :0.11
     Physical ACSviewlog file Size (GBs) :0.02

0 Helpful

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-05-2016 03:04 AM

Jatin, did you give up on me ? lol

0 Helpful

Jatin Katyal
Cisco Employee
Cisco Employee
In response to bryan.hepding1

‎02-05-2016 06:45 AM

No way - Just too busy. What is your email address?

~ Jatin

~Jatin
0 Helpful

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-10-2016 11:18 PM

Any updates? We are still experiencing this issue, Jatin.

0 Helpful

bryan.hepding1
Level 1
Level 1
In response to Jatin Katyal

‎02-03-2016 10:05 PM

Can you assist me with anything else to solve this problem?

0 Helpful
Customers Also Viewed These Support Documents