Summary:
In most cases, the default logging levels set in CloudCenter should be sufficient to understand what is happening during the startup and transaction processing of the CCO and CCM appliances.
There are some scenarios where additional information is desired or required in order to successfully troubleshoot an issue. Depending on what is being troubleshot, you may need to look at the output from the Orchestrator (CCO) or the Manager (CCM), or both.
This article includes the areas and instruction where enhanced logging can be enabled for the osmosix.log file, as well as examples of which appliance you will want to look at for the most relevant log detail.
CloudCenter version 4.7 brings additional ways to more easily view the log files, without having to enter the command line of any given appliance. Additionally version 4.8 enhances this by giving you logging detail from the worker itself, which greatly improves the ability to troubleshoot. Information about how to set up an ELK server that 4.7 and above leverages is not covered in this article. Please reference the following URL for more information about ELK server logging: Download Log File - Version 4.6 and 4.7 - CloudCenter Docs
Things to be aware of:
Increasing the level of logging can negatively impact the performance of the machine where this has been enabled. The impact usually isn't significant enough to warrant an after hours change window, but still is not recommended to maintain as set in production for long periods of time.
As with any best practice, ensure you have a back up of any files you will be editing, in case a revert to original is required.
How To:
To enable enhanced logging of the osmosix.log file do the following:
- Log into the console of the desired appliance (CCM, CCO) either through SSH or direct.
- Switch to root: "sudo -i" <-- without quotes
- Edit the log4j.xml file with this command:
- "vi /usr/local/tomcat/webapps/ROOT/WEB-INF/classes/log4j.xml" <-- without quotes
- Press the "i" key to enter edit (insert) mode.
- Scroll down to locate the section called <!-- Application Loggers -->
Under here are a number of different areas that can be adjusted.
Each logger type has a 'logger name' and 'level value'.
- Change the "info" field to "debug" for the logger you wish to enhance.
In most scenarios, the loggers 'com.osmosix' and 'com.cliqr' are of most interest. The other loggers can be increased as well, but they could introduce too much information, confusing the debugging process.
Which Appliance and Which Logger for Which Scenario?
Not all logging will need to be enabled in all appliances to fit your needs. When determining if you need enhanced logging, you first need to look at the problem you are trying to troubleshoot. Some scenarios and the choice are more obvious than others. Below are a few examples of where you want to look for the log information:
Are you trying to troubleshoot an issue where....
...you see a problem while inside of the UI of the CCM?
- Look at the CCM logs
...you have an API that is sending calls to CloudCenter, such as from an external source like Jenkins or Service Now?
- Look at the CCM logs
...you submit a job that contains an External Service that you are having troubles with?
- Look at the CCO logs of the Cloud where you submit the job
...the application isn't deploying on the destination cloud, or errs out?
- Look at the CCO logs in most cases
...your Orchestrator isn't registering with the Manager?
- You may need to look at both the CCM and related CCO. (assuming the CCO is already registered and connected to the AMQP server)
Loggers
<logger name="com.osmosix">
Enable debug in most scenarios that require enhanced logging.
<logger name="com.cliqr">
Enable debug in most scenarios that require enhanced logging.
<logger name="com.osmosix.commons.communication">
Enable usually when troubleshooting communications related issues. This produces a lot of extra logging, so be aware.
