CSR Slow performance and slow log processing
Title:CSR Slow performance and errors in the logfiles
Category / Product: Contenty Security Reporter (CSR)
Description:
Content Security Reporter (CSR) 2.9.0 is a reporting software solution that helps you identify and analyze a broad range of data collected from your network devices. Content Security Reporter (CSR) features include essential reporting capabilities that allow you to report and analyze data. Similarly The performance optimization options ensure that CSR runs efficiently.
Issue:
Customers complaining Slow performance , Customer sees a lot of jobs in waiting state.There are logs getting accumulated and filling up CSR hard disk.
Troubleshooting:
1. Capture CSR feedback file
2. Report Server Settings > Job Queue > Cancel all the ongoing jobs
3. Report Server Settings > Directories > Disable all
4. Check Total size of installation drive and space left on the same
5. Check DB size of MariaDB ( if internal been used ) or check for external DB , see if there is High resource usage on database server
6. Partitioning database for improving reporting performance can be enabled
7. Task Manager > Details > Expand the tab > Verify the PID,CPU,Memory used by "csrss.exe"
8. Check processing jobs, set back to default 2
9. Open Task Manager > Performance > Open Resource Monitor > Expand to capture CPU & memory
10. Memory and processing jobs are configured under “Report Server Settings -> Performance Options”
Verify the feedback file and review the server_err file : We are observing Timeout Error.
2024-03-01 00:00:20,173 WARN [Thread: Detail Processor (csr_fct_exact_access)] [com.mcafee.mesa.reportdbin.logProcessing.detail.DetailProcessor] Waiting for detail persister to become available.
2024-03-01 00:00:20,224 ERROR [Thread: LogDataProcessor] [com.mcafee.mesa.reportdbin.logProcessing.LogDataProcessor] Timeout error occurred while adding to processor queue DetailProcessor
2024-03-01 00:00:20,237 ERROR [Thread: LogDataProcessor] [com.mcafee.mesa.reportdbin.logProcessing.LogDataProcessor] Timeout error occurred while adding to processor queue SummaryProcessor
Workaround:
Follow the above troubleshooting steps to review and verify the issue. Check the Performance Options under Report Server Settings.
1. View and configure the maximum amount of log processing jobs that can run simultaneously.
Skyhigh Security recommends using the default value of 2. This can be viewed under
Report Server Settings > From the Setting Categories menu, select Performance Options.
2. change the memory allocation from 1 GB to 6 or 8GB i.e. Go to Report Server Settings > Performance Options > edit > change it to 6GB or 8 GB > save it.
Note: you need to restart the CSR service.
3. In same Performance option you see summary cache > select web summary cache > Actions > Edit > Add one extra zero to existing max entries number > save it.
4. In Cache > select (Web) Site Information > Actions >edit > change max entries to 500000 > save it.
Note: step 3 & 4 doesn't require CSR service restart.
Verification:
Check if all the logs are processed, Monitor the log processing Data on the dashboard.