Skip to main content

Check out Interactive Visual Stories to gain hands-on experience with the SSE product features. Click here.

Skyhigh Security

Automated Diagnostic Data Collection

Overview

The mwgResInfo is a command-line tool to monitor resource consumption, track connection health, and automate actions, such as troubleshooting data collection or service restarts when specific thresholds require action in a Skyhigh Secure Web Gateway (SWG) environment.

The tool is available as a script in the path below : /opt/mwg/bin/mwgResInfo.sh.

Key Benefits

  • Automated Troubleshooting. Configure intelligent triggers to execute diagnostic tasks (such as generating log bundles or core dumps) the moment an issue occurs.
  • Active Monitoring. Continuously monitor critical health metrics including memory utilization, CPU load, thread counts, and number of connections. 

Note: The tool is for Advanced Troubleshooting only and should not be continuously run for a prolonged period of time until the impact of it is carefully reviewed. Running this tool may impact SWG running processes, data processing, and resource availability depending on the data being collected. 
Before scheduling trigger commands, please review your system conditions and resource limits:

  • Disk Space: Ensure sufficient disk space is available before running commands like a core dump collection.
  • System Downtime: Commands such as feedback collection and core collection will pause the SWG process until the collection is complete. Please factor this into your scheduling. 

 

 

 

 

 

 

Usage Syntax

mwgResInfo.sh [-i seconds] [-s seconds] [[-c number -l number -AND|-OR -c number -l number...] [-p number] -e command [-ot]
[-n]] [[-gl] [-gi]] [-noLog]

Task Reference

Use this table to identify the flags needed for specific monitoring goals.

Task Goal Flags Description
Set Monitoring Timing -i, -s
  • -i:  Specifies the logging interval in seconds and determines how frequently the script checks and records system metrics. If you omit this flag, the script automatically applies a default interval of 180 seconds.  
    Default Value: 180 seconds
     
  • -s: Specifies the script execution duration in seconds. The script terminates automatically once it reaches the configured runtime. If you omit this flag, the script runs continuously until you manually stop it. 
    Default Value: Runs forever

Specify Target Metrics -c, -l, -p
  • -c: Specifies the system metric that the script monitors. Provide an integer value from 1 through 7 to select your target metric.

    For details about value range, see Condition (-c) Reference.
    Default value: N/A
     
  • -l: Defines the threshold for the monitored metric (specified in -c option).

    For example, -c 6 -l 90 triggers the configured action when CPU usage exceeds 90%.
    Default value: N/A 
     
  • -p: Specifies the target application or process for the script to monitor against your configured performance thresholds (-c and -l). Provide an integer value from 0 through 6 to select the process. When the monitored process exceeds your defined threshold, the script executes the configured trigger action.
    Default Value: 0 (All applications)

    For details about values, see Process (-p) Reference.

    For example: -p 1 -c 6 -l 90, triggers the configured action if the mwg-core process hits 90% CPU usage. 
Create Complex Triggers -AND, -OR

Chains multiple -c and -l pairs to create complex trigger conditions.

Note: Mixing -AND and -OR in the same command execution is not supported.

Example for  AND condition:

mwgResInfo.sh -p 1 -c 5 -l 8.0 -AND -c 4 -l 1000 -AND -c 6 -l 75 \ -e "gcore -o /opt/mwg/core_dump \$(pgrep -n mwg-core)" -ot -noLog 

ALL specified conditions must be true simultaneously to trigger the action. 

Example for OR condition:

mwgResInfo.sh -p 1 -c 2 -l 5000000 -OR -c 3 -l 5000 \ -e "/etc/init.d/mwg restart" -noLog 

If any of the above specified condition is true, an action will be triggered.

Automate Actions -e, -ot, -n
  • -e: Specifies the command or action that the script runs when a monitored metric exceeds its threshold. When system metrics meet the conditions defined by the -c and -l flags, the script automatically executes this specified command.
    Default Value: N/A

    For example:
    -e "/etc/init.d/mwg  restart" automatically restarts the service when the trigger condition is reached. Multiple commands can be specified using this option.

    For example:
    -e "echo "Capture for reaching threshold" | /opt/Skyhigh/ldt/getlogs/getlogs.sh /opt/mwg/debug ; gcore -o /opt/mwg/log/debug/mwg_core_dump $(pgrep -n mwg-core); /etc/init.d/mwg restart"

     

  • -ot: Executes the command specified by the -e flag only once when system metrics first meet your threshold condition. This restriction prevents the script from repeatedly triggering the action while the monitored metric remains above your configured threshold.
    Default Value: N/A
     

  • -n: Terminates script execution after system metrics meet your threshold condition for the first time. Upon reaching the threshold, the script executes the command specified by the -e flag (if defined) and exits immediately. If you omit this flag, the script continues monitoring in the background after a trigger event occurs.
    Default Value:
    N/A 

Manage Logs and Reports -noLog, -gl, -gi
  • -noLog: Disables the creation of metric log files. By default, the script continuously records monitored statistics (such as CPU and memory utilization) to local disk log files (for example,mwgResInfo*.log). 

    Use this option to operate the script as a lightweight background watchdog that silently monitors the system and executes your configured -e action the moment system metrics breach your defined thresholds.
    Default Value: N/A
     

  • -gl: Enables automatic, periodic collection of system logs by executing the getlogs utility at regular intervals. The script saves the resulting compressed ZIP archive to the local debug directory (/opt/mwg/bin/../log/debug/getlogs_<timestamp>.zip). If the host system lacks the getlogs utility, the script automatically attempts to install it. This function operates completely independently of any monitoring thresholds (-c or -l) or trigger conditions. 
    Default Value: N/A
     

  • -gi: Specifies the collection interval in seconds for the getlogs utility enabled via the -gl flag. If you deploy the -gl flag but do not specify the -gi parameter, the script automatically applies a default execution interval of 86,400 seconds (24 hours). 

    Example:
    To instruct the script to collect logs every hour, append -gl -gi 3600 to your command string. Default Value: 86400 seconds (24 hours).

Condition (-c) Reference

Use these values with the -c flag to define which metric triggers the command specified by -e.

Value Monitored Metric (-c)
1 Virtual memory usage(Kilobytes)
2 Physical memory usage(Kilobytes)
3 Number of threads
4 Sockets in CLOSE_WAIT state
5 Machine load average(Float/Integer)
6 CPU usage(in percentage)
7 Working queue length

Process (-p) Reference

Use these values with the -p flag to specify which application is evaluated against -c conditions.

Note: Background logging captures all processes regardless of this setting.

Value Supervised Process
0 All applications (Default)
1 mwg-core
2 mwg-coordinator
4 mwg-saas-connector
5 HSM agent
6 gamserver

 Logging

By default, the script logs the following statistics for all Skyhigh Secure Web Gateway applications every 180 seconds to the file ../opt/mwg/log/debug/mwgResInfo*.log in the following sequence:

  • Virtual Memory Consumption: Total amount of memory address space an application has requested from the operating system, including both physical RAM and memory temporarily swapped to disk. 
  • Physical Memory Consumption: Amount of actual, physical RAM currently assigned to and actively used by the specific application being monitored (for example, mwg-core).
  • Total Physical RAM in use by all processes: Total amount of physical memory currently being used by everything running on the system. This includes the operating system, all gateway services, and any other background tasks. 
  • Number of threads: Displays the total count of concurrent execution threads currently spawned by a specific process.
  • Machine Load Average: Displays the system's computational workload over a period of time, representing the average number of processes actively using or waiting for CPU time.
  • Number of connections in CLOSE_WAIT state: Tracks the network connections that the remote side has closed, but the local application has not yet finished shutting down.
  • CPU percent of each service: Measures the percentage of processing power actively being consumed by an individual process, normalized per CPU core.
  • Working queue length (TCP accepts queue depth on MWG listening ports—parameter applicable only for mwg-core service): This represents the depth of the TCP accept queue on the gateway's listening ports (applicable only specifically to the mwg-core process). A growing queue length indicates that incoming network connections are arriving and piling up faster than the gateway can actively process them.

Use Cases and Examples 

Monitor Memory Leaks

Capture logs and core files when virtual memory exceeds 16GB for mwg-core and automatically restart the service.

Command.
For 13.x onwards:

mwgResInfo.sh -p 1 -c 1 -l 16000000 -e "echo "Capture for reaching threshold" | /opt/Skyhigh/ldt/getlogs/getlogs.sh /opt/mwg/debug ; gcore -o /opt/mwg/log/debug/mwg_core_dump $(pgrep -n mwg-core); /etc/init.d/mwg restart" -n -noLog

The following list describes the commands used in the above example:

  1. Trigger Condition:
    • -p 1: Instructs the script to specifically monitor the mwg-core process.
    • -c 1 -l 16000000: Sets the trigger condition to Virtual Memory usage (-c 1) and the threshold to 16 GB (-l 16000000 KB). 
  2. Executed actions:
    • If the mwg-core process's virtual memory hits or exceeds 16 GB, it will sequentially execute three wrapped commands to capture debug data before restarting the service:
      • /opt/Skyhigh/ldt/getlogs/getlogs.sh /opt/mwg/debug: Runs a system log collection script and saves the diagnostic logging output to the specified debug directory.
      • gcore -o /opt/mwg/log/debug/mwg_core_dump $(pgrep -n mwg-core)Generates a core memory dump (gcore) of the currently running mwg-core process and saves it to your debug log directory. This is crucial for analyzing memory leaks or crashes.
      • /etc/init.d/mwg restartRestarts the Secure Web Gateway service to clear the high memory condition and restore normal operations.
  3. Script Behavior Modifiers:
    • -n (Run Until): Ensures that as soon as this 16 GB threshold is breached and the sequence of commands finishes executing, the script will completely terminate. It will not stay alive to monitor the system anymore.
    • -noLog: Prevents the script from writing its own continuous statistics text files (mwgResInfo*.log) to the hard drive while it is waiting for the trigger, keeping disk activity silent until the event actually happens.

For 12.2.x:

mwgResInfo.sh -p 1 -c 1 -l 16000000 -e "echo "Capture for reaching threshold" | /opt/McAfee/ldt/getlogs/getlogs.sh  /opt/mwg/debug ; gcore -o /opt/mwg/log/debug/mwg_core_dump $(pgrep -n mwg-core); /etc/init.d/mwg restart" -n -noLog

Trigger feedback generation only when CPU usage is over 95% and working queue depth exceeds 50.

Command.

mwgResInfo.sh -p 1 -c 6 -l 95 -AND -c 7 -l 50 -e "./feedback.sh" -ot

Schedule Diagnostic Captures

Enable automated GetLogs capture for 48 hours while running standard log metrics every 5 minutes, terminating after 7 days.

Command.

mwgResInfo.sh -i 300 -gl -gi 172800 -s 604800

The following list describes the commands used in the above example: 

  • -i 300: Changes the script's standard metric checking interval. Instead of the default 180 seconds, it is changed to 300 seconds. 
  • -gl: Turns on the automated GetLogs feature to periodically capture full system diagnostic logs. 
  • -gi 172800: Customizes the GetLogs frequency. It will gather and zip up the system logs every 48 hours (172,800 seconds) and save them to the debug directory. 
  • -s 604800: Sets a hard stop for the script. It will run continuously in the background for exactly 7 days (604,800 seconds) and then automatically terminate. 

Restart Service by Thread Count

Restart the SaaS connector service automatically if the number of threads exceeds 500.

Command.

mwgResInfo.sh -i 60 -p 4 -c 3 -l 500 -e "echo 'Saas-connector has hit 500+ threads. Will restart service ';/etc/init.d/mwg restart " -o

The following list describes the commands used in the above example: 

  • -i 60: Changes the script's standard metric checking interval to 60 seconds. 
  • -p4: Monitors only mwg-saas-connector
  • -c3 -l500: Triggers -e if number of threads is greater than 500.
  • -e: Prints status of saas-connector with echo command and then restarts the mwg services.
  • Was this article helpful?