|Limited Availability: SIEM Integration (Inline) is a Limited Availability feature. To enable this feature, contact Skyhigh Support.
Before you Begin
Ensure the following prerequisites are met:
- You must have the Skyhigh Cloud Connector user role to configure Cloud Connector. For details, see About User Roles and Access Levels.
- A Skyhigh CASB User Interface (UI) can only be accessed from a network on which your Cloud Connector is installed, or if you are accessing UI from a different VLAN then you should be on the same network as Cloud Connector. You must have your On-Prem Cloud Connector in a reachable location so that you can store or retrieve sensitive data from Cloud Connector.
NOTE: The Logging Client is integrated into Skyhigh Cloud Connector (on-premise) and is renamed as the Log Collector on the Cloud Connector SIEM Integration (Inline) tab. The Log Collector functions in the same manner as the Logging Client.
The Log Collector (found under Settings > Infrastructure > Cloud Connector > SIEM Integration (Inline)) collects Debug and Error log files to perform analysis with Skyhigh SSE. It can also collect Security Service Edge web access data in your network or feed directly into your reporting and analytics tools. The Log Collector retrieves and stores the logs at configured intervals. You can save the logs to a local directory or send them from Skyhigh CASB to your third-party SIEM systems (Security Information and Event Management) through a Syslog server.
The Log Collector can simultaneously download the data originating from different log types such as SWG, Remote Browser Isolation (RBI), Private Access, and Cloud Firewall which eliminates running multiple instances to collect data from different log types. A single Enterprise Cloud Connector accommodates downloading multiple log types at the same time.
NOTE: You must have SWG, RBI, PA, and Cloud Firewall licenses to use this feature.
Log Collector with Skyhigh Cloud Connector
This section explains how the logs are fetched and stored (in a local directory or on the Syslog server) when a CC is up and running and when a CC is down.
- ► How Logs are Collected and Stored using Cloud Connector
- CC fetches the configuration details provided on the Log Collector Configuration page.
- Using this information, CC connects to Forensic API. Forensic API is interconnected to the Greenplum database that stores the organization’s logs. The data in these logs originates from SWG, Remote Browser Isolation (RBI), Private access, and Cloud Firewall.
- CC fetches the logs from the Greenplum database through Forensic API according to the configured log type and timestamp that are set on the Log Collector Configuration page. Timestamp refers to the time interval period that is set for Scheduling Interval to fetch the logs from the database.
- Forensic API is region-specific. You should select the region for the endpoint based on your geographic location. For example, if you are accessing the logs from India, select the region as IN and not other regions which slows down the process of collecting and storing the logs.
- These logs can be stored in a local directory or sent to a Syslog server.
NOTE: Even if the CC is down for a certain period, no data loss will occur. When the CC resumes, it starts fetching and saving the logs from the time the last successful request was made by the Log Collector.
Migrate to Log Collector
NOTE: This section is intended only for the existing Logging Client users. For new users, the Log Collector gets installed automatically when you download and install the Cloud Connector installer version 6.5.2 and above. To download the latest version of the Cloud Connector, see Download Skyhigh Cloud Connector.
Before migrating to the Log Collector, you must stop the existing Logging Client. Once you migrate to the Log Collector, you can start collecting the logs from the Log Collector. The logs collected during the migration period will not be lost.
IMPORTANT: Stop the existing Logging Client to avoid duplicate events before enabling the Log Collector. You can stop the Service by clicking Stop Service on the existing Logging Client configuration page.
Perform the below migration steps to collect the logs from the Log Collector:
- ► Step 1
Stop the existing Logging Client and collect the toTime timestamp. The Logging Client has pulled the data until the toTime timestamp. The toTime is recorded in the time_stamp table of the configuration.db.
You can get the toTime timestamp in two different ways:
- By running the online tools
- By running the Python script from the Logging Client
Use the online tool
Use online tools such as https://sqliteonline.com/ and from the tool, select configuration.db file (File > Open DB), and then run
SELECT fromTime, toTime FROM time_stampcommand to get toTime.
NOTE: Convert toTime into yyyy.MM.dd.HH.mm.ss format.
Run the Python script from the Logging Client
Run the below Python script from the existing Logging Client to know the toTime timestamp.
NOTE: Python 3.x should be installed to run the below script.
import sqlite3 #Creating file path dbfile = "C:\<logging-client-tool-installed-path>\app-x.x.x\configuration.db" #Create a SQL connection to our SQLite database con = sqlite3.connect(dbfile) #Query to execute query = "select toTime from time_stamp" cur = con.cursor() #Execute query results = cur.execute(query) for x in results: print("To Time: " + x) con.close()
Output of the above command:
- ► Step 2
Download and install Cloud Connector version 6.5.2 and above. For installation details, see Download Skyhigh Cloud Connector.
- ► Step 3
Start the Cloud Connector, and then configure Log Collector on the SIEM Integration (Inline) tab. For Log Collector configurations, see Log Collector Configuration. Once you complete the configuration, you must STOP the Cloud Connector.
- ► Step 4
Run the below CLI commands to update the API version to the latest version and to update the toTime timestamp of the last successful request made by the Logging Client. Once you execute the below commands, start the CC to collect and process the logs.
NOTE: By default, the API version is set to the latest version. However, you can update the API version through CLI commands. Remote Browser Isolation (RBI), Private Access, and Cloud Firewall support API version 9 and above whereas the SWG supports all the API versions from 1 until the latest version. For details, see Reporting Fields.
CLI commands to update the API version.
Operating System CLI Command
root@EC-VM12:/shn/wlc# ./shnlpcli ufav --version 12
PS C:\shnlp> .\shnlpcli.exe ufav --version 12
CLI commands to update the toTime timestamp of the last successful request made by the Logging Client. This command allows the Cloud Connector to make consecutive requests from the updated toTime timestamp.
Operating System CLI Command Linux
root@EC-VM12:/shn/wlc# ./shnlpcli updateLoggingClientLastSuccess --logType "SWG" --updateTime <toTime>
PS C:\shnlp> .\shnlpcli.exe updateLoggingClientLastSuccess --logType "SWG" --updateTime <toTime>
Log Collector Configuration
You can find the Log Collector Configuration on the Skyhigh CASB user interface (located under Settings > Infrastructure > Cloud Connector > SIEM Integration (Inline)). By default, the Log Collector Configuration is turned OFF. Use the toggle button on the Log Collector Configuration to turn it ON, and then configure the settings based on the table below:
Choose a region or country code depending on where your data is stored. By default, US is the selected region:
Enter the scheduling interval, a periodic interval starting from 2 minutes up to 30 minutes. The Cloud Connector fetches the logs from the database according to the time interval that you set. By default, the interval is set to 2 minutes.
You can simultaneously download different types of logs, including logs with data originating from the SWG, Remote Browser Isolation (RBI), Private Access, and Cloud Firewall. By default, the Log Type is selected as SWG. The Log Collector downloads log data based on the chosen log types.
For details about the licenses, contact Skyhigh Support.
|Save in Directory
Use this option to store the log files in your local directory. If you select this option, choose the path to download the logs to your system. The logs are collected in the form of gzip format, and then it is converted to CSV format and stored in the local directory.
|Send as Syslog
Use this option to send the log files to the Syslog server. If you choose to Send as Syslogs, configure the following options:
IMPORTANT: If you are migrating from the Logging Client, follow the process provided in the Migrate to Log Collector after configuring the Log Collector.
Once the Log Collector is successfully configured, the Cloud Connector pulls the Log Collector configurations and you will see the following message in the debug log.
24 Jan 2024 22:55:37 [DEBUG] [taskScheduler-1] c.s.l.c.t.LoggingClientTask| Logging Client enabled,fetching Data
If you experience any issues with the configurations, contact Skyhigh Support for assistance.
Replay Logging Client data:
Used to retrieve the logs within the specified time frame.
root@EC-VM12:/shn/swathi/qaar/lc# ./shnlpcli replayLoggingClientData --startTime 2023.12.11.04.55.00 --endTime 2023.12.11.07.15.0 --logType SWG
Update Logging Client last success:
Used to pull the logs from the specified time.
root@EC-VM12:/shn/swathi/qaar/lc# ./shnlpcli updateLoggingClientLastSuccess --updateTime 2024.01.29.08.40.00 --logType SWG
Update forensic API version:
Used to update the forensic API version.
root@EC-VM12:/shn/swathi/qaar/lc# ./shnlpcli ufav --version 12
Show the current forensic API version:
Used to display the configured forensic API version.
root@EC-VM12:/shn/swathi/qaar/lc# ./shnlpcli sfav
IMPORTANT: The following command lists all CLI commands available in CC:
root@ecqa-automation00-new:/shn/balu-perf# ./shnlpcli --help