Maintain Uptime with High Availability (HA)
High Availability (HA) in Log Stream ensures uninterrupted log delivery even when one or more Cloud Connector (CC) instances become unavailable.
NOTE: Cloud Connector version 6.9.0 and higher supports High Availability (HA). To achieve HA, Skyhigh recommends configuring at least two Cloud Connectors with the same log types and regions. Cloud Connectors with distinct log types and regions do not support HA.
How High Availability Works
When two or more Cloud Connectors are configured with the same region and log types, and the Enable High Availability for Log Stream checkbox is selected, they form a logical cluster. If one or more Cloud Connector instances become unavailable, the cluster automatically redistributes the load and handles failover, minimizing downtime.
For example, if CC 1 and CC 2 are configured for the North America region and are set to handle Web and RBI log types, Log Stream will split the workload between them. If either of the Cloud Connectors fails, the available Cloud Connector takes over the entire processing load, maintaining uninterrupted log ingestion.
Illustrate High Availability Scenarios in Log Stream
Skyhigh Log Stream supports various HA configurations for Cloud Connectors. The Log Stream application behavior varies based on the assigned log types and regions across Cloud Connectors, as well as the status of the High Availability (HA) setting.
The four common configuration scenarios illustrating how Log Stream behaves under different HA configurations are listed below.
- Common Configurations with HA. For details, see Common Configurations with HA.
- Common Configurations without HA. For details, see Common Configurations without HA.
- Distinct Configurations with HA. For details, see Distinct Configurations with HA.
- Distinct Configurations without HA. For details, see Distinct Configurations without HA.
Common Configurations with HA
This scenario explains the behavior of Log Stream when CC 1 and CC 2 are configured with the same log types and regions.
| Cloud Connector | Configuration | Region | Log Types | HA Enabled | Output Sink |
|---|---|---|---|---|---|
| CC1 | Configuration 1 | North America | Web, RBI | Yes | Syslog1, Syslog2 |
| CC1 | Configuration 2 | India | Web, RBI, Private Access, CFW | Yes | Syslog1, Syslog2 |
| CC2 | Configuration 1 | North America | Web, RBI, Private Access, CFW | Yes | Syslog1 |
Behavior:
- Web and RBI logs from the North America region are shared between CC 1 and CC 2, enabling HA. The Log Stream service automatically distributes the processing load between the two.
- If CC 1 configuration 1 becomes unavailable, CC 2 configuration 1 immediately picks up the full load for shared log types, ensuring uninterrupted log delivery.
- Private Access and Cloud Firewall logs are configured only on CC 2, making them exclusive. If CC 2 fails, these logs will not be collected until restored.
- Likewise, CC 1 configuration 2 processes logs for the India region independently. If CC 1 configuration 2 fails, log processing from the India region is stopped until Cloud Connector 1 Configuration 2 becomes available again.
Common Configurations without HA
This scenario explains the behavior of Log Stream when CC 1 and CC 2 are configured with the same log types and regions, but HA is disabled.
Behavior:
- Each Cloud Connector pulls the same logs independently.
- If Cloud Connectors write to the same output sink, the data will be duplicated.
- In case of a failure, other Cloud Connectors do not take over the workload.
Distinct Configurations with HA
This scenario explains the behavior of Log Stream when CC 1 and CC 2 are configured with different log types and regions, but HA is enabled.
Behavior:
- HA grouping has no effect since there are no shared log responsibilities.
- Each Cloud Connector operates independently and processes only assigned logs.
- If a Cloud Connector fails, no other instance can take over, resulting in downtime for those logs.
Distinct Configurations without HA
This scenario explains the behavior of Log Stream when CC 1 and CC 2 are configured with different log types and regions, but HA is disabled.
Behavior:
- Cloud Connectors run independently, pulling distinct log data.
- No duplication occurs as log sources do not overlap.
- If a Cloud Connector fails, its assigned logs are not collected until the service is restored.