High Availability (HA)

IBM® watsonx.data uses Multi-Zone Regions (MZRs) on both IBM Cloud and AWS to provide high availability. The various components of watsonx.data are deployed in Active-Active and Active-Only setup to help ensure high availability and resilience.

Active-Active

In an Active-Active setup, multiple instances of a component are running simultaneously across different Availability Zones (AZs). These instances are load balanced and can handle requests in parallel.

Key characteristics:

• Redundancy - If one instance or AZ fails, others continue to serve traffic without interruption.
• Load distribution - Traffic is distributed across all active instances, improving performance and reducing latency.
• Automatic failover - No manual intervention is needed; the system reroutes traffic automatically.

Benefits:

• High fault tolerance.
• Seamless user experience during zone failures.
• Better resource usage.

In watsonx.data most of the components are deployed in Active-Active setup with replicas in multiple zones to help ensure continuous availability. For example, Metadata Services (MDS) in the Enterprise plan.

Active-Only

In an Active-Only setup, a component runs in only one Availability Zone at a time. If that zone fails, the component must be restarted or redeployed in another zone.

Key characteristics:

• Single active instance per component.
• Automatic restart in a new zone upon failure.
• Slight delay during failover due to restart time.

Benefits:

• Simpler architecture.
• Reduced resource consumption.
• Resilience with a brief downtime during failover.

In watsonx.data, single-tenant components are deployed in Active-Only setup. These single-tenant components, which include Presto engine and metastore components, are strategically distributed across three AZs for capacity and failover. These components restart in a new zone during failure. For example, Metadata Services (MDS) in the Lite plan.

In Multi-Zone Regions (MZRs), Presto and MDS are distributed across different zones.

When a single availability zone fails in an MZR, or a hardware failure occurs in any region, the workloads automatically fail and restart in other zones within that region. Every watsonx.data instance comes with a default cross-regional Metadata bucket and an optional Trial bucket(10 GB). Both the buckets are enabled with IBM Cloud® Object Storage Versioning. The data is backed up by enabling replication to a separate IBM Cloud Object Storage Account. However, for any external bucket that the customer brings into watsonx.data instance, the customer is responsible for those backups.

In a regional disaster, you receive an email that includes all the steps that you need to follow. See responsibilities for watsonx.data. Single-tenant components operate on an 'Active Only' model, ensuring immediate restart on new nodes that provide the same service if there is a failure.

Single-tenant components are strategically distributed across 3 AZs to enhance reliability. When an AZ fails, sufficient capacity to initiate the required services on the available AZs is ensured. This minimizes any impact that is caused by an AZ outage.

Responsibilities

Application-level high availability

Applications that communicate over networks and cloud services are subject to transient connection failures. Design your applications to retry connections when a temporary loss in connectivity to your deployment or to IBM Cloud, causes errors. As watsonx.data is a managed service, regular updates and maintenance occur as part of normal operations. Such maintenance occasionally causes a temporary service interruption.

Your applications must be designed to handle temporary interruptions to the service, implement error handling for failed commands, and implement retry logic to recover from a temporary interruption.

The following are some of the error codes that might be expected during the temporary service interruptions:

If a Presto coordinator node restarts, be it for maintenance purposes or due to a system failure, applications are required to reestablish their connection with the Presto engine.

Several minutes of unavailability or connection interruptions are not expected. Open a support ticket with details if you have time periods longer than a minute with no connectivity so that the interruptions are investigated.

Disaster Recovery Strategy

IBM® watsonx.data provides mechanisms to protect your data and restore service functions. Business continuity plans are in place to achieve targeted recovery point objective (RPO) and recovery time objective (RTO) for the service. The following table outlines the targets for watsonx.data.

The backup interval is reduced for the service Milvus in SaaS to improve the restore RPO from 24 hours to 2 hours.

Locations