Terms and definitions

GRS in Client location

The GRS enablement in IBM Power Virtual Server Private Cloud enables asynchronous replication of data between the primary location infrastructure and the secondary location infrastructure. The two infrastructure locations have the identical set of capabilities that IBM Power Virtual Server in IBM data center provides.

Provide the network configuration between the primary location and the secondary location for replication, which includes the following prerequisites:

• The network bandwidth must be greater than or equal to 10 Gbps.
• The network latency must be less than or equal to 200 ms.

During the first synchronization, the entire data from primary volumes is copied to the auxiliary volumes. For subsequent synchronizations, only the changes between the two synchronization operations are copied. The effective Recovery Point Objective (RPO) depends on the capability of the underlying network throughput and the application characteristics. If the network throughput is insufficient to reach the defined RPO, then the time duration between the synchronization increases.

If the defined RPO exceeds for more than 8 hours, the IBM operations team alerts you.

Replication is not supported between the VMs in IBM Power Virtual Server in IBM data center environment and IBM Power Virtual Server Private Cloud in Client location environment.

Pricing for GRS

Part numbers are used for calculating the cost of GRS based on the storage tier that is associated with the primary volume. For more information, see Pricing for Global Replication Services (GRS).

Power Virtual Server regions that support GRS

The following table shows the location pairs that support replication:

To get a list of replication-enabled location names, use the following API or CLI commands:

• API: Get the disaster recovery site details for the current location

• CLI: ibmcloud pi disaster-recovery. Set the --all-regions flag as true.

Verify whether the primary and secondary locations to be used for replication are enabled for replication services and are in the list of replication-enabled pairs.

In the following example, the wdc07 and dal10 data centers are in a replication-enabled pair:

{
  "location": "dal10",
  "replicationSites": [
    {
      "ReplicationPoolMap": [
        {
          "remotePool": "General-Flash-92",
          "volumePool": "General-Flash-83"
        }
      ],
      "isActive": true,
      "location": "wdc07"
    }
  ]
}

Power Virtual Server storage pools that support GRS

You can allocate replication-enabled volumes only from storage pools that support replication. To identify the storage pools that support replication, use the following API or CLI commands:

• API: Storage capacity for all available storage pools in a region

• CLI: ibmcloud pi storage-pools

If a storage pool supports replication, the replication-enabled property is set to true.

When you create a volume that is replication-enabled, by default the volume is created within one of the storage pools that supports replication. If you want to update a volume to enable replication, ensure that you create it within a storage pool that supports replication. Otherwise, the update operation fails.

Creating a volume group for GRS

A volume group is a Power Virtual Server managed resource. By using the volume group, you can enable, disable, and manage a storage replication consistency group. The volume group contains the volumes that must be recovered at the time of disaster. You can add the replication-enabled volumes to a volume group. A replication-enabled volume can be a part of only one volume group at a time. In addition, all volumes in a volume group must be a part of the same storage pool.

When you create a volume group on the primary location, the storage replication consistency group is created in the storage backend at both primary and secondary locations. The storage replication consistency group stores the consistent copy for the volumes in the volume group. To perform any operations on the consistency group, you must perform the operation on the volume group that represents the consistency group. Power Virtual Server does not directly manage the consistency group in the storage backend. A storage backend is the storage subsystem that contains the storage pool and the storage controller.

When you onboard a volume on the secondary location, if the volume group is not yet created, the volume is created and the onboarded auxiliary volume is added to it. This volume group is visible and managed by Power Virtual Server on the secondary location. For more information, see Onboarding an auxiliary volume.

During the first data synchronization, the entire data from the primary volume is copied to the auxiliary volumes. For subsequent data synchronizations, only the changes between the two synchronization operations are copied. The effective Recovery Point Objective (RPO) depends on the capability of the network throughput and the application characteristics. If the network throughput is insufficient to meet the defined RPO, the duration between the data synchronization increases.

A volume group is used to enable, disable, and manage a replication-enabled consistency group in storage volumes. By using a consistency group, you can perform operations on multiple volumes instead of managing each volume individually. The consistency group has a state property that indicates the state of the copy operation of the volumes in the group.

Refer to the following table that lists different states of the replication-enabled consistency group in storage volumes.

If you perform any operation on a volume group at one location, it affects the associated volume group on the other location. For example, consider that a volume group is stopped on a primary location. After sometime, the associated volume group on the secondary location is updated to reflect the replication status of the pair in the volume group. In this case, the replicationStatus field displays the state of the volume group as disabled on both primary and secondary locations.

Before you perform any operation on a volume group at one location, confirm the replication status of both the volume groups at the primary and secondary locations.

You can group the replication-enabled volumes that are similar in function. For example, volumes that are related to a specific workload can be grouped under one volume group. When additional replication-enabled volumes are created, you can add a volume to an existing volume group with similar volumes based on the function. You can create a new volume group for the volumes if their function is not same as the existing volumes.

Preparation for disaster recovery

Consider that you have virtual server instances with data volumes that are running workloads. If a failure occurs and if the data volumes are replicated, you can recover the data volumes from the secondary location. To enable the volume replication service, complete the following actions:

• Actions on the primary site
• Actions on the secondary site

You must complete the actions on the primary site before you proceed with the actions on the secondary site.

Performing a failover and fallback operation

If a disaster occurs on the primary location, access to all the storage volumes that are allocated on the primary location is lost. The replication relationship for the replication-enabled primary volumes is broken with the secondary location. The state of the consistency group (volume group) changes to inconsistent-disconnected. On the primary location, no volume is associated with the primary role in the volume group.

Complete the following steps to perform the failover operations on the secondary location:

1. Stop the auxiliary volume group and access the auxiliary volume on the secondary location. See, Failover to the secondary location
2. Verify that the auxiliary volume group is in an idling state

When you complete the failover steps, the volumes are ready to accept I/O requests. The virtual server instance that these volumes are attached to can be powered on and you can continue the execution.

    Request Body:
    {
      "start": {
        "source": "aux"
      }
    }

    Request Body:
    {
      "stop": {
        "access": true
      }
    }

    Request Body:
    {
      "start": {
        "source": "master"
      }
    }

Updating an existing volume as a replication-enabled volume

You can modify an existing volume to be replication-enabled if it is created in a storage pool that supports the replication process. To verify, query the details of the volume, get the details of the storage pool that contains the volume, and verify it with the list of storage pools that support replication. For more information about storage pools that support replication, see Power Virtual Server storage pool that supports GRS.

Use the following API and CLI commands to query the volume details:

• API: Perform an action on a volume group. Set the VOLUME_ID parameter to the primary volume ID. Using the API, query the volume details as defined in the following code:

    Request Body:
    {
      "replicationEnabled": true
    }

• CLI: ibmcloud pi volume-group action. Set the VOLUME_ID parameter to the primary volume ID, and query if the --replication-enabled flag is set to True.

Adding a replication-enabled volume to an existing volume group

You can add replication-enabled volumes to an existing volume group by using the following API and CLI commands:

• API: updates the volume group. Set the VOLUME_GROUP_ID parameter to the primary volume group ID. Using the API, add the primary volume IDs to the volume group as defined in the following code:

    Request Body:
    {
      "addVolumes": [
        "PRIMARY_VOLUME_ID"
      ]
    }

• CLI: ibmcloud pi volume-group update. Set the VOLUME_GROUP_ID parameter to the primary volume group ID, and set the --add-member-ids flag to the primary volume IDs that must be added to the volume group.

Disabling replication of a volume

When you disable replication of a primary volume, the associated auxiliary volume on the secondary location is deleted.

Failure to complete this procedure in the specified order might result in the loss of source volume data. First, you must complete the actions on the primary location and then complete the actions on the secondary location.

To disable replication of a primary volume, perform the following steps:

1. Actions on the primary location

   1. Remove the primary volume from the volume group
   2. Delete the primary volume group, if empty
   3. Disable replication for the primary volume
   4. Verify whether the replication is disabled for the primary volume

2. Actions on the secondary location

You can perform these steps only if you complete the onboarding operation of the auxiliary volume.

1. Remove the auxiliary volume from the volume group
2. Delete the auxiliary volume group, if empty
3. Delete the auxiliary volume

    Request Body:
    {
      "removeVolumes": [
        "PRIMARY_VOLUME_ID"
      ]
    }

    Request Body:
    {
      "replicationEnabled": false
    }

    Request Body:
    {
      "removeVolumes": [
        "AUXILIARY_VOLUME_ID"
      ]
    }

Modifying a replication-enabled primary volume

You can modify the attributes of a replication-enabled primary volume. To modify some of the properties of the primary and auxiliary volumes, you must perform related actions on the primary and secondary locations.

  Request Body:
  {
    "size": SIZE_IN_GB
  }

Changing the bootable and shareable properties of a primary volume

To change the values of the bootable or shareable properties on the primary volume, use the following API or CLI commands:

-API: Update a cloud instance volume. Set the VOLUME_ID parameter to the primary volume ID and set the --bootable or --shareable flags to True or False value.

-CLI: ibmcloud pi volume update. Set the value of the VOLUME_ID parameter to the primary volume ID and set the value of the --bootable or --shareable flags to True or False value.

A volume can be either shareable or bootable, but cannot be both.

After you change the bootable or shareable property values for a primary volume on the primary location, you must update the same property for the auxiliary volume on the secondary location.

Changing the tier of a primary volume

You cannot change the tier of a replication-enabled volume.

Deleting a replication-enabled primary volume

To delete a volume or a replication-enabled primary volume, the status of the volume must indicate one of the following states: available, error, error_restoring, error_extending, or error_managing. In addition, you cannot delete the volume if it is in the migrating or attached state, belongs to a volume group, has snapshots, or if it is disassociated from its snapshots after a transfer.

To delete a primary volume, you must complete the actions on the primary and secondary locations:

• Complete the following actions on the primary location:

  1. Remove the primary volume from its volume group, if the primary volume is associated with a volume group
  2. Delete the primary volume

• Complete the following actions on the secondary location, if the auxiliary volume is onboarded on the secondary location:

  1. Remove an auxiliary volume from its volume group, if the auxiliary volume is associated with a volume group
  2. Delete the auxiliary volume

If the auxiliary volume is not deleted from the secondary site, an out-of-band periodic check that occurs every 24 hours sets the auxiliary volume to an ERROR state. You cannot use volume when it is in ERROR state and the only operation that you can perform is to delete the volume.

    Request Body:
    {
      "removeVolumes": [
        "PRIMARY_VOLUME_ID"
      ]
    }

Deleting a replication-enabled auxiliary volume

If you delete an auxiliary volume, the associated primary volume is also deleted.

When you disable the replication service on the primary volume or delete the primary volume, the replication relationship between the primary volume and the secondary volume is deleted in the storage backend. If the auxiliary volume on the secondary location is associated with a volume group, remove the auxiliary volume from the volume group. Delete the auxiliary volume manually. If you do not delete the auxiliary volume from the secondary site, an out-of-band periodic check that occurs every 24 hours sets the auxiliary volume to an ERROR state. Confirm the status of the auxiliary volume by checking the outOfBandDeleted property of the auxiliary volume.

GRS impacts on other Power Virtual Server operations

The impacts of GRS on other Power Virtual Server operations are as follows:

• Image interface is not changed and replication is not supported for images
• Operation interface and the affinity policies for storage pool for virtual server instances are not changed. The replication and non-replication-enabled volumes support volume attachment and detachment.
• Snapshot and capture operations are allowed for a virtual server instance with replication and non-replication-enabled volumes when the volumes are from the same storage pool. The snapshot and capture operations are performed by using the non-replication-enabled volumes internally even though the volumes of a virtual server instance are replication-enabled.
• Cloned volume is replication-enabled when you clone a replication-enabled volume by default. During cloning, you can specify whether the cloned volume must be replication-enabled or not. If you want the cloned volume to be replication-enabled, specify the policy for the clone volume. You can use the following methods to clone a volume:
  • Clone a volume by using API.
  • Clone a volume by using CLI.
  • Clone a volume by using Terraform.

Best practices for GRS

• Set the Shareable and Bootable flag explicitly on onboarded volumes, if required.
• Start the onboarding of auxiliary volumes only when the primary volumes and volume group are in a consistent copying state. You can get the volume details by using the get volume API to determine the state of the primary volumes and volume group in the mirroring state on the primary site. Verify that the volume group is created successfully and it is in a consistent copying state by using the get storage details of the volume group API.

• Get volume details by using the following methods:
  • Power Cloud API.
  • Terraform.
• Get the details of the volume groups storage details by using the following methods:
  • Power Cloud API.
  • Power Virtual Server CLI
  • Terraform.

• When you add or remove a primary or an auxiliary volume from a volume group from one location, perform the same operation from the other location to keep the data in sync.
• Delete the volumes from the primary and secondary location. Volumes are charged when you delete an auxiliary volume but fail to delete the primary volume.
• Use the primary location for all the volume operations and perform operations on the auxiliary volume on the secondary location only during failover.