Creating and managing duplicate volumes

You can create a duplicate of an existing IBM Cloud® Block Storage for Classic. The duplicate volume inherits the capacity and performance options of the original volume by default. However, both attributes can be changed manually. The duplicate has a copy of the data up to the point-in-time of the snapshot that was used to create it. The duplicate volume can be dependent or independent from the original volume.

If you are a Dedicated account user of IBM Cloud® Kubernetes Service, see your options for duplicating a volume in the IBM Cloud Kubernetes Service Documentation.

Because the duplicate is based on the data in a point-in-time snapshot, snapshot space is required on the original volume before you can create a duplicate. For more information about snapshots and how to order snapshot space, see the Snapshot Documentation.

Types of duplicate volumes

Independent duplicate

Independent duplicates can be created from both primary and replica volumes. The duplicate is created in the same data center as the original volume. If you create a duplicate from a replica volume, the duplicate volume is created in the same data center as the replica volume.

Common uses for an independent duplicate volume

Gold copy. Use a storage volume as a gold copy that you can create multiple instances from for various uses.
Data refreshes. Create a copy of your production data to mount to your nonproduction environment for testing.
Development and Testing. Create up to four simultaneous duplicates of a volume at one time to create duplicate data for development and testing.

Dependent duplicate

Dependent duplicate volumes are created by using a snapshot from the primary volume. Replica volumes cannot be used to create or update dependent duplicates.

Common uses for a dependent duplicate volume

Disaster Recovery Testing. Create a duplicate of your source volume and compare it to the replica. By comparing the duplicate to the replica you can verify that the data that is being replicated is intact and can be used if a disaster occurs, without interrupting the replication.
Restore from Snapshot. Restore data on the original volume with specific files and date from a snapshot without overwriting the entire original volume with the snapshot restore function.
Data refreshes. Create a copy of your production data to mount to your nonproduction environment for testing.
Development and Testing. Create up to four simultaneous duplicates of a volume at one time to create duplicate data for development and testing.

Creating a duplicate volume in the UI

You can create duplicate volume from the CLI and in the IBM Cloud console in a couple of ways.

Creating a duplicate from the Storage List in the UI

Go to your list of Block Storage for Classic in the IBM Cloud console by clicking Infrastructure > Storage > Block Storage for Classic.
Select a volume from the list and click the ellipsis > Duplicate Volume.
Select whether the duplicate is to be dependent or independent.
Select the snapshot option to be used to create the duplicate. You can choose an existing Snapshot or take a new one.
The location entries remain the same as the original volume.
Hourly or Monthly Billing – you can choose to provision the duplicate LUN with hourly or monthly billing. The billing type for the original volume is automatically selected. If you want to choose a different billing type for your duplicate storage, you can make that selection here.
You can update the size of the new volume so that it's larger than the original. The size of the original volume is set by default.

Block Storage for Classic can be resized to 10 times the original size of the volume.
You can update the snapshot space for the new volume to add more, less, or no snapshot space.
You can select the OS Type to be different than the original volume or to stay the same.
You can specify an IOPS or IOPS Tier for the new volume if you want to. The IOPS designation of the original volume is set by default. Available Performance and size combinations are displayed.
- If your original volume is 0.25 IOPS Endurance tier, you can't make a new selection.
- If your original volume is 2, 4, or 10 IOPS Endurance tier, you can move anywhere between those tiers for the new volume.
Check the box if you read and agreed to the terms and conditions.
Click Create to continue.

Creating a duplicate from the Volume detail page the UI

Go to your list of Block Storage for Classic.
Click a LUN from the list to view the details page. (It can either be a replica or a primary volume.)
Click Actions > Duplicate Volume.
Select whether the duplicate is to be dependent or independent.
Select the snapshot option to be used to create the duplicate. You can choose an existing Snapshot or take a new one.
The location entries remain the same as the original volume.
Hourly or Monthly Billing – you can choose to provision the duplicate LUN with hourly or monthly billing. The billing type for the original volume is automatically selected. If you want to choose a different billing type for your duplicate storage, you can make that selection here.
You can update the size of the new volume so that it's larger than the original. The size of the original volume is set by default.

Block Storage for Classic can be resized to 10 times the original size of the volume.
You can update the snapshot space for the new volume to add more, less, or no snapshot space.
You can select the OS Type to be different than the original volume or to stay the same.
You can specify an IOPS or IOPS Tier for the new volume if you want to. The IOPS designation of the original volume is set by default. Available Performance and size combinations are displayed.
- If your original volume is 0.25 IOPS Endurance tier, you can't make a new selection.
- If your original volume is 2, 4, or 10 IOPS Endurance tier, you can move anywhere between those tiers for the new volume.
Check the box if you read and agreed to the terms and conditions.
Click Create to continue.

After you click Create, the order confirmation window appears. When you close the window, you return to the resources list. You can go back to your list of Block Storage for Classic volumes to click the newly provisioned duplicate. The volume details section displays information such as Duplicate Type, a link to the parent volume's details page and the name of the snapshot that was used to create the duplicate.

Creating a duplicate from the CLI

Before you begin, decide on the CLI client that you want to use.

You can either install the IBM Cloud CLI and install the SL plug-in with ibmcloud plugin install sl. For more information, see Extending IBM Cloud CLI with plug-ins.
Or, you can install the SLCLI.

Creating a duplicate from the IBMCLOUD CLI

You can use the ibmcloud sl block volume-duplicate command to create a duplicate for your block share. The following example creates an independent duplicate of the block share 560382016.

ibmcloud sl block volume-duplicate 560382016
This action will incur charges on your account. Continue?> y
OK
Order 110554892 was placed.
 > Storage as a Service
 > File Storage
 > 500 GBs
 > 4 IOPS per GB
 > 500 GB (Snapshot Space)

You may run 'ibmcloud sl block volume-list --order 110554892' to find this block volume after it is ready.

Your new duplicate is ready within minutes.

$ ibmcloud sl block volume-list --order 110554892
id          username             datacenter   storage_type             capacity_gb   bytes_used   IOPs   ip_addr                                 lunId   active_transactions   rep_partner_count   notes
560391190   SL02SEV1414935_269   dal09        endurance_block_storage   500           -            -      fsf-dal0902b-fz.service.softlayer.com   -       1                     0                   -

When you want to create a dependent duplicate of your volume, use the command with the --dependent-duplicate option. See the following example.

$ ibmcloud sl block volume-duplicate 560391190 --dependent-duplicate
This action will incur charges on your account. Continue?> y
OK
Order 110553472 was placed.
 > Storage as a Service
 > File Storage
 > 500 GBs
 > 4 IOPS per GB
 > 500 GB (Snapshot Space)

You may run 'ibmcloud sl block volume-list --order 110553472' to find this block volume after it is ready.

For more information about all of the parameters that are available for this command, see ibmcloud sl block volume-duplicate.

Creating a duplicate from the SLCLI

To create an independent duplicate Block Storage for Classic volume, you can use the following command.

$ slcli block volume-duplicate --help
Usage: slcli block volume-duplicate [OPTIONS] ORIGIN_VOLUME_ID

Options:
  -o, --origin-snapshot-id INTEGER
                                  ID of an origin volume snapshot to use for
                                  duplcation.
  -c, --duplicate-size INTEGER    Size of duplicate block volume in GB. ***If
                                  no size is specified, the size of the origin
                                  volume will be used.***
                                  Minimum: [the size
                                  of the origin volume]
  -i, --duplicate-iops INTEGER    Performance Storage IOPS, between 100 and
                                  6000 in multiples of 100 [only used for
                                  performance volumes] ***If no IOPS value is
                                  specified, the IOPS value of the origin
                                  volume will be used.***
                                  Requirements: [If
                                  IOPS/GB for the origin volume is less than
                                  0.3, IOPS/GB for the duplicate must also be
                                  less than 0.3. If IOPS/GB for the origin
                                  volume is greater than or equal to 0.3,
                                  IOPS/GB for the duplicate must also be
                                  greater than or equal to 0.3.]
  -t, --duplicate-tier [0.25|2|4|10]
                                  Endurance Storage Tier (IOPS per GB) [only
                                  used for endurance volumes] ***If no tier is
                                  specified, the tier of the origin volume
                                  will be used.***
                                  Requirements: [If IOPS/GB
                                  for the origin volume is 0.25, IOPS/GB for
                                  the duplicate must also be 0.25. If IOPS/GB
                                  for the origin volume is greater than 0.25,
                                  IOPS/GB for the duplicate must also be
                                  greater than 0.25.]
  -s, --duplicate-snapshot-size INTEGER
                                  The size of snapshot space to order for the
                                  duplicate. ***If no snapshot space size is
                                  specified, the snapshot space size of the
                                  origin block volume will be used.***
                                  Input
                                  "0" for this parameter to order a duplicate
                                  volume with no snapshot space.
  --billing [hourly|monthly]      Optional parameter for Billing rate (default
                                  to monthly)
  -h, --help                      Show this message and exit.

Dependent duplicate volumes can be ordered from the CLI, too, with the option --dependent-duplicate TRUE.

slcli block volume-duplicate --dependent-duplicate TRUE <primary-vol-id>

For more information about available command options, see block volume-duplicate.

Creating a duplicate LUN with the API

To order an independent duplicate Block Storage for Classic volume with the API, you can make a POST call. The following REST API example creates an independent duplicate for an Endurance (IOPS tiers) volume.

URL - https://USERNAME:APIKEY@api.softlayer.com/rest/v3.1/SoftLayer_Product_Order/placeOrder
Type - POST

Request body -

{
    "parameters":[{
    "complexType": "SoftLayer_Container_Product_Order_Network_Storage_AsAService",
    "packageId": 531,
    "duplicateOriginVolumeId":<PrimaryId>,
    "isDependentDuplicateFlag": 0,
    "prices": [{"id": 19497}, {"id": 16479}, {"id": 12931}, {"id": 15749}, {"id":10407}],
    "quantity": 1,
    "osFormatType":{
     "keyName": "LINUX"
    },
    "location": 2,
    "volumeSize":23
    }]
}

To order a dependent duplicate for a Performance (custom IOPS) volume, make a POST /SoftLayer_Product_Order/placeOrder call like the following REST API example.

URL - https://USERNAME:APIKEY@api.softlayer.com/rest/v3.1/SoftLayer_Product_Order/placeOrder
Type - POST

Request body -

{
    "parameters":[{
    "complexType": "SoftLayer_Container_Product_Order_Network_Storage_AsAService",
    "packageId": 531,
    "duplicateOriginVolumeId":1327277,
    "isDependentDuplicateFlag": 1,
    "prices": [{"id": 15751}, {"id": 19487}, {"id": 18983}, {"id": 15749}, {"id":10407}],
    "quantity": 1,
    "iops":454,
    "osFormatType":{
        "keyName": "LINUX"
    },
    "location": 2,
    "volumeSize":23
    }]
}

For more information about the API and the options, see the API Reference.

Managing your duplicate volume

All duplicate volumes can be accessed by a host for read and write operations as soon as the volume is provisioned.

However, snapshots and replication of independent duplicate volumes aren't allowed until the data copy from the original to the duplicate is complete and the duplicate volume is fully independent. Depending on the size of the data, the separation process can take several hours. When it's complete, the duplicate can be managed and used as an independent volume.

While data is being copied from the original volume to the independent duplicate, you can see that the status indicator on the details page shows the duplication is in progress. During this time, you can attach to a host, and read and write to the volume, but you can't create snapshot schedules or initiate a refresh. When the separation process is complete, the new volume is independent from the original, and can be managed with snapshots and replication as normal. After the conversion is complete, the independent volume can be manually refreshed by using a snapshot from the parent volume.

Dependent duplicates do not go through the separation process and can be refreshed manually at any time. The refresh process can be initiated from the CLI, with the API and in the UI. Later, if you want to convert the dependent duplicate into an independent volume, you can initiate that process by using the UI, the CLI, or the API, too.

The dependent duplicate volume locks the original snapshot so the snapshot cannot be deleted while the dependent duplicate exists.

Updating data on the duplicate from the parent volume in the UI

As time passes and the primary volume changes, the duplicate volume can be updated with these changes to reflect the current state through the refresh action. The refresh involves taking a snapshot of the primary volume and then updating the duplicate volume by using the data from that snapshot.

If the duplicate volume is independent, you can stop a running refresh operation and start a new one.

Go to your list of Block Storage for Classic in the IBM Cloud console by clicking Infrastructure > Storage > Block Storage for Classic.
Locate the duplicate volume and click its name to view the volume details.
Click Actions > Restore parent snapshot.
From the list of snapshots, select the parent snapshot that holds the data that you want to restore to the duplicate volume. If the duplicate volume that you're refreshing is an independent volume, you can stop a running operation and force a new restore to start. If you want to force any current refresh process to stop, check the box before you proceed.

Restoring data from a snapshot results in the loss of any data that was created or modified since the selected snapshot was taken. During the refresh transaction, the duplicate volume is disabled and must be remounted after the refresh is completed.
Click Yes to start the refresh. The refresh can take a while to complete. The status bar shows the percentage of data that is copied to the volume. To see updated status, refresh the page in the browser.

Converting a dependent volume to an independent duplicate in the UI

Go to your list of Block Storage for Classic in the IBM Cloud console by clicking Infrastructure > Storage > Block Storage for Classic.
Locate the duplicate volume and click its name to view the volume details.
Click Actions > Convert Dependent Duplicate.
Check the box to confirm that you want to proceed with the conversion.
Click Yes.

The conversion process can take some time to complete. The bigger the volume is, the longer it takes to convert it. You can view the status of the process on the volume details page under the Duplicate conversion status header.

Updating data on the duplicate from the parent volume from the CLI

Refreshes can be initiated by using the following command.

slcli block volume-refresh <duplicate-vol-id> <primary-snapshot-id>

A refresh incurs no downtime on the primary volume. However, during the refresh transaction, the duplicate volume is disabled and must be remounted after the refresh is completed.

The refresh process can be time-consuming. If you find that you have new data that you want to copy to the independent duplicate volume, you can issue the slcli block volume-refresh command with the --force-refresh option to stop all ongoing and pending refresh transactions, and initiate a new refresh.

The force refresh process works only on independent volumes.

For more information about available command options, see slcli block volume-refresh.

Converting a dependent volume to an independent duplicate from the CLI

If you want to use the dependent volume as a stand-alone volume in the future, you can convert it to a normal, independent IBM Cloud® Block Storage for Classic volume through the SLCLI. Use the following command.

slcli block volume-convert <dependent-vol-id>

The conversion process can take some time to complete. The bigger the volume is, the longer it takes to convert it. Use the following command to check on the progress.

slcli block duplicate-convert-status <dependent-vol-id>

The following example shows the output that you can expect.

slcli block duplicate-convert-status 370597202
Username            Active Conversion Start Timestamp   Completed Percentage
SL02SEVC307608_74   2022-06-13 14:59:17                 90

For more information about available command options, see duplicate-convert-status.

Updating data on the duplicate from the parent volume with the API

A refresh incurs no downtime on the primary volume. However, during the refresh transaction, the duplicate volume is disabled and must be remounted after the refresh is completed.

The refresh process can be time-consuming. You might find that you have new data that you want to add to the duplicate before the running refresh is finished. If that's the case, you can make a second call to refreshDuplicate and specify the second forceRefresh parameter as true to stop all ongoing and pending refresh transactions, and initiate a new refresh. If the second parameter is set to false or it is not specified, the call fails if another refresh is already in progress.

The force refresh process works only on independent volumes.

REST API example

URL - https://USERNAME:APIKEY@api.softlayer.com/rest/v3.1/SoftLayer_Network_Storage/duplicateVolumeId/refreshDuplicate
Type - POST

Request body -

{
 "parameters": [primaryVolumeSnapshotId, true OR false]
}

SOAP API example

URL - https://api.softlayer.com/soap/v3.1/SoftLayer_Network_Storage
Type - POST

Request body -

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://api.service.softlayer.com/soap/v3.1/">
<SOAP-ENV:Header>
 <ns1:authenticate>
  <username>USERNAME</username>
  <apiKey>APIKEY</apiKey>
 </ns1:authenticate>
 <ns2:SoftLayer_Network_StorageInitParameters>
  <id>duplicate Volume Id</id>
 </ns2:SoftLayer_Network_StorageInitParameters>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
 <ns1:refreshDuplicate>
   <snapshotId xsi:type="int">primary Volume Snapshot Id</snapshotId>
   <forceRefresh xsi:type="boolean">true</forceRefresh> <-- (remove this tag for normal refresh)
 </ns1:refreshDuplicate>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

For more information about the API and the options, see the API Reference and SoftLayer_Network_Storage::refreshDuplicate.

Converting a dependent volume to an independent duplicate with the API

If you want to use the dependent volume as a stand-alone volume in the future, you can convert it to a normal, independent IBM Cloud® Block Storage for Classic volume with the API. See the following example that uses the REST API.

URL - https://USERNAME:APIKEY@api.softlayer.com/rest/v3.1/SoftLayer_Network_Storage/<storageId>/convertCloneDependentToIndependent
Type - POST
Request body - blank

For more information about the API and the options, see the API Reference.

Canceling a storage volume with a dependent duplicate

Canceling a parent volume that has active dependent volumes requires canceling the dependent duplicate volumes first.