Migrating Block Storage for VPC volumes from Gen-1 to Gen-2 profiles

Understanding the profile generations

Block Storage for VPC has evolved from Generation 1 (tier-based) to Generation 2 (Software Defined Performance) profiles.

Generation 1 (tier-based)

IOPS are calculated based on volume size. For more information, see Block Storage profiles.

Generation 2 (SDP)

Custom IOPS independent of volume size. For more information, see SDP Block Storage profile.

For the latest details on block storage profiles and their capabilities, see Block Storage profiles.

Before you begin

Before you migrate your volumes, complete the following steps.

• Install the IBM Cloud CLI.
• Ensure you have kubectl or oc access to your cluster.
• Verify that the VPC Block CSI Driver add-on is version 5.0 or later. To check the version, run the following IBM MobileFirst Platform Command Line Interface

  oc get deployment -n kube-system vpc-block-csi-controller -o jsonpath='{.spec.template.spec.containers[0].image}'
  

• Ensure that the VPC Block CSI Driver add-on is installed. For more information, see Installing the VPC Block CSI Driver add-on.
• Ensure that the VPC Block CSI Driver add-on is configured. For more information, see Configuring the VPC Block CSI Driver add-on.
• Ensure that the VPC Block CSI Driver add-on is running. For more information, see Verifying the VPC Block CSI Driver add-on.
• Ensure that the VPC Block CSI Driver add-on is up-to-date. For more information, see Upgrading the VPC Block CSI Driver add-on.
• Document your current performance requirements.
• Create a backup or snapshot of your data. For more information, see Setting up snapshots with the Block Storage for VPC cluster add-on.

Identify volumes for migration

Find PVCs that use Gen-1 block storage profiles.

1. List all block storage PVCs in your cluster.

   oc get pvc --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,STORAGECLASS:.spec.storageClassName,STATUS:.status.phase | grep vpc-block
   

2. Get the VPC volume ID from a PVC.

   PV_NAME=$(oc get pvc <pvc-name> -n <namespace> -o jsonpath='{.spec.volumeName}')
   VOLUME_ID=$(oc get pv $PV_NAME -o jsonpath='{.spec.csi.volumeHandle}')
   echo "Volume ID: $VOLUME_ID"
   

   Example output

   Volume ID: r026-3afe630d-1a47-482c-aade-e37f5e0bb9f4
   

Check the current volume configuration

1. View the current block volume profile and configuration.

   ibmcloud is vol $VOLUME_ID
   

   Example output before migration

   ID                      r026-3afe630d-1a47-482c-aade-e37f5e0bb9f4
   Name                    pvc-d4a2b016-a5e4-46de-9214-a64f5fc8e344
   Status                  available
   Attachment state        attached
   Capacity                100
   IOPS                    3000
   Bandwidth(Mbps)         393
   Profile                 5iops-tier
   Storage Generation      1
   Adjustable IOPS         false
   Busy                    false
   

2. Note the following values from the output. You'll use these to verify the migration was successful.

   • Profile: 5iops-tier (Gen-1)
   • Storage Generation: 1
   • Adjustable IOPS: false

Start the migration

The migration process runs in the background while your volume remains attached to your pod and your application continues running. IOPS are automatically adjusted to SDP defaults, and the process typically takes 5-15 minutes.

1. Initiate the migration to the SDP profile.

   ibmcloud is volume-job-create $VOLUME_ID --profile sdp
   

Monitor migration progress

1. Check the migration job status. If the output indicates Status: updating and Busy: true, the migration is in progress. When the output shows Status: available and Busy: false, the migration is complete.

   ibmcloud is volume-jobs $VOLUME_ID
   

2. Check the volume status.

   watch -n 30 "ibmcloud is vol $VOLUME_ID | grep -E 'Status|Busy|Profile'"
   

   Example output during migration

   Status                  updating
   Busy                    true
   Profile                 5iops-tier
   

Verify migration completion

1. Confirm that the volume is using the Gen-2 profile.

   ibmcloud is vol $VOLUME_ID
   

   Example output after migration

   ID                          r026-3afe630d-1a47-482c-aade-e37f5e0bb9f4
   Name                        pvc-d4a2b016-a5e4-46de-9214-a64f5fc8e344
   Status                      available
   Attachment state            attached
   Capacity                    100
   IOPS                        4000
   Bandwidth(Mbps)             1000
   Profile                     sdp
   Storage Generation          2
   Adjustable IOPS             true
   Adjustable Capacity States  unattached,attached
   Adjustable IOPS States      unattached,attached
   Busy                        false
   

2. Verify the following success indicators:

   • Profile: sdp
   • Storage Generation: 2
   • Adjustable IOPS: true
   • IOPS: Increased (for example, 3000 to 4000)
   • Bandwidth: Improved

Updating the PersistentVolume metadata

1. Update the PV metadata to reflect the Gen-2 profile.

   oc patch pv $PV_NAME --type='json' -p='[
     {"op": "replace", "path": "/spec/storageClassName", "value": "ibmc-vpc-block-sdp"},
     {"op": "replace", "path": "/spec/csi/volumeAttributes/profile", "value": "sdp"},
     {"op": "replace", "path": "/spec/csi/volumeAttributes/iops", "value": "4000"}
   ]'
   

2. Verify the update.

   oc get pv $PV_NAME -o jsonpath='{.spec.storageClassName}'
   

   Expected output

   ibmc-vpc-block-sdp
   

   The PVC continues to show the original StorageClass name. This behavior is expected because the storageClassName field in a PVC is immutable. The actual storage profile is determined by the PV, which you updated in this step.

Validate your application

Verify that your application is functioning correctly after the migration.

1. Check the pod status. Verify that all pods are in the Running state.

   oc get pods -n <namespace>
   

2. Verify the volume attachment. Confirm that the volume is mounted correctly.

   oc describe pod <pod-name> -n <namespace> | grep -A 5 "Volumes:"
   

3. Check the application logs. Look for any errors related to storage or I/O operations.

   oc logs <pod-name> -n <namespace> --tail=50
   

4. Test the application endpoint if applicable. Verify that the application responds correctly.

   curl -I https://<app-endpoint>/health
   

Create a Gen-2 StorageClass

Create an SDP StorageClass for provisioning new volumes.

1. Create a YAML file for the StorageClass. Save the following configuration as storageclass-sdp.yaml.

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: ibmc-vpc-block-sdp-general
     labels:
       app: ibm-vpc-block-csi-driver
   provisioner: vpc.block.csi.ibm.io
   parameters:
     profile: "sdp"
     iops: "5000"
     csi.storage.k8s.io/fstype: "ext4"
     billingType: "hourly"
     encrypted: "false"
     region: ""
     zone: ""
     tags: "generation:2,profile:sdp"
   reclaimPolicy: Delete
   allowVolumeExpansion: true
   volumeBindingMode: WaitForFirstConsumer
   

2. Apply the StorageClass.

   oc apply -f storageclass-sdp.yaml
   

3. Verify the StorageClass creation.

   oc get storageclass ibmc-vpc-block-sdp-general
   

Adjust IOPS and throughput to optimize performance

After migration, you can adjust IOPS and throughput (bandwidth) dynamically based on your workload requirements.

• Start with 5,000 IOPS for general workloads.
• Use 10,000 or more IOPS for databases.
• Monitor usage and adjust based on actual requirements.

Troubleshooting

Next steps

After completing the migration, consider the following next steps.

• Migrate additional volumes by repeating this tutorial.
• Monitor volume performance and adjust IOPS as needed.
• Update your deployment templates to use Gen-2 StorageClasses for new volumes.
• Review Block Storage capacity and performance for optimization guidance.