Why migrate to the CSI driver?

The legacy FlexVolume driver is tentatively planned to be unsupported by the end of 2027. Plan to migrate your workloads to the CSI driver before support ends.

Kubernetes has deprecated the FlexVolume specification across the industry in favor of the Container Storage Interface (CSI) standard. Migrating your storage resources to the IBM Cloud Object Storage CSI driver provides the following benefits:

• Managed add-on lifecycle: The CSI driver is managed as a cluster add-on with automated patch updates and health management. The add-on installs and updates the s3fs or rclone mounter and the cos-csi-mounter service on your worker nodes, so you don't install or maintain these host components manually.
• Standard CSI architecture: The controller server and node server run as containerized pods and follow the standard CSI model. The node server forwards mount and unmount requests from the kubelet to the cos-csi-mounter systemd service on the worker node over a Unix socket.
• Choice of mounters: You can choose the s3fs or rclone mounter, with its own mount options, to match the access pattern and performance needs of each workload.

Differences between FlexVolume and CSI

Review the key architectural and operational differences between the legacy FlexVolume driver and the modern CSI driver.

How the migration process works

The cos-flex-to-csi tool provides a safe, non-destructive migration path:

1. Discovery: Scans your cluster to identify all FlexVolume PVCs and their associated secrets, and checks whether backing PVs have auto-delete bucket protection enabled.
2. Dry run: Previews the exact resource definitions and names that the tool generates before applying any changes to your cluster.
3. Resource creation: Creates a new CSI-compatible Secret and PVC (suffixed with -csi-migrated) in the same namespace as the original FlexVolume PVC. The original FlexVolume resources are not changed. Both resources point to the same underlying Object Storage bucket without modifying bucket contents.
4. Verification and cutover: Your original FlexVolume PVC remains active until you verify that the new CSI PVC is bound, update your application deployments to reference the new PVC, and manually decommission the legacy resources.

Before you begin

Before you install and run the migration tool, confirm that your environment meets the following requirements.

• Your cluster is a VPC Gen 2 cluster running IBM Cloud Kubernetes Service or Red Hat OpenShift on IBM Cloud. Classic infrastructure clusters are not supported because the IBM Cloud Object Storage CSI driver add-on is available only on VPC clusters.
• At least one worker node in your cluster runs Ubuntu or Red Hat CoreOS (RHCOS).
• The IBM Cloud Object Storage CSI driver add-on is installed in your cluster. To install it, see Installing the IBM Cloud Object Storage cluster add-on.
• Your KUBECONFIG environment variable is set and you have kubectl (or oc) access to your cluster.
• The cos-flex-to-csi tool is run from a Linux (amd64 or arm64) or macOS (amd64 or arm64) machine with outbound internet access to GitHub. If your environment does not have internet access, use the --no-upgrade flag on the migrate command to skip the automatic self-upgrade check during migration.
• The identity used to run the tool has the required RBAC permissions.

Required permissions

The cos-flex-to-csi tool uses your active KUBECONFIG credentials to read and create resources in your cluster. The identity must have the following RBAC permissions before you run the tool.

Installing the migration tool

The cos-flex-to-csi binary is distributed as a standalone executable from the ibmcloud-object-storage-plugin GitHub repository. Download and install the latest release for your operating system and architecture.

The tool is available for the following platforms. Archive names follow the pattern cos-flex-to-csi-v<version>-<os>-<arch>.tar.gz.

The install script in the following steps detects your operating system and architecture automatically.

1. Save the following script to a file named install-cos-flex-to-csi.sh.

   #!/bin/bash
   HOST_UNAME="$(uname -s)"
   case "$HOST_UNAME" in
     Linux)  HOST_OS="linux"  ;;
     Darwin) HOST_OS="darwin" ;;
     *)      echo "Unsupported OS: $HOST_UNAME"; exit 1 ;;
   esac
   
   HOST_MACHINE="$(uname -m)"
   case "$HOST_MACHINE" in
     x86_64)          HOST_ARCH="amd64" ;;
     arm64|aarch64)   HOST_ARCH="arm64" ;;
     *)               echo "Unsupported architecture: $HOST_MACHINE"; exit 1 ;;
   esac
   
   GITHUB_REPO="IBM/ibmcloud-object-storage-plugin"
   COS_FLEX_TO_CSI_ARCHIVE="$(curl -fsSL \
     "https://api.github.com/repos/${GITHUB_REPO}/releases/latest" \
     | grep -o "cos-flex-to-csi-v[^\"[:space:]]*-${HOST_OS}-${HOST_ARCH}\.tar\.gz" \
     | grep -v '\.sha256$' \
     | head -n 1)"
   
   curl -fL -O "https://github.com/${GITHUB_REPO}/releases/latest/download/${COS_FLEX_TO_CSI_ARCHIVE}"
   tar -xzf "$COS_FLEX_TO_CSI_ARCHIVE"
   

2. Make the script executable and run it to download and extract the latest release.

   chmod +x install-cos-flex-to-csi.sh && ./install-cos-flex-to-csi.sh
   

3. Verify the installation to confirm the binary is ready to use.

   ./cos-flex-to-csi version
   

   The output shows the current version of the tool, confirming the installation was successful.

Discovering FlexVolume PVCs

Before migrating, scan your cluster to identify all FlexVolume-based PVCs and their associated secrets. The output shows the current migration status of each PVC so you know exactly what needs to be migrated.

1. Run the discover command against your target namespace or across all namespaces.

   • To scan a specific namespace, run the following command.

     ./cos-flex-to-csi discover -n NAMESPACE
     

   • To scan all namespaces in the cluster, run the following command.

     ./cos-flex-to-csi discover --all-namespaces
     

2. Review the discovery output table.

   The output lists each FlexVolume-based PVC, whether its backing PV has the ibm.io/auto-delete-bucket annotation enabled, its associated secret, migration status, and the names of the migrated CSI resources if migration has already been completed. PVC and secret names are shown in the NAMESPACE/NAME format.

   5 COS FlexVolume PVC(s) found
   
   ┌────┬───────────────────┬────────────────────────────────┬──────────────────────┬────────────────┬───────────────────────────┬────────────────────────────────────┐
   │ #  │ COS FLEX PVC NAME │ HAS AUTO-DELETE-BUCKET ENABLED │ COS FLEX SECRET NAME │ MIGRATION DONE │ COS CSI PVC NAME          │ COS CSI SECRET NAME                │
   ├────┼───────────────────┼────────────────────────────────┼──────────────────────┼────────────────┼───────────────────────────┼────────────────────────────────────┤
   │ 1  │ default/pvc-1     │ NO                             │ default/secret-1     │ NO             │ -                         │ -                                  │
   │ 2  │ prod/pvc-2        │ NO                             │ prod/secret-2        │ YES            │ prod/pvc-2-csi-migrated   │ prod/secret-2-pvc-2-csi-migrated   │
   │ 3  │ staging/pvc-3     │ ⚠ YES                          │ staging/secret-3     │ NO             │ -                         │ -                                  │
   │ 4  │ dev/pvc-4         │ NO                             │ shared/secret-4      │ YES            │ dev/pvc-4-csi-migrated    │ dev/secret-4-pvc-4-csi-migrated    │
   │ 5  │ team-a/pvc-5      │ NO                             │ team-a/secret-5      │ ⚠ FAILED       │ team-a/pvc-5-csi-migrated │ team-a/secret-5-pvc-5-csi-migrated │
   └────┴───────────────────┴────────────────────────────────┴──────────────────────┴────────────────┴───────────────────────────┴────────────────────────────────────┘
   
   Summary: 2 pending, 1 failed, 2 migrated — 5 total PVCs
   

   The FlexVolume driver allows a PVC and its secret to be in different namespaces. The CSI driver requires a PVC and its secret to be in the same namespace. For this reason, the tool creates the CSI secret in the same namespace as the PVC. In the previous example, the FlexVolume PVC dev/pvc-4 uses the secret shared/secret-4, so the CSI secret is created as dev/secret-4-pvc-4-csi-migrated. The original secret in the shared namespace is not changed.

   The MIGRATION DONE column shows the migration status of each PVC.

   Values of the MIGRATION DONE column

   Value	Meaning
   NO	The CSI PVC does not exist yet. The PVC still needs to be migrated.
   YES	The CSI PVC exists and is in the Bound state.
   ⚠ FAILED	The CSI PVC exists but is not in the Bound state, for example because the migrate command timed out while it waited for the PVC to bind.

   If any migrations failed, discover prints a Failed migration(s) detected: CSI PVC exists but is not Bound message after the table. To resolve the failure, see Resolving PVCs that fail to bind.

   The HAS AUTO-DELETE-BUCKET ENABLED column shows whether the ibm.io/auto-delete-bucket: true annotation is set on the backing PV of the FlexVolume PVC.

   Values of the HAS AUTO-DELETE-BUCKET ENABLED column

   Value	Meaning
   NO	The annotation is not set to true. The PVC can be migrated.
   ⚠ YES	The annotation is set to true. Migration is blocked until you patch the annotation to false.

   If any PVCs are blocked, discover prints a ⚠ MIGRATION BLOCKED — auto-delete-bucket: true detected on N PVC(s) warning after the table. The warning includes a kubectl patch command for each affected backing PV that sets the annotation to false, for example:

   kubectl patch pv PV_NAME -p '{"metadata":{"annotations":{"ibm.io/auto-delete-bucket":"false"}}}'
   

   After you apply the patches, run discover again to confirm, and then run migrate. For more information, see Resolving blocked PVCs.

3. Identify the PVCs that require migration.

   The tool determines migration status at run time. It inspects the presence, the annotation (app.kubernetes.io/created-by=cos-flex-to-csi-migration-tool), and the phase of the expected CSI PVC in the cluster. A PVC shows YES in the MIGRATION DONE column only when all of these checks pass. A PVC shows ⚠ FAILED when the CSI PVC exists but is not in the Bound state.

   Review the list and note which PVCs show NO or ⚠ FAILED in the MIGRATION DONE column and whether any show ⚠ YES in the HAS AUTO-DELETE-BUCKET ENABLED column before continuing.

Previewing the migration with a dry run

Before applying changes to your cluster, use the --dry-run flag to preview exactly what resources the tool creates. A dry run does not modify any cluster resources.

1. Run the migrate command with the --dry-run flag. Replace NAMESPACE and PVC_NAME with your values.

   ./cos-flex-to-csi migrate -n NAMESPACE --pvc PVC_NAME --dry-run
   

2. Review the output to verify that the generated resource names match your expectations before running the live migration. Pay particular attention to the naming convention described in Understanding migrated resource names.

Migrating your PVCs

After reviewing the dry-run output, run the migrate command for each PVC you want to migrate. The tool creates a new CSI secret and PVC alongside the existing FlexVolume resources without removing the originals.

At the start of every migrate run, the tool automatically checks the GitHub Releases API for a newer release and upgrades itself if one is available. If your environment does not have internet access to GitHub, include the --no-upgrade flag on the migrate command. For more information, see Self-upgrade.

1. Run the migrate command to create the CSI resources.

   • Migrate a PVC named my-pvc in the production namespace.

     ./cos-flex-to-csi migrate -n production --pvc my-pvc
     

   • Migrate a PVC in an air-gapped environment, skipping the self-upgrade check.

     ./cos-flex-to-csi migrate -n production --pvc my-pvc --no-upgrade
     

2. Repeat this step for each PVC that shows NO in the MIGRATION DONE column from the discovery step. If a PVC is blocked due to an auto-delete bucket annotation, see Resolving blocked PVCs.

   The migrate command waits up to 30 seconds for the new CSI PVC to reach the Bound state. If the PVC does not bind in that time, the command fails with a CSI PVC did not reach Bound state within the timeout period error, and discover shows ⚠ FAILED for the PVC. For more information, see Resolving PVCs that fail to bind.

Verifying the migration and updating workloads

After migration, confirm that the new CSI PVC is bound before updating your workloads to use it.

1. Check that the migrated PVC is bound. The migration is successful when the STATUS column shows Bound for the new CSI PVC, which is named ORIGINAL_PVC_NAME-csi-migrated.

   kubectl get pvc -n NAMESPACE
   

   Look for the -csi-migrated PVC in the output and confirm its STATUS is Bound. If the STATUS is not Bound, see Resolving PVCs that fail to bind.

2. Run discover again to confirm that MIGRATION DONE now shows YES for the migrated PVC, and that the COS CSI PVC NAME and COS CSI SECRET NAME columns are populated.

   ./cos-flex-to-csi discover -n NAMESPACE
   

3. Update your workload deployment YAML files to reference the new CSI PVC name (ORIGINAL_PVC_NAME-csi-migrated). For examples of deployment configurations, see Deploying an app that uses IBM Cloud Object Storage.

4. Apply the updated deployment to your cluster.

   kubectl apply -f deployment.yaml
   

5. Verify that your application pods restart and mount the volume successfully.

6. Manually remove the original FlexVolume-based PVC and secret from your cluster when you are ready. Cleanup of the original resources is not automated in the current version of the tool. The original secret might be in a different namespace than the PVC and might be used by other PVCs. Check that no other PVC still uses the secret before you delete it.

Understanding migrated resource names

All resources created by the migration tool follow a consistent naming pattern so that the original FlexVolume resources and the new CSI resources can coexist safely during transition.

The following naming rules are used by the tool.

• PVC: Each migrated PVC always gets the suffix -csi-migrated.
• Secret: The migrated secret name is built as <original-secret>-<pvc-name>-csi-migrated.
• Namespace: The migrated PVC and the migrated secret are always created in the same namespace as the original FlexVolume PVC, even when the original secret is in a different namespace. The CSI driver requires a PVC and its secret to be in the same namespace.

This secret naming rule ensures that every migrated PVC gets its own CSI secret, even when the original FlexVolume configuration reused a single secret across multiple PVCs.

CLI reference

The following table lists the top-level flag of the cos-flex-to-csi tool.

Self-upgrade

The tool can upgrade itself to the latest release. The self-upgrade mechanism downloads the new binary from GitHub over HTTPS and automatically verifies its integrity against the SHA-256 checksum archive (*.sha256) that is distributed alongside each release. You do not need to verify the checksum manually.

The behavior depends on how you run the tool.

The --no-upgrade flag applies only to the migrate command. It has no effect on ./cos-flex-to-csi --upgrade. In air-gapped environments, use --no-upgrade with migrate, and manually download a specific verified release when you need to upgrade.

Troubleshooting

Review the following solutions for issues encountered when discovering or migrating resources with the cos-flex-to-csi tool.

❌ PV PV_NAME has annotation ibm.io/auto-delete-bucket: true

Migration cannot proceed: the bucket would be auto-deleted when the Flex PVC is removed.
To fix this, patch the annotation to false on the PV and then re-run:

  kubectl patch pv PV_NAME -p '{"metadata":{"annotations":{"ibm.io/auto-delete-bucket":"false"}}}'

Creating CSI PVC...
✅ CSI PVC created: PVC_NAME-csi-migrated
  ⏳ PVC status: Pending (waiting...)

❌ CSI PVC did not reach Bound state within the timeout period.

To investigate:
  kubectl get pvc PVC_NAME-csi-migrated -n NAMESPACE
  kubectl describe pvc PVC_NAME-csi-migrated -n NAMESPACE

Limitations

The following scenarios are not supported in the current version of the migration tool.

• Classic infrastructure clusters are not supported. The IBM Cloud Object Storage CSI driver add-on is available only on VPC Gen 2 clusters, so migration is not possible on Classic clusters.
• Cleanup of original FlexVolume resources is not automated. You must manually remove the original PVCs and secrets after verifying the migration and updating your workloads. Because an original secret can be in a different namespace than the PVC and can be used by other PVCs, check that no other PVC still uses a secret before you delete it.
• Unsupported node operating systems: At least one worker node in the cluster must run Ubuntu or Red Hat CoreOS (RHCOS). Clusters with only RHEL nodes are not supported.
• Air-gapped environments: The automatic self-upgrade feature requires outbound internet access to GitHub. Use the --no-upgrade flag on the migrate command to skip the upgrade check in air-gapped environments. Do not run ./cos-flex-to-csi --upgrade in air-gapped environments. Instead, manually download a specific verified release.
• Backing PVs with ibm.io/auto-delete-bucket: true: Migration is blocked for any FlexVolume PVC whose backing PV has the ibm.io/auto-delete-bucket: true annotation. You must manually patch the annotation to false on the PV before you can migrate. For more information, see Resolving blocked PVCs.

If your cluster configuration is not listed as supported, do not proceed with migration until support is confirmed in a future release.

Next steps

• After verifying that your migrated PVCs are bound, update your workloads to reference the new CSI PVCs.
• After updating your workloads, manually remove the original FlexVolume PVCs and secrets from your cluster. Before you delete a secret, check that no other PVC still uses it.
• Review Deploying an app that uses IBM Cloud Object Storage to explore additional PVC configuration options available with the CSI driver.
• Review the IBM Cloud Object Storage add-on parameter reference to customize your CSI driver configuration.

Related links

• Installing the IBM Cloud Object Storage cluster add-on
• Deploying an app that uses IBM Cloud Object Storage
• ibmcloud-object-storage-plugin releases