--- name: containers-storage-cos-install-addon title: Installing the IBM Cloud Object Storage cluster add-on description: You can enable the IBM Cloud Object Storage add-on from the IBM Cloud console or the CLI. last-updated: 2026-08-26 --- > ## Documentation Index > The table of contents for this documentation set is at https://cloud.ibm.com/docs/containers?format=markdown > The index for all IBM Cloud docs is at: https://cloud.ibm.com/docs/llms.txt > Use these files to discover more information as needed. # Installing the IBM Cloud Object Storage cluster add-on {: #storage-cos-install-addon} You can enable the IBM Cloud Object Storage add-on from the IBM Cloud console or the CLI. {: shortdesc} Prerequisites: - The IBM Cloud Object Storage add-on requires at least 0.3 vCPU and 360 MB of memory. - The add-on is available for Red Hat CoreOS (RHCOS) and Ubuntu worker nodes. If your cluster has both RHEL and RHCOS nodes, then the add-on is deployed only on the RHCOS nodes. - Set up an [IBM Cloud Object Storage instance](https://cloud.ibm.com/docs/containers?topic=containers-storage-cos-understand&format=markdown#create_cos_service). - **Optional** If you plan to use bucket versioning, your service credentials must have **Manager** or **Writer** permissions to enable or disable bucket versioning on the bucket. For more information, see [Getting started with versioning](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-versioning&format=markdown#versioning-getting-started). ## Understanding bucket creation and removal {: #cos-addon-bucket-cd} - You can use an existing bucket by specifying the bucket name in your PVC. - If you provide a bucket name and that bucket doesn't exist, then a bucket with that name is created. - If you don't provide a bucket name, then a bucket with the naming convention `s3fs--xxx` or `rclone--xxx` is created, based on the mounter type. - Buckets are deleted based on reclaim policy defined in your storage class. - If `reclaimPolicy: Delete` is set, the bucket is deleted when the PVC is deleted. - If `reclaimPolicy: Retain` is set, the bucket is retained even after the PVC is deleted. ## Enabling the IBM Cloud Object Storage add-on from the console {: #enable-cos-addon-console} {: ui} 1. From the [IBM Cloud Kubernetes Service cluster dashboard](https://cloud.ibm.com/containers/cluster-management/clusters){: external}, select the cluster where you want to enable the add-on. 1. In the **Add-ons** section, locate the **Cloud Object Storage** add-on and click **Install**. 1. In the **Install add-on: Cloud Object Storage** panel, select a version from the **Version** drop-down. 1. Optional: Configure the following parameters. `maxVolumesPerNode` : Set the maximum number of IBM Cloud Object Storage volumes that can be mounted on a single node. The default value is `0`, which means no limit is applied. `restrictNodeServerScheduling` : Set to `true` to restrict the nodeserver pods to run only on nodes that are labeled `cos.csi.ibm.io/csi-node=true`. The default value is `false`, which means nodeserver pods are scheduled on all nodes. 1. Click **Install**. The add-on might take several minutes to deploy and become ready for use. 1. Verify the installation. In the **Add-ons** section, confirm that the **Cloud Object Storage** add-on shows a **Normal** health state. ## Enabling the IBM Cloud Object Storage add-on from the CLI {: #enable-cos-addon} {: cli} Before you begin: [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster&format=markdown) 1. Update the `container-service` plug-in to the most recent version. ```sh ibmcloud update && ibmcloud plugin update container-service ``` {: pre} 1. List the add-ons and find the version you want to install. ```sh ibmcloud ks cluster addon versions ``` {: pre} 1. Review the add-on options. ```sh ibmcloud ks cluster addon options --addon ibm-object-csi-driver [--version VERSION] ``` {: pre} 1. Install the add-on. ```sh ibmcloud ks cluster addon enable ibm-object-csi-driver --cluster CLUSTER [--version VERSION] ``` {: pre} 1. Verify the installation. ```sh ibmcloud ks cluster addon ls --cluster CLUSTER ``` {: pre} ```sh OK Name Version Health State Health Status ibm-object-csi-driver 1.0 normal Addon Ready. For more info: http://ibm.biz/addon-state (H1500) ``` {: screen} 1. List the available storage classes. The driver supports both regional and cross-regional storage classes for the `s3fs` and `rclone` mounters. ```sh kubectl get sc | grep object ``` {: pre} ```sh ibm-object-storage-smart-cross-region-rclone cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-smart-cross-region-rclone-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-smart-cross-region-s3fs cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-smart-cross-region-s3fs-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-smart-rclone cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-smart-rclone-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-smart-s3fs cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-smart-s3fs-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-standard-cross-region-rclone cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-standard-cross-region-rclone-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-standard-cross-region-s3fs cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-standard-cross-region-s3fs-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-standard-rclone cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-standard-rclone-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-standard-s3fs cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-standard-s3fs-retain cos.s3.csi.ibm.io Retain Immediate false 17h ``` {: screen} ## Restricting nodeserver pod scheduling {: #cos-addon-restrict-node-scheduling} By default, the COS CSI driver nodeserver pods are scheduled on all nodes in the cluster. You can use the `restrictNodeServerScheduling` parameter to restrict nodeserver pod scheduling to only the nodes that are labeled with `cos.csi.ibm.io/csi-node=true`. You can configure `restrictNodeServerScheduling` when you enable the add-on, or update it later by patching the ConfigMap. - To set `restrictNodeServerScheduling` when you enable the add-on, include the `--param` flag in the enable command. ```sh ibmcloud ks cluster addon enable ibm-object-csi-driver --cluster CLUSTER --param "restrictNodeServerScheduling=true" ``` {: pre} - To update `restrictNodeServerScheduling` after the add-on is already enabled, follow these steps. 1. List the nodes in your cluster and determine where you want the COS driver pods to run. ```sh kubectl get nodes ``` {: pre} Example output ```sh NAME STATUS ROLES AGE VERSION 10.241.0.11 Ready 5d2h v1.35.5+IKS 10.241.0.12 Ready 5d2h v1.35.5+IKS 10.241.0.13 Ready 5d2h v1.35.5+IKS 10.241.128.10 Ready 5d2h v1.35.5+IKS 10.241.128.11 Ready 5d2h v1.35.5+IKS 10.241.128.9 Ready 5d2h v1.35.5+IKS 10.241.65.12 Ready 5d2h v1.35.5+IKS 10.241.65.13 Ready 5d2h v1.35.5+IKS 10.241.65.14 Ready 5d2h v1.35.5+IKS ``` {: screen} 1. Check that nodeserver pods are currently running on all nodes. ```sh kubectl get pods -n ibm-object-csi-operator -l app.kubernetes.io/component=node -o wide ``` {: pre} Example output ```sh NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ibm-object-csi-node-2pj2j 3/3 Running 0 145m 172.17.14.10 10.241.0.12 ibm-object-csi-node-7bhwh 3/3 Running 0 145m 172.17.1.72 10.241.65.12 ibm-object-csi-node-7l9hc 3/3 Running 0 145m 172.17.17.6 10.241.128.9 ibm-object-csi-node-cxzt7 3/3 Running 0 145m 172.17.39.72 10.241.0.11 ibm-object-csi-node-dw6qs 3/3 Running 0 145m 172.17.46.77 10.241.128.10 ibm-object-csi-node-rpcvr 3/3 Running 0 145m 172.17.32.198 10.241.65.13 ibm-object-csi-node-swqtg 3/3 Running 0 145m 172.17.16.69 10.241.0.13 ibm-object-csi-node-sxbbs 3/3 Running 0 145m 172.17.26.7 10.241.65.14 ibm-object-csi-node-xm8bt 3/3 Running 0 145m 172.17.20.200 10.241.128.11 ``` {: screen} 1. Label the nodes where you want nodeserver pods to be scheduled. ```sh kubectl label nodes NODE-NAME-1 NODE-NAME-2 cos.csi.ibm.io/csi-node=true ``` {: pre} Example output ```sh node/10.241.0.11 labeled node/10.241.0.12 labeled ``` {: screen} 1. Enable the restriction by updating the ConfigMap. ```sh kubectl patch cm managed-addon-ibm-object-csi-driver -n kube-system \ --type merge -p '{"data":{"restrictNodeServerScheduling":"true"}}' ``` {: pre} Example output ```sh configmap/managed-addon-ibm-object-csi-driver patched ``` {: screen} 1. Verify that nodeserver pods are only scheduled on labeled nodes. ```sh kubectl get pods -n ibm-object-csi-operator -l app.kubernetes.io/component=node -o wide ``` {: pre} Example output ```sh NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ibm-object-csi-node-cxzt7 3/3 Running 0 145m 172.17.39.72 10.241.0.11 ibm-object-csi-node-7bhwh 3/3 Running 0 145m 172.17.1.72 10.241.65.12 ``` {: screen} | Setting | Behavior | | --- | --- | | `restrictNodeServerScheduling: "false"` (default) | Nodeserver pods are scheduled on all nodes. | | `restrictNodeServerScheduling: "true"` | Nodeserver pods are scheduled only on nodes labeled `cos.csi.ibm.io/csi-node=true`. | {: caption="restrictNodeServerScheduling options" caption-side="bottom"} {: #cos-addon-restrict-node-scheduling-table} ## Setting the maximum volumes per node {: #cos-addon-max-volumes-per-node} By default, the COS CSI driver does not limit the number of volumes that can be mounted on a single node. You can use the `maxVolumesPerNode` parameter to set a maximum number of volumes per node. You can configure `maxVolumesPerNode` when you enable the add-on, or update it later by patching the ConfigMap. - To set `maxVolumesPerNode` when you enable the add-on, include the `--param` flag in the enable command. ```sh ibmcloud ks cluster addon enable ibm-object-csi-driver --cluster CLUSTER --param "maxVolumesPerNode=VALUE" ``` {: pre} - To update `maxVolumesPerNode` after the add-on is already enabled, patch the managed add-on ConfigMap. ```sh kubectl patch cm managed-addon-ibm-object-csi-driver -n kube-system --type merge -p '{"data":{"maxVolumesPerNode":"VALUE"}}' ``` {: pre} Example output ```sh configmap/managed-addon-ibm-object-csi-driver patched ``` {: screen} | Setting | Behavior | | --- | --- | | `maxVolumesPerNode: "0"` (default) | No limit on the number of volumes that can be mounted per node. | | `maxVolumesPerNode: "VALUE"` | Limits the number of volumes that can be mounted on a single node to the specified value. | {: caption="maxVolumesPerNode options" caption-side="bottom"} {: #cos-addon-max-volumes-per-node-table} ## Deploying an app that uses IBM Cloud Object Storage {: #cos-addon-app} ### Create a Kubernetes secret that contains your COS credentials. {: #cos-addon-app-secret} 1. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster&format=markdown) 1. Save the following configuration as a file called `secret.yaml`. Provide either IAM credentials or HMAC credentials, but not both. - For **IAM credentials**, use `apiKey` and `serviceId` from your IBM Cloud Object Storage service instance. - For **HMAC credentials**, use `accessKey` and `secretKey` from your IBM Cloud Object Storage service instance. ```yaml apiVersion: v1 kind: Secret type: cos-s3-csi-driver metadata: name: cos-secret-1 # Name your secret. This same name is used for the PVC in the following steps. namespace: # Specify the namespace where you want to create the secret. data: # --- IAM credentials (provide apiKey + serviceId) --- apiKey: serviceId: # --- HMAC credentials --- accessKey: secretKey: # --- Optional credential fields (base64-encoded) --- kpRootKeyCRN: resourceConfigApiKey: # Required only when quotaLimit is "true". stringData: # --- Optional config fields (plain text) --- cosEndpoint: "https://" # Overrides the cosEndpoint from the storage class. locationConstraint: "-standard" # Overrides the locationConstraint from the storage class. iamEndpoint: "" # Overrides the default iam endpoint set in COS CSI Driver objectPath: "" # Optional. Subdirectory within the bucket to mount, for example "data". bucketName: # Optional. If you don't provide a bucket name, a bucket with the naming convention s3fs--xxx or rclone--xxx is created. bucketVersioning: "false" # Set to "true" to enable bucket versioning. Set to "false" to disable versioning. Must be a string value. quotaLimit: "false" # Set to "true" to enforce a hard quota on the bucket equal to the PVC storage size. Requires resourceConfigApiKey. mountOptions: | # uid=3000 # Optional: Run as non-root user. Must match runAsUser in SecurityContext of pod spec. # Review or update the following default s3fs mount options #multipart_size=52 #multireq_max=20 #max_dirty_data=5120 #parallel_count=20 #max_stat_cache_size=100000 #retries=5 #kernel_cache #max_background=1000 # Review or update the following default rclone mount options #acl=private #bucket_acl=private #upload_cutoff=100Mi #chunk_size=16Mi #max_upload_parts=1000 #upload_concurrency=8 #multi_thread_streams=8 #disable_checksum=true ``` {: codeblock} `apiKey` : Required for IAM authentication. Enter the base64-encoded IBM Cloud IAM API key for your IBM Cloud Object Storage service instance. You can find the API key in your service credentials under `apikey`. Provide either `apiKey` + `serviceId` **or** `accessKey` + `secretKey`, but not both. `serviceId` : Required for IAM authentication. Enter the base64-encoded resource instance ID for your IBM Cloud Object Storage service instance. You can find this value in your service credentials under `resource_instance_id`. `accessKey` : Required for HMAC authentication. Enter the base64-encoded HMAC access key ID. You can find this value in your service credentials under `cos_hmac_keys.access_key_id`. Provide either `accessKey` + `secretKey` **or** `apiKey` + `serviceId`, but not both. `secretKey` : Required for HMAC authentication. Enter the base64-encoded HMAC secret access key. You can find this value in your service credentials under `cos_hmac_keys.secret_access_key`. `kpRootKeyCRN` : Optional. Enter the base64-encoded root key CRN from your Key Protect instance. To retrieve the CRN, go to your KMS instance in the [IBM Cloud console](https://cloud.ibm.com){: external}, open **Keys**, click the root key, and copy the **CRN** from the key details. Applies to new buckets only; you cannot add encryption to an existing bucket. `iamEndpoint` : Optional. Enter the IBM Cloud IAM token endpoint URL as plain text. By default, the driver uses `https://private.iam.cloud.ibm.com` for VPC clusters and `https://iam.cloud.ibm.com` for Classic clusters. Override this value only if you need to use a different IAM endpoint. `cosEndpoint` : Optional. Enter the IBM Cloud Object Storage endpoint URL as plain text, for example `https://s3.us.cloud-object-storage.appdomain.cloud`. When provided, this value overrides the `cosEndpoint` set in the storage class. Use this field if your bucket is in a different region or uses a direct or private endpoint. For a list of available endpoints, see [IBM Cloud Object Storage endpoints](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpoints&format=markdown). `locationConstraint` : Optional. Enter the location constraint string as plain text, for example `us-standard` or `us-geo-smart`. When provided, this value overrides the `locationConstraint` set in the storage class. The location constraint determines the bucket class and the region where the bucket is stored. `objectPath` : Optional. Enter the path to a subdirectory within the bucket to mount as plain text, for example `data`. Use this option to give an app access to only a specific folder within a shared bucket rather than the entire bucket root. `resourceConfigApiKey` : Required when `quotaLimit` is set to `"true"`. Enter the same base64-encoded `apikey` value from your IBM Cloud Object Storage service credentials that you used for the `apiKey` field above. `bucketName` : Optional. Enter the name of an existing bucket to use, or a name for a new bucket to create. If the bucket name you provide doesn't exist, the driver creates it. If you leave this field empty, a bucket is automatically created with the naming convention `s3fs--xxx` or `rclone--xxx` based on the mounter type. The bucket name must be globally unique in IBM Cloud Object Storage. `bucketVersioning` : Optional. Controls bucket versioning. Set to `"true"` to enable versioning, or `"false"` to disable versioning on a bucket where versioning is already enabled. Must be a string value. When versioning is enabled, IBM Cloud Object Storage retains multiple versions of every object in the bucket, protecting against accidental deletion and overwrites. Note that the service credentials must have **Manager** or **Writer** permissions to enable or disable bucket versioning. For more information, see [Getting started with versioning](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-versioning&format=markdown#versioning-getting-started). `quotaLimit` : Optional. Set to `"true"` to enforce a hard storage quota on the bucket. When enabled, the bucket quota is set equal to the `storage` size requested in the PVC. If the quota is reached, write operations to the bucket fail until data is deleted. Requires `resourceConfigApiKey` to be set. Defaults to `"false"`. Must be a string value. `mountOptions` : You can customize the mount options for either `s3fs` or `rclone` by editing the `mountOptions` in your secret. To run as a non-root user, uncomment and set `uid=` to match the `runAsUser` field in the `securityContext` of your pod spec. Align the options that you specify with the storage class that your PVC uses. To review the default values for a storage class, run `oc describe storageclass ` or `kubectl describe storageclass `. For more information, see the [s3fs mount options](https://github.com/IBM/ibm-object-csi-driver/blob/main/cos-csi-mounter/server/s3fs.go){: external} and the [`rclone` mount options](https://github.com/IBM/ibm-object-csi-driver/blob/main/cos-csi-mounter/server/rclone.go){: external}. Currently, the add-on is enabled to support a fixed set of mount options with proper validation for each mount option. If you want to use any other mount options that are not in the validation list, contact support to enable those options. {: note} 1. Encode all the secret data parameters to base64. ```sh echo -n "" | base64 ``` {: pre} 1. Update the `secret.yaml` with the base64 encoded values. 1. Create the secret. ```sh kubectl apply -f secret.yaml ``` {: pre} ### Create a PVC {: #cos-addon-app-pvc} You can either use a single secret across multiple PVCs or one secret per PVC. You can manage this behavior by using the following annotations in the PVC yaml. These annotations help the driver map the PVC to the correct secret. ```yaml cos.csi.driver/secret: "" ``` {: codeblock} Make sure that your secret, PVC, and pods are all in the same namespace {: note} Example PVC for a 1-to-1 secret to PVC mapping by giving your PVC the same name as the secret you created earlier. ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cos-secret-1 # Give your PVC the same name as the secret you created in the previous step. namespace: # The namespace where you want to create the PVC. spec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi storageClassName: # The storage class you want to use. ``` {: codeblock} Example PVC for using 1 secret to many PVCs by using annotations to specify the secret. ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cos-csi-pvc1 namespace: # The namespace where you want to create the PVC. annotations: cos.csi.driver/secret: "" spec: accessModes: - ReadWriteOnce resources: requests: storage: 256Mi storageClassName: # The storage class you want to use. ``` {: codeblock} 1. Choose one of the previous examples and customize it for your use case. For a list of storage classes, see the [Storage class reference](#cos-sc-ref-addon). 1. Create the PVC. ```sh kubectl apply -f pvc.yaml ``` {: pre} ### Create a deployment {: #cos-addon-app-dep} 1. Save the following configuration to a file called `dep.yaml`. ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: labels: app: spec: replicas: 1 selector: matchLabels: app: template: metadata: labels: app: spec: containers: - name: app-frontend image: # Enter your app image. imagePullPolicy: IfNotPresent volumeMounts: - mountPath: # For example `/dev` name: cos-csi-volume volumes: - name: cos-csi-volume persistentVolumeClaim: claimName: # Enter the name of the PVC you created earlier. ``` {: codeblock} 1. Create the deployment. ```sh kubectl apply -f dep.yaml ``` {: pre} ## Disabling the IBM Cloud Object Storage add-on {: #disable-cos-addon} The existing secrets, PVCs, and deployments are not deleted by disabling the add-on or by patch updates. There are no disruptions to existing customer workloads. {: note} 1. Run the following command to disable the add-on. ```sh ibmcloud ks cluster addon disable ibm-object-csi-driver --cluster CLUSTER ``` {: pre} Example output ```sh Data and resources that you created for the add-on might be deleted when the add-on is disabled. Continue? [y/N]> y Disabling add-on ibm-object-csi-driver for cluster XXX... OK ``` {: screen} 1. Verify the add-on was removed. ```sh ibmcloud ks cluster addon ls --cluster CLUSTER ``` {: pre} ## Migrating from the Helm plug-in to the cluster add-on {: #cos-addon-migrate-helm} 1. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster&format=markdown) 1. Get the details of your PVCs and select one to migrate. ```sh kubectl get pvc --all-namespaces -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name' | tail -n +2 | while read namespace pvc; do kubectl describe pvc "$pvc" -n "$namespace" | grep 'volume.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs' > /dev/null ; if [ $? -eq 0 ]; then echo "PVC: $pvc in Namespace: $namespace uses ibm.io/ibmc-s3fs storage provisioner"; fi; done ``` {: pre} Example output ```txt PVC: pvc-test in Namespace: default uses ibm.io/ibmc-s3fs storage provisioner ``` {: screen} 1. Describe the PVC and get the bucket name. ```sh kubectl describe pvc | grep ibm.io/bucket: ``` {: pre} Example output ```txt ibm.io/bucket: test-s3 ``` {: screen} 1. Re-create your secret with the bucket name included. ```yaml apiVersion: v1 kind: Secret type: cos-s3-csi-driver metadata: name: cos-secret-1 # Name your secret. namespace: # Specify the namespace where you want to create the secret. data: accessKey: secretKey: stringData: bucketName: mountOptions: | # uid=3000 # Optional: Run as non-root user. Must match runAsUser in SecurityContext of pod spec. key1=value1 key2=value2 ``` {: codeblock} 1. Find the storage class that was used in your PVC. ```sh kubectl describe pvc | grep StorageClass: ``` {: pre} Example command for a PVC called `test-s3`. ```sh kubectl describe pvc test-s3 | grep StorageClass: ``` {: pre} Example output ```txt StorageClass: ibmc-s3fs-smart-perf-regional ``` {: screen} 1. [Review the new storage classes](#cos-sc-ref-addon) that are available with the add-on and select a replacement class. * If you used a `flex` class, choose one of the new `smart` classes. * If you used a `standard` classes, choose one of the new `standard` classes. * The `cold` and `vault` classes are no longer available with the add-on; choose a `smart` or `standard` class instead. 1. Review the details of your PVC. ```sh kubectl describe pvc test-s3 ``` {: pre} Example output ```txt Name: pvc-test Namespace: default StorageClass: ibmc-s3fs-smart-perf-regional Status: Bound Volume: pvc-c625474d-31f0-4929-bc3e-feace1fb42fb Labels: Annotations: ibm.io/auto-create-bucket: true ibm.io/auto-delete-bucket: true ibm.io/bucket: bha-test-s23 ibm.io/secret-name: satstoragesecret pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs Finalizers: [kubernetes.io/pvc-protection] Capacity: 3Gi Access Modes: RWO VolumeMode: Filesystem Used By: test-pod Events: ``` {: screen} 1. Create a replacement PVC that uses a new storage class and references the secret you created earlier. ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cos-csi-pvc1 namespace: # The namespace where you want to create the PVC. annotations: cos.csi.driver/secret: "cos-secret-1" # Secret created in step 4 spec: accessModes: - ReadWriteOnce resources: requests: storage: 256Mi storageClassName: # The storage class you picked based on old storage class mapping. ``` {: codeblock} 1. Verify the PVC is `Bound`. ```sh kubectl get pvc ``` {: pre} 1. Get the details of your app. ```sh kubectl get pods ``` {: pre} 1. Scale down your app to zero. ```sh kubectl scale deployment --replicas=0 my-app ``` {: pre} 1. Create a replacement deployment that references the PVC you created in the previous step. 1. After the new deployment is running, you can delete the old deployment. 1. Repeat these steps for each PVC that you want to migrate. ## IBM Cloud Object Storage cluster add-on storage classes {: #cos-sc-ref-addon} The IBM Cloud Object Storage cluster add-on provides storage classes for the `s3fs` and `rclone` mounters. Choose a storage class that fits your data access requirements. The storage class determines the bucket class, reclaim policy, and default mount behavior for the bucket that is created for your workload. Standard : Use for hot data that is accessed frequently. Common use cases are web or mobile apps. Vault : Use for workloads or cool data that are accessed infrequently, such as once a month or less. Common use cases are archives, short-term data retention, digital asset preservation, tape replacement, and disaster recovery. Cold : Use for cold data that is rarely accessed (every 90 days or less), or inactive data. Common use cases are archives, long-term backups, historical data that you keep for compliance, or workloads and apps that are rarely accessed. Smart : Use for workloads and data that do not follow a specific usage pattern, or when the usage pattern is difficult to predict. Decide on the level of resiliency for the data that is stored in your bucket. For more information, see [Regions and endpoints](https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpoints&format=markdown). Cross-region : Your data is stored across three regions within a geolocation for highest availability. If you have workloads that are distributed across regions, requests are routed to the nearest regional endpoint. The IBM Cloud Object Storage endpoint for the geolocation is automatically set based on the location that your cluster is in. For example, if your cluster is in `US South`, then your storage classes are configured to use the `US GEO` endpoint for your buckets. Choose a storage class with `cross-region` in its name. Regional : Your data is replicated across multiple zones within one region. If you have workloads that are located in the same region, you see lower latency and better performance than in a cross-regional setup. The regional endpoint is automatically set based on the location that your cluster is in. For example, if your cluster is in `US South`, then your storage classes are configured to use `US South` as the regional endpoint for your buckets. Choose a storage class without `cross-region` in its name. | Name | Bucket class | Resiliency | Mounter | Reclaim policy | Binding mode | | --- | --- | --- | --- | --- | --- | | ibm-object-storage-smart-cross-region-rclone | Smart | Cross-regional | `rclone` | Delete | Immediate | | ibm-object-storage-smart-cross-region-rclone-retain | Smart | Cross-regional | `rclone` | Retain | Immediate | | ibm-object-storage-smart-cross-region-s3fs | Smart | Cross-regional | `s3fs` | Delete | Immediate | | ibm-object-storage-smart-cross-region-s3fs-retain | Smart | Cross-regional | `s3fs` | Retain | Immediate | | ibm-object-storage-smart-rclone | Smart | Regional | `rclone` | Delete | Immediate | | ibm-object-storage-smart-rclone-retain | Smart | Regional | `rclone` | Retain | Immediate | | ibm-object-storage-smart-s3fs | Smart | Regional | `s3fs` | Delete | Immediate | | ibm-object-storage-smart-s3fs-retain | Smart | Regional | `s3fs` | Retain | Immediate | | ibm-object-storage-standard-cross-region-rclone | Standard | Cross-regional | `rclone` | Delete | Immediate | | ibm-object-storage-standard-cross-region-rclone-retain | Standard | Cross-regional | `rclone` | Retain | Immediate | | ibm-object-storage-standard-cross-region-s3fs | Standard | Cross-regional | `s3fs` | Delete | Immediate | | ibm-object-storage-standard-cross-region-s3fs-retain | Standard | Cross-regional | `s3fs` | Retain | Immediate | | ibm-object-storage-standard-rclone | Standard | Regional | `rclone` | Delete | Immediate | | ibm-object-storage-standard-rclone-retain | Standard | Regional | `rclone` | Retain | Immediate | | ibm-object-storage-standard-s3fs | Standard | Regional | `s3fs` | Delete | Immediate | | ibm-object-storage-standard-s3fs-retain | Standard | Regional | `s3fs` | Retain | Immediate | {: caption="COS cluster add-on storage classes" caption-side="bottom"} To review the detailed bucket configuration for a storage class, run `oc describe storageclass ` or `kubectl describe storageclass `. ### Storage class parameters {: #cos-sc-ref-addon-params} All cluster add-on storage classes include the following core parameters. | Parameter | Description | | --- | --- | | `client` | Identifies the client type that the driver uses. The add-on storage classes use `awss3`. | | `cosEndpoint` | Defines the IBM Cloud Object Storage endpoint for the bucket region. | | `csi.storage.k8s.io/node-publish-secret-name` | References the name of the secret that contains your IBM Cloud Object Storage credentials. | | `csi.storage.k8s.io/node-publish-secret-namespace` | References the namespace of the secret that contains your IBM Cloud Object Storage credentials. | | `locationConstraint` | Defines the bucket class and region, such as `au-syd-smart` or `au-syd-standard`. | | `mounter` | Specifies whether the storage class uses the `s3fs` or `rclone` mounter. | {: caption="Core parameters for COS cluster add-on storage classes" caption-side="bottom"} ### Default `s3fs` storage class mount options {: #cos-sc-ref-addon-s3fs} The `s3fs` storage classes use the following default mount options. | Mount option | Description | | --- | --- | | `multipart_size=52` | Sets the part size, in MB, for each multipart request. | | `multireq_max=20` | Sets the maximum number of parallel requests for listing objects. | | `max_dirty_data=5120` | Flushes dirty data to S3 after a specified number of MB are written. The minimum supported value is `50`. A value of `-1` disables this behavior. | | `parallel_count=20` | Sets the number of parallel requests for uploading large objects. `s3fs` uploads large objects by using multipart requests and sends requests in parallel. | | `max_stat_cache_size=100000` | Sets the maximum number of entries in the stat cache and symbolic link cache. | | `retries=5` | Sets the number of times to retry a failed S3 transaction. | | `kernel_cache` | Enables the kernel buffer cache for the volume mount point. Data that is read from IBM Cloud Object Storage is stored in the kernel cache to help provide faster read access. Kernel cache is enabled for the standard and smart `s3fs` storage classes. | | `max_background=1000` | Sets the maximum number of background FUSE requests that can be queued before the kernel blocks new requests. Increasing this value improves throughput for workloads with high concurrency. | {: caption="Default mount options for COS add-on s3fs storage classes" caption-side="bottom"} ### Default `rclone` storage class mount options {: #cos-sc-ref-addon-rclone} The `rclone` storage classes use the following default mount options. | Mount option | Description | | --- | --- | | `acl=private` | Ensures that uploaded objects are not publicly accessible. | | `bucket_acl=private` | Sets the default ACL for buckets that `rclone` creates to `private`. | | `upload_cutoff=100Mi` | Uploads files larger than `100 MiB` by using multipart upload. Smaller files are uploaded in a single request. | | `chunk_size=16Mi` | Sets the size of each part in a multipart upload. | | `max_upload_parts=1000` | Sets the maximum number of parts per multipart upload and indirectly caps the maximum supported file size with the configured `chunk_size`. With `chunk_size=16Mi`, the maximum file size is `16 GiB`. | | `upload_concurrency=8` | Sets the number of parts that are uploaded in parallel during a multipart upload. | | `multi_thread_streams=8` | Sets the number of threads used to download a single object when using multi-thread downloading. | | `disable_checksum=true` | Disables MD5 checksum calculation on upload. Improves performance for large files where checksum computation adds significant overhead. | {: caption="Default mount options for COS add-on rclone storage classes" caption-side="bottom"}