IBM Cloud Docs
OpenShift Data Foundation for remote devices

OpenShift Data Foundation for remote devices

Set up OpenShift Data Foundation for IBM Cloud Satellite® clusters. You can use Satellite storage templates to create storage configurations. When you assign a storage configuration to your clusters, the storage drivers of the selected storage provider are installed in your cluster. Be aware that charges occur when you use the OpenShift Data Foundation service. Use the Cost Estimator to generate a cost estimate based on your projected usage.

OpenShift Data Foundation is available in only internal mode, which means that your apps run in the same cluster as your storage. External mode, or storage heavy configurations, where your storage is located in a separate cluster from your apps is not supported.

Before you can deploy storage templates to clusters in your location, make sure you set up Satellite Config by selecting the Enable cluster admin access for Satellite Config option in the console or including the --enable-config-admin option when you create your cluster.

You cannot scope Satellite storage service to resource groups. However, if you are scoping other resources such as location and cluster to resource groups, you need to add Satellite reader and link administrator role for all resources in the account.

Understanding which ODF storage template to use

You have two options when deploying OpenShift Data Foundation to your Satellite clusters, which are the odf-local and odf-remote storage templates.

  • odf-local: Choose this template when you have local storage available to your worker nodes. If your storage volumes are visible when running lsblk, you can use these disks when deploying ODF if they are raw and unformatted.
  • odf-remote: Choose this template if you have a CSI driver installed in your cluster. For example, the azuredisk-csi-driver driver. You can use the CSI driver to dynamically provision storage volumes when deploying ODF.

The ODF cluster add-on is not supported on Satellite clusters. You must use either the odf-local or odf-remote storage template to deploy ODF on Satellite.

Prerequisites for using ODF for remote devices

  1. Make sure you have the following permissions.
    • Editor for the Billing service.
    • Manager and Editor for Kubernetes service.
    • Satellite Link Administrator and Reader for the Satellite service.
  2. Before you can create a storage configuration, follow the steps to set up a Satellite location.
  3. Create a Satellite cluster.
    • Make sure that you select the Enable cluster admin access for Satellite Config option when you create the cluster. If you don't enable Administrator (admin) access for Satellite Config when creating your cluster, you must re-create your cluster and enable admin access before you can deploy storage.
  4. Your cluster must have a minimum of 3 worker nodes with at least 16CPUs and 64GB RAM per worker node.
  5. Your cluster must have a remote block storage provisioner available. For example, you can deploy the AWS EBS Satellite storage template to install the EBS block drivers that you can then use to provision AWS EBS volumes for ODF.
  6. The OCP version must be compatible with the ODF template version that you want to install. For example, for version 4.8 clusters, you must use template version 4.8.
  7. Optional: Create an IBM IBM Cloud Object Storage service instance.
    1. Create HMAC credentials for your IBM Cloud Object Storage instance.
    2. Create a Kubernetes secret that uses your IBM Cloud Object Storage HMAC credentials.

Optional: Creating the IBM Cloud Object Storage service instance

If you want to use IBM Cloud Object Storage as your object service, Create an Object Storage service instance and HMAC credentials. The Object Storage instance that you create is used as the NooBaa backing store in your ODF configuration. The backing store is the underlying storage for the data in your NooBaa buckets. If you don't specify an IBM Cloud Object Storage service instance when you create your storage configuration, the default NooBaa backing store is configured. You can create more backing stores, including IBM Cloud Object Storage backing stores after assigning the configuration to to your clusters and installing ODF.

Create an instance of IBM Cloud Object Storage for the backing store of your ODF remote configuration. Then, create a set of HMAC credentials and a Kubernetes secret that uses your IBM Cloud Object Storage HMAC credentials.

  1. Create an IBM Cloud Object Storage service instance.
    ibmcloud resource service-instance-create noobaa-store cloud-object-storage standard global
    
  2. Create HMAC credentials. Make a note of your credentials.
    ibmcloud resource service-key-create cos-cred-rw Writer --instance-name noobaa-store --parameters '{"HMAC": true}'
    

Creating and assigning a configuration in the console

  1. Review the parameter reference.

  2. From the Locations console, select the location where you want to create a storage configuration.

  3. Select Storage > Create storage configuration

  4. Enter a name for your configuration.

  5. Select the Storage type.

  6. Select the Version and click Next

  7. If the Storage type that you selected accepts custom parameters, enter them on the Parameters tab.

  8. If the Storage type that you selected requires secrets, enter them on the Secrets tab.

  9. On the Storage classes tab, review the storage classes that are deployed by the configuration or create a custom storage class.

  10. On the Assign to service tab, select the service that you want to assign your configuration to.

  11. Click Complete to assign your storage configuration.

Creating a configuration in the CLI

  1. Review the parameter reference for the template version that you want to use.

  2. Log in to the IBM Cloud CLI.

    ibmcloud login
    
  3. List your Satellite locations and note the Managed from column.

    ibmcloud sat location ls
    
  4. Target the Managed from region of your Satellite location. For example, for wdc target us-east. For more information, see Satellite regions.

    ibmcloud target -r us-east
    
  5. If you use a resource group other than default, target it.

    ibmcloud target -g <resource-group>
    
  6. Copy one of the following example command for the template version that you want to use. For more information about the command, see ibmcloud sat storage config create in the command reference.

    Example command to create a version 4.12 configuration.

    ibmcloud sat storage config create --location LOCATION --name NAME --template-name odf-remote --template-version 4.12 --param "osd-size=OSD-SIZE"  --param "osd-storage-class=OSD-STORAGE-CLASS"  --param "num-of-osd=NUM-OF-OSD"  [--param "worker-nodes=WORKER-NODES"]  --param "odf-upgrade=ODF-UPGRADE"  --param "billing-type=BILLING-TYPE"  [--param "ibm-cos-endpoint=IBM-COS-ENDPOINT"]  [--param "ibm-cos-location=IBM-COS-LOCATION"]  [--param "ibm-cos-access-key=IBM-COS-ACCESS-KEY"]  [--param "ibm-cos-secret-key=IBM-COS-SECRET-KEY"]  --param "cluster-encryption=CLUSTER-ENCRYPTION"  --param "iam-api-key=IAM-API-KEY"  --param "perform-cleanup=PERFORM-CLEANUP"  --param "kms-encryption=KMS-ENCRYPTION"  [--param "kms-instance-name=KMS-INSTANCE-NAME"]  [--param "kms-instance-id=KMS-INSTANCE-ID"]  [--param "kms-base-url=KMS-BASE-URL"]  [--param "kms-token-url=KMS-TOKEN-URL"]  [--param "kms-root-key=KMS-ROOT-KEY"]  [--param "kms-api-key=KMS-API-KEY"]  --param "ignore-noobaa=IGNORE-NOOBAA" 
    

    Example command to create a version 4.13 configuration.

    ibmcloud sat storage config create --location LOCATION --name NAME --template-name odf-remote --template-version 4.13 --param "osd-size=OSD-SIZE"  --param "osd-storage-class=OSD-STORAGE-CLASS"  --param "num-of-osd=NUM-OF-OSD"  [--param "worker-nodes=WORKER-NODES"]  --param "odf-upgrade=ODF-UPGRADE"  --param "billing-type=BILLING-TYPE"  [--param "ibm-cos-endpoint=IBM-COS-ENDPOINT"]  [--param "ibm-cos-location=IBM-COS-LOCATION"]  [--param "ibm-cos-access-key=IBM-COS-ACCESS-KEY"]  [--param "ibm-cos-secret-key=IBM-COS-SECRET-KEY"]  --param "cluster-encryption=CLUSTER-ENCRYPTION"  --param "iam-api-key=IAM-API-KEY"  --param "perform-cleanup=PERFORM-CLEANUP"  --param "kms-encryption=KMS-ENCRYPTION"  [--param "kms-instance-name=KMS-INSTANCE-NAME"]  [--param "kms-instance-id=KMS-INSTANCE-ID"]  [--param "kms-base-url=KMS-BASE-URL"]  [--param "kms-token-url=KMS-TOKEN-URL"]  [--param "kms-root-key=KMS-ROOT-KEY"]  [--param "kms-api-key=KMS-API-KEY"]  --param "ignore-noobaa=IGNORE-NOOBAA"  --param "disable-noobaa-LB=DISABLE-NOOBAA-LB"  --param "encryption-intransit=ENCRYPTION-INTRANSIT" 
    

    Example command to create a version 4.14 configuration.

    ibmcloud sat storage config create --location LOCATION --name NAME --template-name odf-remote --template-version 4.14 --param "osd-size=OSD-SIZE"  --param "osd-storage-class=OSD-STORAGE-CLASS"  --param "num-of-osd=NUM-OF-OSD"  [--param "worker-pools=WORKER-POOLS"]  [--param "worker-nodes=WORKER-NODES"]  --param "odf-upgrade=ODF-UPGRADE"  --param "billing-type=BILLING-TYPE"  [--param "ibm-cos-endpoint=IBM-COS-ENDPOINT"]  [--param "ibm-cos-location=IBM-COS-LOCATION"]  [--param "ibm-cos-access-key=IBM-COS-ACCESS-KEY"]  [--param "ibm-cos-secret-key=IBM-COS-SECRET-KEY"]  --param "cluster-encryption=CLUSTER-ENCRYPTION"  --param "iam-api-key=IAM-API-KEY"  --param "perform-cleanup=PERFORM-CLEANUP"  --param "kms-encryption=KMS-ENCRYPTION"  [--param "kms-instance-name=KMS-INSTANCE-NAME"]  [--param "kms-instance-id=KMS-INSTANCE-ID"]  [--param "kms-base-url=KMS-BASE-URL"]  [--param "kms-token-url=KMS-TOKEN-URL"]  [--param "kms-root-key=KMS-ROOT-KEY"]  [--param "kms-api-key=KMS-API-KEY"]  --param "ignore-noobaa=IGNORE-NOOBAA"  --param "disable-noobaa-LB=DISABLE-NOOBAA-LB"  --param "encryption-intransit=ENCRYPTION-INTRANSIT"  --param "add-single-replica-pool=ADD-SINGLE-REPLICA-POOL"  --param "taint-nodes=TAINT-NODES"  --param "prepare-for-disaster-recovery=PREPARE-FOR-DISASTER-RECOVERY" 
    
  7. Customize the command based on the settings that you want to use.

  8. Run the command to create a configuration.

  9. Verify your configuration was created.

    ibmcloud sat storage config get --config CONFIG
    

Creating a configuration in the API

  1. Generate an API key, then request a refresh token. For more information, see Generating an IBM Cloud IAM token by using an API key.

  2. Review the parameter reference for the template version that you want to use.

  3. Copy one of the following example requests and replace the variables that you want to use.

    Example request to create a version 4.12 configuration.

    curl -X POST "https://containers.cloud.ibm.com/global/v2/storage/satellite/createStorageConfigurationByController" -H "accept: application/json" -H "Authorization: TOKEN" -H "Content-Type: application/json" -d "{ \"config-name\": \"string\", \"controller\": \"string\", \"storage-class-parameters\": [ { \"additionalProp1\": \"string\", \"additionalProp2\": \"string\", \"additionalProp3\": \"string\" } ], \"storage-template-name\": \"odf-remote\", \"storage-template-version\": \"4.12\", \"update-assignments\": true, \"user-config-parameters\": { \"entry.name\": \"OSD-SIZE\", { \"entry.name\": \"OSD-STORAGE-CLASS\", { \"entry.name\": \"NUM-OF-OSD\", { \"entry.name\": \"WORKER-NODES\", { \"entry.name\": \"ODF-UPGRADE\", { \"entry.name\": \"BILLING-TYPE\", { \"entry.name\": \"IBM-COS-ENDPOINT\", { \"entry.name\": \"IBM-COS-LOCATION\", { \"entry.name\": \"CLUSTER-ENCRYPTION\", { \"entry.name\": \"PERFORM-CLEANUP\", { \"entry.name\": \"KMS-ENCRYPTION\", { \"entry.name\": \"KMS-INSTANCE-NAME\", { \"entry.name\": \"KMS-INSTANCE-ID\", { \"entry.name\": \"KMS-BASE-URL\", { \"entry.name\": \"KMS-TOKEN-URL\", { \"entry.name\": \"IGNORE-NOOBAA\",\"user-secret-parameters\": { \"entry.name\": \"IBM-COS-ACCESS-KEY\",{ \"entry.name\": \"IBM-COS-SECRET-KEY\",{ \"entry.name\": \"IAM-API-KEY\",{ \"entry.name\": \"KMS-ROOT-KEY\",{ \"entry.name\": \"KMS-API-KEY\",}
    

    Example request to create a version 4.13 configuration.

    curl -X POST "https://containers.cloud.ibm.com/global/v2/storage/satellite/createStorageConfigurationByController" -H "accept: application/json" -H "Authorization: TOKEN" -H "Content-Type: application/json" -d "{ \"config-name\": \"string\", \"controller\": \"string\", \"storage-class-parameters\": [ { \"additionalProp1\": \"string\", \"additionalProp2\": \"string\", \"additionalProp3\": \"string\" } ], \"storage-template-name\": \"odf-remote\", \"storage-template-version\": \"4.13\", \"update-assignments\": true, \"user-config-parameters\": { \"entry.name\": \"OSD-SIZE\", { \"entry.name\": \"OSD-STORAGE-CLASS\", { \"entry.name\": \"NUM-OF-OSD\", { \"entry.name\": \"WORKER-NODES\", { \"entry.name\": \"ODF-UPGRADE\", { \"entry.name\": \"BILLING-TYPE\", { \"entry.name\": \"IBM-COS-ENDPOINT\", { \"entry.name\": \"IBM-COS-LOCATION\", { \"entry.name\": \"CLUSTER-ENCRYPTION\", { \"entry.name\": \"PERFORM-CLEANUP\", { \"entry.name\": \"KMS-ENCRYPTION\", { \"entry.name\": \"KMS-INSTANCE-NAME\", { \"entry.name\": \"KMS-INSTANCE-ID\", { \"entry.name\": \"KMS-BASE-URL\", { \"entry.name\": \"KMS-TOKEN-URL\", { \"entry.name\": \"IGNORE-NOOBAA\", { \"entry.name\": \"DISABLE-NOOBAA-LB\", { \"entry.name\": \"ENCRYPTION-INTRANSIT\",\"user-secret-parameters\": { \"entry.name\": \"IBM-COS-ACCESS-KEY\",{ \"entry.name\": \"IBM-COS-SECRET-KEY\",{ \"entry.name\": \"IAM-API-KEY\",{ \"entry.name\": \"KMS-ROOT-KEY\",{ \"entry.name\": \"KMS-API-KEY\",}
    

    Example request to create a version 4.14 configuration.

    curl -X POST "https://containers.cloud.ibm.com/global/v2/storage/satellite/createStorageConfigurationByController" -H "accept: application/json" -H "Authorization: TOKEN" -H "Content-Type: application/json" -d "{ \"config-name\": \"string\", \"controller\": \"string\", \"storage-class-parameters\": [ { \"additionalProp1\": \"string\", \"additionalProp2\": \"string\", \"additionalProp3\": \"string\" } ], \"storage-template-name\": \"odf-remote\", \"storage-template-version\": \"4.14\", \"update-assignments\": true, \"user-config-parameters\": { \"entry.name\": \"OSD-SIZE\", { \"entry.name\": \"OSD-STORAGE-CLASS\", { \"entry.name\": \"NUM-OF-OSD\", { \"entry.name\": \"WORKER-POOLS\", { \"entry.name\": \"WORKER-NODES\", { \"entry.name\": \"ODF-UPGRADE\", { \"entry.name\": \"BILLING-TYPE\", { \"entry.name\": \"IBM-COS-ENDPOINT\", { \"entry.name\": \"IBM-COS-LOCATION\", { \"entry.name\": \"CLUSTER-ENCRYPTION\", { \"entry.name\": \"PERFORM-CLEANUP\", { \"entry.name\": \"KMS-ENCRYPTION\", { \"entry.name\": \"KMS-INSTANCE-NAME\", { \"entry.name\": \"KMS-INSTANCE-ID\", { \"entry.name\": \"KMS-BASE-URL\", { \"entry.name\": \"KMS-TOKEN-URL\", { \"entry.name\": \"IGNORE-NOOBAA\", { \"entry.name\": \"DISABLE-NOOBAA-LB\", { \"entry.name\": \"ENCRYPTION-INTRANSIT\", { \"entry.name\": \"ADD-SINGLE-REPLICA-POOL\", { \"entry.name\": \"TAINT-NODES\", { \"entry.name\": \"PREPARE-FOR-DISASTER-RECOVERY\",\"user-secret-parameters\": { \"entry.name\": \"IBM-COS-ACCESS-KEY\",{ \"entry.name\": \"IBM-COS-SECRET-KEY\",{ \"entry.name\": \"IAM-API-KEY\",{ \"entry.name\": \"KMS-ROOT-KEY\",{ \"entry.name\": \"KMS-API-KEY\",}
    

Creating a storage assignment in the console

If you didn't assign your configuration to a cluster or service when you created it, you can create an assignment by completing the following steps.

  1. Open the Satellite console in your browser.
  2. Select the location where you created your storage configuration.
  3. Click the Locations tab, then click Storage.
  4. Click the storage configuration that you want to assign to a cluster group.
  5. On the Configuration details page, click Create storage assignment.
  6. In the Create an assignment pane, enter a name for your assignment.
  7. From the Version drop-down list, select the storage configuration version that you want to assign.
  8. Click Create to create the assignment.
  9. Verify that your storage configuration is deployed to your cluster.
    1. From the Satellite console, navigate to your Location and select Storage
    2. Click the storage configuration that you created and review the Assignments tab.
    3. Click the Assignment that you created and review the Rollout status for your configuration.

Creating an assignment in the CLI

  1. List your storage configurations and make a note of the storage configuration that you want to assign to your clusters.

    ibmcloud sat storage config ls
    
  2. Get the ID of the cluster, cluster group, or service that you want to assign storage to.

    To make sure that your cluster is registered with Satellite Config or to create groups, see Setting up clusters to use with Satellite Config.

    Example command to list cluster groups.

    ibmcloud sat group ls
    

    Example command to list clusters.

    ibmcloud oc cluster ls --provider satellite
    

    Example command to list Satellite services.

    ibmcloud sat service ls --location <location>
    
  3. Assign your storage configuration to the cluster, group, or service that you retrieved earlier. For more information, see the ibmcloud sat storage assignment create command.

    Example command to assign a configuration to a cluster group.

    ibmcloud sat storage assignment create --group GROUP --config CONFIG --name NAME
    

    Example command to assign a configuration to a cluster.

    ibmcloud sat storage assignment create --cluster CLUSTER --config CONFIG --name NAME
    

    Example command to assign a configuration to a service cluster.

    ibmcloud sat storage assignment create --service-cluster-id CLUSTER --config CONFIG --name NAME
    
  4. Verify that your assignment is created.

    ibmcloud sat storage assignment ls (--cluster CLUSTER | --config CONFIG | --location LOCATION | --service-cluster-id CLUSTER)
    

Creating a storage assignment in the API

  1. Copy one of the following example requests.

    Example request to assign a configuration to a cluster.

    curl -X POST "https://containers.cloud.ibm.com/global/v2/storage/satellite/createAssignmentByCluster" -H "accept: application/json" -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" -d "{ \"channelName\": \"CONFIGURATION-NAME\", \"cluster\": \"CLUSTER-ID\", \"controller\": \"LOCATION-ID\", \"name\": \"ASSIGNMENT-NAME\"}"
    

    Example request to assign configuration to a cluster group.

    curl -X POST "https://containers.cloud.ibm.com/global/v2/storage/satellite/createAssignment" -H "accept: application/json" -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" -d "{ \"channelName\": \"CONFIGURATION-NAME\", \"cluster\": \"string\", \"groups\": [ \"CLUSTER-GROUP\" ], \"name\": \"ASSIGNMENT-NAME\"}"
    
  2. Replace the variables with your details and run the request.

  3. Verify the assignment was created by listing your assignments.

    curl -X GET "https://containers.cloud.ibm.com/global/v2/storage/satellite/getAssignments" -H "accept: application/json" -H "Authorization: Bearer TOKEN"
    

Updating storage assignments in the console

You can use the Satellite console to apply the latest patch updates to your assignments.

  1. From the Locations page in the Satellite console, select your location.

  2. Click the Storage tab to view your configurations.

  3. Click the configuration you want to update.

  4. Click information Information (i) icon to apply the latest revision or patch.

  5. Optional: Enable automatic patch updates for your storage assignment. Enabling automatic patch updates ensures that your assignment always has the latest security fixes.

If you enable automatic patch updates, you must still apply major updates manually.

Manually upgrading assignments in the CLI

Upgrade an assignment to use the latest storage template revision.

  1. List your Satellite storage assignments, make a note of the Satellite assignment you want to upgrade.

    ibmcloud sat storage assignment ls
    
  2. List the Satellite storage templates to see the latest available versions.

    ibmcloud sat storage template ls
    
  3. Upgrade the Satellite assignment.

    Example command to upgrade an assignment.

    ibmcloud sat storage assignment upgrade --assignment ASSIGNMENT
    

Enabling automatic patch updates for configurations and assignments in the CLI

You can use the sat storage assignment autopatch enable CLI to enable automatic patch updates for your assignments. Enabling automatic patch updates applies the latest storage template revisions (patches) automatically. You must still apply major updates manually.

  1. List your Satellite storage configurations. Make a note of the configuration ID.

    ibmcloud sat storage assignment ls
    
  2. Run one of the following example commands to enable automatic patch updates for your configuration and its associated assignments. Enter the configuration ID that you retrieved in the previous step.

    Example command to enable automatic patch updates for an assignment.

    ibmcloud sat storage assignment autopatch enable --config CONFIG  (--all | --assignment ASSIGNMENT-ID [--assignment ASSIGNMENT-ID])
    

    Example command to enable automatic patch updates for all storage assignments under a given configuration.

    ibmcloud sat storage assignment autopatch enable --config CONFIG --all
    

    Example command to disable automatic patch updates for all assignments under a specific configuration.

    ibmcloud sat storage assignment autopatch disable --config CONFIG --all
    

    Example command to disable automatic patch updates for an single assignment and a specific configuration.

    ibmcloud sat storage assignment autopatch disable --config CONFIG --assignment ASSIGNMENT-ID
    

    Example command to disable automatic patch updates for an multiple assignment and a specific configuration.

    ibmcloud sat storage assignment autopatch disable --config CONFIG --assignment ASSIGNMENT-ID --assignment ASSIGNMENT-ID
    

Upgrading a configuration and assignments in the API

You can use the /v2/storage/satellite/updateAssignment API to update your assignments with new clusters or cluster groups. Set updateConfigVersion to true to apply the revision update.

  1. Copy the following example request and replace the variables for the cluster groups and assignments that you want to update.

    curl -X PATCH "https://containers.cloud.ibm.com/global/v2/storage/satellite/updateAssignment" -H "accept: application/json" -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" -d "{ \"groups\": [ \"CLUSTER-GROUPS\" ], \"name\": \"ASSIGNMENT-NAME\", \"updateConfigVersion\": true, \"uuid\": \"ASSIGNMENT-ID\"}"
    
  2. Run the request.

  3. Get the details of you assignment to verify the update.

    curl -X GET "https://containers.cloud.ibm.com/global/v2/storage/satellite/getAssignment?uuid=ASSIGNMENT-ID" -H "accept: application/json" -H "Authorization: Bearer TOKEN"
    

Enabling automatic patch updates for assignments in the API

You can use the /v2/storage/satellite/setAssignmentAutoupgrade API to enable automatic patch updates for your assignments. Enabling automatic patch updates applies the latest storage template revisions (patches) automatically. You must still apply major updates manually.

  1. Copy the following example request and replace the variables for the cluster groups and assignments that you want to update.

    curl -X PATCH "https://containers.cloud.ibm.com/global/v2/storage/satellite/setAssignmentAutoupgrade" -H "accept: application/json" -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" -d { "config": "string", "controller": "string", "autopatch": boolean,"assignment" : { "all": boolean, "uuid": ["string", "string", ...], } }
    
  2. Run the request.

  3. Get the details of you assignment to verify the upgrade.

    curl -X GET "https://containers.cloud.ibm.com/global/v2/storage/satellite/getAssignment?uuid=ASSIGNMENT-ID" -H "accept: application/json" -H "Authorization: Bearer TOKEN"
    

Optional: Adding additional worker nodes to your ODF configuration

  1. Add more worker nodes to your ODF configuration.

    ibmcloud sat storage config param set --config <config-name> -p "worker-nodes=<comma-separated values of new worker-nodes followed by list of old worker-nodes>" --apply
    
  2. Verify that the storage configuration resources are deployed.

    1. Get the storagecluster that you deployed and verify that the phase is Ready.
      oc get storagecluster -n openshift-storage
      
      Example output
      NAME                 AGE   PHASE   EXTERNAL   CREATED AT             VERSION
      ocs-storagecluster   72m   Ready              2021-02-10T06:00:20Z   4.6.0
      
    2. Get a list of pods in the openshift-storage namespace and verify that the status is Running.
      oc get pods -n openshift-storage
      
      Example output
      NAME                                                              READY   STATUS      RESTARTS   AGE
      csi-cephfsplugin-9g2d5                                            3/3     Running     0          8m11s
      csi-cephfsplugin-g42wv                                            3/3     Running     0          8m11s
      csi-cephfsplugin-provisioner-7b89766c86-l68sr                     5/5     Running     0          8m10s
      csi-cephfsplugin-provisioner-7b89766c86-nkmkf                     5/5     Running     0          8m10s
      csi-cephfsplugin-rlhzv                                            3/3     Running     0          8m11s
      csi-rbdplugin-8dmxc                                               3/3     Running     0          8m12s
      csi-rbdplugin-f8c4c                                               3/3     Running     0          8m12s
      csi-rbdplugin-nkzcd                                               3/3     Running     0          8m12s
      csi-rbdplugin-provisioner-75596f49bd-7mk5g                        5/5     Running     0          8m12s
      csi-rbdplugin-provisioner-75596f49bd-r2p6g                        5/5     Running     0          8m12s
      noobaa-core-0                                                     1/1     Running     0          4m37s
      noobaa-db-0                                                       1/1     Running     0          4m37s
      noobaa-endpoint-7d959fd6fb-dr5x4                                  1/1     Running     0          2m27s
      noobaa-operator-6cbf8c484c-fpwtt                                  1/1     Running     0          9m41s
      ocs-operator-9d6457dff-c4xhh                                      1/1     Running     0          9m42s
      rook-ceph-crashcollector-169.48.170.83-89f6d7dfb-gsglz            1/1     Running     0          5m38s
      rook-ceph-crashcollector-169.48.170.88-6f58d6489-b9j49            1/1     Running     0          5m29s
      rook-ceph-crashcollector-169.48.170.90-866b9d444d-zk6ft           1/1     Running     0          5m15s
      rook-ceph-drain-canary-169.48.170.83-6b885b94db-wvptz             1/1     Running     0          4m41s
      rook-ceph-drain-canary-169.48.170.88-769f8b6b7-mtm47              1/1     Running     0          4m39s
      rook-ceph-drain-canary-169.48.170.90-84845c98d4-pxpqs             1/1     Running     0          4m40s
      rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-6dfbb4fcnqv9g   1/1     Running     0          4m16s
      rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-cbc56b8btjhrt   1/1     Running     0          4m15s
      rook-ceph-mgr-a-55cc8d96cc-vm5dr                                  1/1     Running     0          4m55s
      rook-ceph-mon-a-5dcc4d9446-4ff5x                                  1/1     Running     0          5m38s
      rook-ceph-mon-b-64dc44f954-w24gs                                  1/1     Running     0          5m30s
      rook-ceph-mon-c-86d4fb86-s8gdz                                    1/1     Running     0          5m15s
      rook-ceph-operator-69c46db9d4-tqdpt                               1/1     Running     0          9m42s
      rook-ceph-osd-0-6c6cc87d58-79m5z                                  1/1     Running     0          4m42s
      rook-ceph-osd-1-f4cc9c864-fmwgd                                   1/1     Running     0          4m41s
      rook-ceph-osd-2-dd4968b75-lzc6x                                   1/1     Running     0          4m40s
      rook-ceph-osd-prepare-ocs-deviceset-0-data-0-29jgc-kzpgr          0/1     Completed   0          4m51s
      rook-ceph-osd-prepare-ocs-deviceset-1-data-0-ckvv2-4jdx5          0/1     Completed   0          4m50s
      rook-ceph-osd-prepare-ocs-deviceset-2-data-0-szmjd-49dd4          0/1     Completed   0          4m50s
      rook-ceph-rgw-ocs-storagecluster-cephobjectstore-a-7f7f6df9rv6h   1/1     Running     0          3m44s
      rook-ceph-rgw-ocs-storagecluster-cephobjectstore-b-554fd9dz6dm8   1/1     Running     0          3m41s
      

Deploying an app that uses OpenShift Data Foundation

You can use the ODF storage classes to create PVCs for the apps in your clusters.

  1. Create a YAML configuration file for your PVC. In order for the PVC to match the PV, you must use the same values for the storage class and the size of the storage.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: ocs-pvc
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: sat-ocs-cephfs-gold
      resources:
        requests:
          storage: 5Gi
    
  2. Create the PVC in your cluster.

    oc apply -f pvc.yaml
    
  3. Create a YAML configuration file for a pod that mounts the PVC that you created. The following example creates an nginx pod that writes the current date and time to a test.txt file.

    apiVersion: v1
    kind: Pod
    metadata:
      name: app
    spec:
      containers:
      - name: app
        image: nginx
        command: ["/bin/sh"]
        args: ["-c", "while true; do echo $(date -u) >> /test/test.txt; sleep 5; done"]
        volumeMounts:
        - name: persistent-storage
          mountPath: /test
      volumes:
      - name: persistent-storage
        persistentVolumeClaim:
          claimName: ocs-pvc
    
  4. Create the pod in your cluster.

    oc apply -f pod.yaml
    
  5. Verify that the pod is deployed. Note that it might take a few minutes for your app to get into a Running state.

    oc get pods
    

    Example output

    NAME                                READY   STATUS    RESTARTS   AGE
    app                                 1/1     Running   0          2m58s
    
  6. Verify that the app can write data.

    1. Log in to your pod.

      oc exec <app-pod-name> -it bash
      
    2. Display the contents of the test.txt file to confirm that your app can write data to your persistent storage.

      cat /test/test.txt
      

      Example output

      Tue Mar 2 20:09:19 UTC 2021
      Tue Mar 2 20:09:25 UTC 2021
      Tue Mar 2 20:09:31 UTC 2021
      Tue Mar 2 20:09:36 UTC 2021
      Tue Mar 2 20:09:42 UTC 2021
      Tue Mar 2 20:09:47 UTC 2021
      
    3. Exit the pod.

      exit
      

Manually upgrading configurations in the CLI

You can upgrade your Satellite storage configurations to get the latest storage template revision within the same major version.

  1. List your Satellite storage configurations, make a note of the Satellite configuration you want to upgrade.

    ibmcloud sat storage config ls
    
  2. Upgrade the Satellite configuration. Note, only the configuration is updated. If you want to upgrade the assignments that use this configuration, you can specify the --include-assignments option or you can manually update each assignment using the assignment update command.

    Example command to upgrade a configuration to the latest revision.

    ibmcloud sat storage config upgrade --config CONFIG [--include-assignments]
    

    Example command to upgrade a configuration and it's associated assignments to the latest revision.

    ibmcloud sat storage config upgrade --config CONFIG --include-assignments
    

Scaling up your ODF configuration

To scale your ODF configuration by adding disks to your worker nodes, increase the num-of-osd parameter value.

ibmcloud sat storage config param set --config <config-name> -p num-of-osd=2 --apply

Removing the ODF remote storage assignment from the command line

Use the command line to remove a storage assignment.

  1. Run the following command to delete your ODF storage assignment.
    oc delete ocscluster --all
    
  2. List your storage assignments and find the one that you used for your cluster.
    ibmcloud sat storage assignment ls (--cluster CLUSTER | --config CONFIG | --location LOCATION | --service-cluster-id CLUSTER)
    
  3. Remove the assignment. After the assignment is removed, the ODF driver pods and storage classes are removed from all clusters that were part of the storage assignment.
    ibmcloud sat storage assignment rm --assignment <assignment_ID>
    

Parameter reference

4.12 parameter reference

Table 1. 4.12 parameter reference
Display name CLI option Type Description Required? Default value
OSD pod volume size osd-size Config The OSD storage size in Gi(Use 512Gi or more). The default value is 512Gi. true 512Gi
OSD pod storage class osd-storage-class Config The storage class to use when dynamically provisioning volumes for the OSD pods. true N/A
Number of OSD volumes num-of-osd Config The number of storage device replicas to create. The default value is 1, which creates 1 device across 3 nodes. Increase by 1 for each additional set of 3 devices. For must use cases, leave the default value of 1. true 1
Worker node names worker-nodes Config A comma separated list of the worker node names where you want to deploy ODF. Leave this field blank to deploy ODF across all worker nodes in your cluster. The minimum number of worker nodes is 3. You can find your worker node names by running oc get nodes. false N/A
Upgrade odf-upgrade Config If you are upgrading an existing ODF installation, set to true. true false
Billing type billing-type Config The billing type you want to use. Choose from essentials or advanced. true advanced
IBM COS endpoint ibm-cos-endpoint Config The IBM COS regional public endpoint. false N/A
IBM COS location constraint ibm-cos-location Config The location constraint that you want to use when creating your bucket. For example us-east-standard. false N/A
Access key ID ibm-cos-access-key Secret Your IBM COS HMAC access key ID . false N/A
Secret access key ibm-cos-secret-key Secret Your IBM COS HMAC secret access key. false N/A
Encryption enabled cluster-encryption Config Set to true if you want to enable cluster-wide encryption. true false
IAM API key iam-api-key Secret Your IAM API key. true N/A
Perform Cleanup perform-cleanup Config Set to true if you want to perform complete cleanup of ODF on assignment deletion. true false
KMS encryption kms-encryption Config Set to true if you want to enable storage class encryption. true false
KMS instance name kms-instance-name Config Your KMS instance name. The instance name must only include alphanumeric characters, -, _ or . and start and end with an alphanumeric character. false N/A
KMS instance id kms-instance-id Config Your KMS instance id. false N/A
KMS instance Base URL kms-base-url Config Your KMS instance public URL to connect to the instance. false N/A
KMS instance API key token URL kms-token-url Config API key token URL to generate token for KMS instance. false N/A
KMS root key kms-root-key Secret KMS root key of your instance. false N/A
KMS IAM API key kms-api-key Secret IAM API key to access the KMS instance. The API key that you provide must have at least Viewer access to the KMS instance. false N/A
Ignore Noobaa ignore-noobaa Config Set to false if you want to deploy MultiCloud Object Gateway (Noobaa) true true

4.13 parameter reference

Table 2. 4.13 parameter reference
Display name CLI option Type Description Required? Default value
OSD pod volume size osd-size Config The OSD storage size in Gi(Use 512Gi or more). The default value is 512Gi. true 512Gi
OSD pod storage class osd-storage-class Config The storage class to use when dynamically provisioning volumes for the OSD pods. true N/A
Number of OSD volumes num-of-osd Config The number of storage device replicas to create. The default value is 1, which creates 1 device across 3 nodes. Increase by 1 for each additional set of 3 devices. For must use cases, leave the default value of 1. true 1
Worker node names worker-nodes Config A comma separated list of the worker node names where you want to deploy ODF. Leave this field blank to deploy ODF across all worker nodes in your cluster. The minimum number of worker nodes is 3. You can find your worker node names by running oc get nodes. false N/A
Upgrade odf-upgrade Config If you are upgrading an existing ODF installation, set to true. true false
Billing type billing-type Config The billing type you want to use. Choose from essentials or advanced. true advanced
IBM COS endpoint ibm-cos-endpoint Config The IBM COS regional public endpoint. false N/A
IBM COS location constraint ibm-cos-location Config The location constraint that you want to use when creating your bucket. For example us-east-standard. false N/A
Access key ID ibm-cos-access-key Secret Your IBM COS HMAC access key ID . false N/A
Secret access key ibm-cos-secret-key Secret Your IBM COS HMAC secret access key. false N/A
Encryption enabled cluster-encryption Config Set to true if you want to enable cluster-wide encryption. true false
IAM API key iam-api-key Secret Your IAM API key. true N/A
Perform Cleanup perform-cleanup Config Set to true if you want to perform complete cleanup of ODF on assignment deletion. true false
KMS encryption kms-encryption Config Set to true if you want to enable storage class encryption. true false
KMS instance name kms-instance-name Config Your KMS instance name. The instance name must only include alphanumeric characters, -, _ or . and start and end with an alphanumeric character. false N/A
KMS instance id kms-instance-id Config Your KMS instance id. false N/A
KMS instance Base URL kms-base-url Config Your KMS instance public URL to connect to the instance. false N/A
KMS instance API key token URL kms-token-url Config API key token URL to generate token for KMS instance. false N/A
KMS root key kms-root-key Secret KMS root key of your instance. false N/A
KMS IAM API key kms-api-key Secret IAM API key to access the KMS instance. The API key that you provide must have at least Viewer access to the KMS instance. false N/A
Ignore Noobaa ignore-noobaa Config Set to false if you want to deploy MultiCloud Object Gateway (Noobaa) true true
Disable Noobaa LB disable-noobaa-LB Config Set to true if you want to disable Noobaa public load balancer true false
In-transit Encryption encryption-intransit Config Set to true if you want to enable in-transit encryption true false

4.14 parameter reference

Table 3. 4.14 parameter reference
Display name CLI option Type Description Required? Default value
OSD pod volume size osd-size Config The OSD storage size in Gi(Use 512Gi or more). The default value is 512Gi. true 512Gi
OSD pod storage class osd-storage-class Config The storage class to use when dynamically provisioning volumes for the OSD pods. true N/A
Number of OSD volumes num-of-osd Config The number of storage device replicas to create. The default value is 1, which creates 1 device across 3 nodes. Increase by 1 for each additional set of 3 devices. For must use cases, leave the default value of 1. true 1
Worker pools names/IDs worker-pools Config Specify a comma separated list of either the names or the IDs of the worker pools you want to use for ODF. Leave this field blank if you want to use worker-nodes. If you want to use all the worker nodes in your cluster for ODF, then leave worker-pools and worker-nodes blank. false N/A
Worker node names worker-nodes Config A comma separated list of the worker node names where you want to deploy ODF. Leave this field blank if you want to use worker-pools. If you want to use all the worker nodes in your cluster for ODF, then then leave worker-nodes and worker-pools blank. The minimum number of worker nodes is 3. You can find your worker node names by running oc get nodes. false N/A
Upgrade odf-upgrade Config If you are upgrading an existing ODF installation, set to true. true false
Billing type billing-type Config The billing type you want to use. Choose from essentials or advanced. true advanced
IBM COS endpoint ibm-cos-endpoint Config The IBM COS regional public endpoint. false N/A
IBM COS location constraint ibm-cos-location Config The location constraint that you want to use when creating your bucket. For example us-east-standard. false N/A
Access key ID ibm-cos-access-key Secret Your IBM COS HMAC access key ID . false N/A
Secret access key ibm-cos-secret-key Secret Your IBM COS HMAC secret access key. false N/A
Encryption enabled cluster-encryption Config Set to true if you want to enable cluster-wide encryption. true false
IAM API key iam-api-key Secret Your IAM API key. true N/A
Perform Cleanup perform-cleanup Config Set to true if you want to perform complete cleanup of ODF on assignment deletion. true false
KMS encryption kms-encryption Config Set to true if you want to enable storage class encryption. true false
KMS instance name kms-instance-name Config Your KMS instance name. The instance name must only include alphanumeric characters, -, _ or . and start and end with an alphanumeric character. false N/A
KMS instance id kms-instance-id Config Your KMS instance id. false N/A
KMS instance Base URL kms-base-url Config Your KMS instance public URL to connect to the instance. false N/A
KMS instance API key token URL kms-token-url Config API key token URL to generate token for KMS instance. false N/A
KMS root key kms-root-key Secret KMS root key of your instance. false N/A
KMS IAM API key kms-api-key Secret IAM API key to access the KMS instance. The API key that you provide must have at least Viewer access to the KMS instance. false N/A
Ignore Noobaa ignore-noobaa Config Set to false if you want to deploy MultiCloud Object Gateway (Noobaa) true true
Disable Noobaa LB disable-noobaa-LB Config Set to true if you want to disable Noobaa public load balancer true false
In-transit Encryption encryption-intransit Config Set to true if you want to enable in-transit encryption true false
Add Single Replica Pool(once enabled, cannot be disabled) add-single-replica-pool Config Enabling this feature creates a single replica pool without data replication, increasing the risk of data loss, data corruption, and potential system instability. Once it is enabled, it cannot be disabled true false
Taint Nodes taint-nodes Config When set the selected worker nodes will be dedicated to Data Foundation use only true false
Prepare for Disaster Recovery prepare-for-disaster-recovery Config Enabling this will set up the storage system for disaster recovery service with the essential configurations in place. This will subsequently allow seamless implementation of DR strategies for your workloads true false

Storage class reference for OpenShift Data Foundation for remote devices

Review the Satellite storage classes for OpenShift Data Foundation. You can describe storage classes in the command line with the oc describe sc <storage-class-name> command.

Table 4. Storage class reference for OpenShift Container storage
Storage class name Type File system Provisioner Volume binding mode Allow volume expansion Reclaim policy
sat-ocs-cephrbd-gold Block ext4 openshift-storage.rbd.csi.ceph.com Immediate True Delete
sat-ocs-cephfs-gold File N/A openshift-storage.cephfs.csi.ceph.com Immediate True Delete
sat-ocs-cephrgw-gold Object N/A openshift-storage.ceph.rook.io/bucket Immediate N/A Delete
sat-ocs-noobaa-gold Default OBC N/A openshift-storage.noobaa.io/obc Immediate N/A Delete
sat-ocs-cephrbd-gold-metro Block ext4 openshift-storage.rbd.csi.ceph.com WaitForFirstConsumer True Delete
sat-ocs-cephfs-gold-metro File N/A openshift-storage.cephfs.csi.ceph.com WaitForFirstConsumer True Delete