Setting up your IBM Cloud Object Storage instance

IBM Cloud Object Storage is persistent, highly available storage that you can mount to your apps. The plug-in is a Kubernetes Flex-Volume plug-in that connects Cloud Object Storage buckets to pods in your cluster. Information stored with IBM Cloud Object Storage is encrypted in transit and at rest, dispersed across many geographic locations, and accessed over HTTP by using a REST API.

If you want to use IBM Cloud Object Storage in a private cluster without public network access, you must set up your IBM Cloud Object Storage service instance for HMAC authentication. If you don't want to use HMAC authentication, you must open up all outbound network traffic on port 443 for the plug-in to work properly in a private cluster.

Creating an IBM Cloud Object Storage instance

Before you can start using object storage in your cluster, you must provision an IBM Cloud Object Storage service instance in your account.

The IBM Cloud Object Storage plug-in works with any s3 API endpoint. For example, you might want to use a local Cloud Object Storage server, such as Minio, or connect to an s3 API endpoint that you set up at a different cloud provider instead of an IBM Cloud Object Storage service instance.

Follow these steps to create an IBM Cloud Object Storage service instance. If you plan to use a local Cloud Object Storage server or a different s3 API endpoint, see the provider documentation to set up your Cloud Object Storage instance.

Open the IBM Cloud Object Storage catalog page.
Enter a name for your service instance, such as cos-backup, and select the same resource group that your cluster is in. To view the resource group of your cluster, run ibmcloud oc cluster get --cluster <cluster_name_or_ID>.
Review the plan options for pricing information and select a plan.
Click Create. The service details page opens.
To continue to set up Object Storage to use with your cluster, see Creating service credentials.

Creating IBM Cloud Object Storage service credentials

Before you begin, Create your object storage service instance.

In the navigation on the service details page for your Object Storage instance, click Service Credentials.
Click New credential. A dialog box opens.
Enter a name for your credentials.
From the Role drop-down list, select the Object Storage access role for the actions that you want storage users in your cluster to have access to. You must select at least Writer service access role to use the auto-create-bucket dynamic provisioning feature. If you select Reader, then you can't use the credentials to create buckets in IBM Cloud Object Storage and write data to it.
Optional: In Add Inline Configuration Parameters (Optional), enter {"HMAC":true} to create more HMAC credentials for the IBM Cloud Object Storage service. HMAC authentication adds an extra layer of security to the OAuth2 authentication by preventing the misuse of expired or randomly created OAuth2 tokens.
Click Add. Your new credentials are in the Service Credentials table.
Click View credentials.
Make note of the apikey to use OAuth2 tokens to authenticate with the IBM Cloud Object Storage service. For HMAC authentication, in the cos_hmac_keys section, note the access_key_id and the secret_access_key.
Store your service credentials in a Kubernetes secret inside the cluster to enable access to your IBM Cloud Object Storage service instance.

Creating a secret for the object storage service credentials

To access your IBM Cloud Object Storage service instance to read and write data, you must securely store the service credentials in a Kubernetes secret. The IBM Cloud Object Storage plug-in uses these credentials for every read or write operation to your bucket.

Follow these steps to create a Kubernetes secret for the credentials of an IBM Cloud Object Storage service instance. If you plan to use a local Cloud Object Storage server or a different s3 API endpoint, create a Kubernetes secret with the appropriate credentials.

Prerequisites

Make sure that you have the Manager service access role for the cluster.
Set up your KMS provider.
Retrieve the apikey, or the access_key_id and the secret_access_key of your IBM Cloud Object Storage service credentials. Note that the service credentials must be enough for the bucket operations that your app needs to perform. For example, if your app reads data from a bucket, the service credentials you see in your secret must have Reader permissions at minimum.

Creating an object storage secret in your cluster

Complete the following steps to create a secret in your cluster for IBM Cloud Object Storage.

If you want to integrate Key Protect encryption when creating new buckets from PVCs in your cluster, you must to include the root key CRN when creating your IBM Cloud Object Storage secret. Note that Key Protect encryption can't be added to existing buckets. Encryption can be used only when you create new buckets. If the root key that is used for encryption is deleted, all the files in the associated buckets are inaccessible.

Access your Red Hat OpenShift cluster.

Get the GUID of your IBM Cloud Object Storage service instance.

ibmcloud resource service-instance <service_name> | grep GUID

Create a Kubernetes secret to store your service credentials. Choose from one the following examples to create your secret.
- Example create secret command that uses an API key.
```
oc create secret generic cos-write-access --type=ibm/ibmc-s3fs --from-literal=api-key=<api_key> --from-literal=service-instance-id=<service_instance_guid>
```
- Example create secret command that uses HMAC authentication.
```
oc create secret generic cos-write-access --type=ibm/ibmc-s3fs --from-literal=access-key=<access_key_ID> --from-literal=secret-key=<secret_access_key>    
```
- Example secret configuration file for enabling encryption with your KMS provider.
  1. Save the following YAML to a file.
```
apiVersion: v1
data:
    access-key: xxx
    secret-key: xxx
    kp-root-key-crn: <CRN> # Key Protect or HPCS root key crn in base64 encoded format 
kind: Secret
metadata:
    name: cos-write-access 
type: ibm/ibmc-s3fs
```
    api-key
    
    Enter the API key that you retrieved from your IBM Cloud Object Storage service credentials earlier. If you want to use HMAC authentication, specify the access-key and secret-key instead.
    
    access-key
    
    Enter the access key ID that you retrieved from your IBM Cloud Object Storage service credentials earlier. If you want to use OAuth2 authentication, specify the api-key instead.
    
    secret-key
    
    Enter the secret access key that you retrieved from your IBM Cloud Object Storage service credentials earlier. If you want to use OAuth2 authentication, specify the api-key instead.
    
    service-instance-id
    
    Enter the GUID of your IBM Cloud Object Storage service instance that you retrieved earlier.
    
    kp-root-key-crn
    
    Enter the base64 encoded Key Protect root key CRN to use Key Protect encryption.
  2. Apply the secret to your cluster.
```
kubectl apply -f <secret_name>
```

Get the secrets in your cluster and verify the output.

oc get secret

Example output

NAME                  TYPE                                  DATA   AGE
cos-write-access      ibm/ibmc-s3fs                         2      7d19h
default-au-icr-io     kubernetes.io/dockerconfigjson        1      55d
default-de-icr-io     kubernetes.io/dockerconfigjson        1      55d
...

Install the IBM Cloud Object Storage plug-in, or if you already installed the plug-in, decide on the configuration for your IBM Cloud Object Storage bucket.
Optional: Add your secret to the default storage classes. Storage Class Reference

Limitations

IBM Cloud Object Storage is based on the s3fs-fuse file system. You can review a list of limitations in the s3fs-fuse repository.
To access a file in IBM Cloud Object Storage with a non-root user, you must set the runAsUser and fsGroup values in your deployment to the same value.