Setting capacity quotas for apps that use IBM Cloud Object Storage

Virtual Private Cloud Classic infrastructure

With IBM Cloud Object Storage, you can dynamically provision buckets for apps running in your IBM Cloud Kubernetes Service clusters. You can also dynamically set capacity quotas on those buckets during provisioning. Quotas can help you manage the resources your workloads use while also avoiding unnecessary charges.

Objectives

In this tutorial, you install the Object Storage plug-in in your cluster and enable quotas for any persistent volume claims (PVC) created with the plug-in.

Then, you create a PVC which dynamically creates a bucket with a quota limit in your Object Storage instance.

After that, you upload a file to your bucket and deploy a simple app to your cluster that mounts the bucket and prints the contents of that file.

Prerequisites

Before beginning this tutorial make sure you have created or installed the following resources and tools.

An IBM Cloud account. For more information, see Creating an account.
The CLI tools including the IBM Cloud CLI, the Containers service CLI plug-in, and the Helm CLI. For more information, see Getting started with the IBM Cloud CLI.
An IBM Cloud Kubernetes Service cluster. If you have a VPC cluster, make sure your VPC has a public gateway attached. For more information, see Creating clusters
An Object Storage instance in the same region as your cluster. For more information, see Provision an instance of Object Storage.

Creating a set of service credentials

Follow the steps to create a set of HMAC service credentials for your Object Storage instance. Note that the credentials you create must have the Manager role to create buckets.
After you create a set of HMAC service credentials, review the details of your credentials and make a note of the apikey, access_key_id, and secret_access_key. Save these values for the next step.

Creating a secret to store your credentials

Create a secret by using the apikey, access_key_id, and secret_access_key from your service credentials.

kubectl create secret generic my-cos-secret --type=ibm/ibmc-s3fs --from-literal=access-key=ACCESS-KEY --from-literal=secret-key=SECRET-KEY --from-literal=res-conf-apikey=API-KEY

Example output.

secret/my-cos-secret created

Verify the secret was created.

kubectl get secrets | grep my-cos

Example output.

my-cos-secret              ibm/ibmc-s3fs                         3      11m

Installing the plug-in

When you install the plug-in in your cluster, make sure to specify the --set quotaLimit=true option. Specifying this option means any buckets you create with PVCs have a quota limit equal to the storage size in the PVC.

Follow the steps to install the plug-in and enable quota limits. If you've already installed the plug-in in your cluster, you can skip this step. To see if the plug-in is already installed, follow the next step.

Verify the plug-in is installed by listing the driver pods.

kubectl get pods -n ibm-object-s3fs | grep object

Example output.

ibmcloud-object-storage-driver-k9x4l             1/1     Running   0          6m52s
ibmcloud-object-storage-driver-kj9m6             1/1     Running   0          6m52s
ibmcloud-object-storage-driver-l8gqk             1/1     Running   0          6m52s
ibmcloud-object-storage-plugin-576fb8bd7-sxlkb   1/1     Running   0          6m52s

Dynamically provisioning a bucket with a quota

You can use dynamic provisioning to automatically create a Object Storage bucket when you a create a PVC.

Copy the following PVC configuration and save it to a file called pvc.yaml. This example PVC automatically creates a bucket with a quota equal to 20Gi.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: my-cos-pvc
  namespace: default
  annotations:
    ibm.io/auto-create-bucket: "true"
    ibm.io/auto-delete-bucket: "true"
    ibm.io/secret-name: "my-cos-secret" 
    ibm.io/quota-limit: "true"
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: ibmc-s3fs-standard-cross-region

Create the PVC in your cluster.

kubectl apply -f pvc.yaml

Example output.

persistentvolumeclaim/my-cos-pvc created

List your PVCs and verify the my-cos-pvc is in the Bound state.

NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                      AGE
my-cos-pvc   Bound    pvc-64a4e0c9-b5ec-40e3-8b7e-77ff47ae6c5e   20Gi       RWO            ibmc-s3fs-standard-cross-region   6s

Navigate to your Object Storage instance in the console and click the Buckets tab.
Review the details of the automatically created bucket. The bucket name is in the format tmp-s3fs-XXXX.
Click the tmp-s3fs-XXXX bucket, then click the Configuration tab.
On the Configuration page, look for the Quota enforcement section. Note that the bucket was automatically created with a quota equal to the size you specified in the PVC. In this example, the value was 20Gi.

Uploading a file to your bucket

Save the following Pod configuration to a file called pod.yaml.

apiVersion: v1
kind: Pod
metadata:
  name: cat-test-file
spec: 
  containers:
    - name: app
      image: nginx
      volumeMounts:
      - name: my-vol
        mountPath: "/mnt"
      command: ["/bin/sh"]
      args: ["-c", "cat mnt/pod.yaml && sleep 5 && exit"]
  volumes:
    - name: my-vol
      persistentVolumeClaim:
        claimName: my-cos-pvc

Navigate to your Object Storage instance in the console and click the Buckets tab.
Click the tmp-s3fs-XXXX bucket, then click Upload.
Upload the pod.yaml file that you saved earlier.

Creating an app that mounts the bucket

Copy the following Pod configuration and save it to a file called pod.yaml. This example pod mounts the bucket that was created by the my-cos-pvc PVC and prints the contents of the pod.yaml file you uploaded earlier.

apiVersion: v1
kind: Pod
metadata:
  name: cat-test-file
spec: 
  containers:
    - name: app
      image: nginx
      volumeMounts:
      - name: my-vol
        mountPath: "/mnt"
      command: ["/bin/sh"]
      args: ["-c", "cat mnt/pod.yaml && sleep 5 && exit"]
  volumes:
    - name: my-vol
      persistentVolumeClaim:
        claimName: my-cos-pvc

Create the pod in your cluster.

oc apply -f pod.yaml

Example output.

pod/cat-test-file created

Get the logs of the cat-test-file pod. In this example, the logs contain the printed contents of the pod.yaml file you uploaded earlier.

apiVersion: v1
kind: Pod
metadata:
  name: cat-test-file
spec: 
  containers:
    - name: app
      image: nginx
      volumeMounts:
      - name: my-vol
        mountPath: "/mnt"
      command: ["/bin/sh"]
      args: ["-c", "cat mnt/pod.yaml && sleep 5 && exit"]
  volumes:
    - name: my-vol
      persistentVolumeClaim:
        claimName: my-cos-pvc

Review

In this tutorial, you installed the Object Storage plug-in in your cluster and enabled quotas for any PVCs created with the plug-in. Then, you created a PVC which dynamically created a bucket with a quota limit in your Object Storage instance. After that, you deployed a simple app that prints the contents of a file in your bucket.

Next steps

Set up the Object Storage CLI to point to your bucket.
Use the Object Storage SDKs to add storage to your apps.