---
name: containers-ts-storage-debug-cos
title: Debugging IBM Cloud Object Storage failures
description: Review the options to debug Object Storage and find the root causes of any failures.
last-updated: 2026-08-06
---

> ## Documentation Index
> The table of contents for this documentation set is at https://cloud.ibm.com/docs/containers?format=markdown
> The index for all IBM Cloud docs is at: https://cloud.ibm.com/docs/llms.txt
> Use these files to discover more information as needed.

# Debugging IBM Cloud Object Storage failures
{: #debug_storage_cos}
{: troubleshoot}
{: support}

Review the options to debug Object Storage and find the root causes of any failures.
{: shortdesc}

## Checking whether the pod that mounts your storage instance is successfully deployed
{: #debug_storage_cos_deploy}

Follow the steps to review any error messages related to pod deployment.
{: shortdesc}

1. List the pods in your cluster. A pod is successfully deployed if the pod shows a status of **Running**.

    ```sh
    kubectl get pods
    ```
    {: pre}

1. Get the details of your pod and review any error messages that are displayed in the **Events** section of your CLI output.

    ```sh
    kubectl describe pod <pod_name>
    ```
    {: pre}

1. Retrieve the logs for your pod and review any error messages.

    ```sh
    kubectl logs <pod_name>
    ```
    {: pre}
    

1. [Review the Object Storage troubleshooting documentation for steps to resolve common errors](https://cloud.ibm.com/docs/containers?topic=containers-cs_sitemap&format=markdown#sitemap_object_storage).  

## Restarting your app pod
{: #debug_storage_cos_restart}

Some issues can be resolved by restarting and redeploying your pods. Follow the steps to redeploy a specific pod.
{: shortdesc}

1. If your pod is part of a deployment, delete the pod and let the deployment rebuild it. If your pod is not part of a deployment, delete the pod and reapply your pod configuration file.
    1. Delete the pod.
        ```sh
        kubectl delete pod <pod_name>
        ```
        {: pre}

        Example output
        ```sh
        pod "nginx" deleted
        ```
        {: screen}

    2. Reapply the configuration file to redeploy the pod.
        ```sh
        kubectl apply -f <app.yaml>
        ```
        {: pre}

        Example output
        ```sh
        pod/nginx created
        ```
        {: pre}

1. If restarting your pod does not resolve the issue, [reload your worker nodes](https://cloud.ibm.com/docs/containers?topic=containers-kubernetes-service-cli&format=markdown#worker-reload-cli).

1. Verify that you use the latest IBM Cloud and IBM Cloud Kubernetes Service plug-in version.

    ```sh
    ibmcloud update
    ```
    {: pre}

    ```sh
    ibmcloud plugin repo-plugins
    ```
    {: pre}

    ```sh
    ibmcloud plugin update
    ```
    {: pre}
    

## Verifying that the storage driver and plug-in pods show a status of **Running**
{: #debug_storage_cos_driver_plugin}

Follow the steps to check the status of your storage driver and plug-in pods and review any error messages.
{: shortdesc}

1. List the pods in the `kube-system` namespace.

    ```sh
    kubectl get pods -n kube-system
    ```
    {: pre}

1. If the storage driver and plug-in pods don't show a **Running** status, get more details of the pod to find the root cause. Depending on the status of your pod, the following commands might fail.
    
    1. Get the names of the containers that run in the driver pod.

        ```sh
        kubectl describe pod POD_NAME -n kube-system
        ```
        {: pre}

    2. Export the logs from the driver pod to a `logs.txt` file on your local machine. 

        ```sh
        kubectl logs <pod_name> -n kube-system > logs.txt
        ```
        {: pre}

    3. Review the log file.

        ```sh
        cat logs.txt
        ```
        {: pre}
        

1. Check the latest logs for any error messages. [Review the Object Storage troubleshooting documentation for steps to resolve common errors](https://cloud.ibm.com/docs/containers?topic=containers-cs_sitemap&format=markdown#sitemap_object_storage).

## Checking whether your PVC is successfully provisioned
{: #debug_storage_cos_pvc}

Follow the steps to check the status of your PVC and review any error messages.
{: shortdesc}

1. Check the status of your PVC. A PVC is successfully provisioned if the PVC shows a status of **Bound**.

    ```sh
    kubectl get pvc
    ```
    {: pre}

    * If the PVC shows a status of **Bound**, the PVC is successfully provisioned.
     
       Example output  
      ```sh
      NAME         STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                AGE
      silver-pvc   Bound     pvc-4b881a6b-ada8-4a44-b568-fe909107d756   24Gi       RWX            ibmc-file-silver            7m29s
      ```
      {: screen}

    * If the status of the PVC shows **Pending**, describe the PVC and review the **Events** section of the output for any warnings or error messages. Note that PVCs that reference storage classes with the volume binding mode set to `WaitForFirstConsumer` remain **Pending** until an app pod is deployed that uses the PVC.

      ```sh
      kubectl describe pvc <pvc_name>
      ```
      {: pre}
      
      Example output
      ```sh
      Name:          local-pvc
      Namespace:     default
      StorageClass:  sat-local-file-gold
      Status:        Pending
      Volume:        
      Labels:        <none>
      Annotations:   <none>
      Finalizers:    [kubernetes.io/pvc-protection]
      Capacity:      
      Access Modes:  
      VolumeMode:    Filesystem
      Mounted By:    <none>
      Events:
      Type     Reason              Age                 From                         Message
      ----     ------              ----                ----                         -------
      Warning  ProvisioningFailed  60s (x42 over 11m)  persistentvolume-controller  storageclass.storage.k8s.io "sat-local-file-gold" not found
      ```
      {: screen}
      

1. [Review the Object Storage troubleshooting documentation for steps to resolve common Object Storage PVC errors](https://cloud.ibm.com/docs/containers?topic=containers-cs_sitemap&format=markdown#sitemap_object_storage).

## Checking and updating the kubectl CLI version
{: #debug_storage_cos_cli}

If you use a `kubectl` CLI version that does not match at least the major.minor version of your cluster, you might experience unexpected results. For example, [Kubernetes does not support](https://kubernetes.io/releases/version-skew-policy/){: external} `kubectl` client versions that are 2 or more versions apart from the server version (n +/- 2).
{: shortdesc}

1. Verify that the `kubectl` CLI version that you run on your local machine matches the Kubernetes version that is installed in your cluster. Show the `kubectl` CLI version that is installed in your cluster and your local machine.
    ```sh
    kubectl version
    ```
    {: pre}

    Example output:
    ```sh
    Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.35", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:53:57Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"darwin/amd64"}
    Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.35+IKS", GitCommit:"e15454c2216a73b59e9a059fd2def4e6712a7cf0", GitTreeState:"clean", BuildDate:"2019-04-01T10:08:07Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
    ```   
    {: screen}

    The CLI versions match if you can see the same version in `GitVersion` for the client and the server. You can ignore the `+IKS` part of the version for the server.

2. If the `kubectl` CLI versions on your local machine and your cluster don't match, either update your cluster or [install a different CLI version on your local machine](https://cloud.ibm.com/docs/containers?topic=containers-cli-install&format=markdown).

## Checking and updating the Object Storage plug-in
{: #debug_storage_cos_plugin}

[Follow the steps to update the Object Storage plug-in](https://cloud.ibm.com/docs/containers?topic=containers-storage_cos_install&format=markdown#update_cos_plugin).

## Checking driver logs and the driver version installed
{: #debug-storage-logs}

To check the driver logs and see the driver version of your Object Storage installation, perform the following steps.

1. List the pods in the kube-system namespace.
    ```sh
    kubectl get pods -n kube-system -o wide -l app=ibmcloud-object-storage-driver
    ```
    {: pre}

1. Review the logs of your pods. 
    ```sh
    kubectl -n kube-system exec -it -- cat /host/log/ibmc-s3fs.log
    ```
    {: pre}

## Updating the Helm chart
{: #debug-helm-update}

To update the Helm chart from any older version, perform the following steps.

1. Uninstall the current Helm chart.
    ```sh
    helm ls --all --all-namespaces
    ```
    {: pre}

    ```sh
    helm uninstall <helm_chart_name> -n <helm_chart_namespace>
    ```
    {: pre}

1. Add and update Helm repo.

    ```sh
    helm repo add ibm-helm https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm
    ```
    {: pre}

    ```sh
    helm repo update
    ```
    {: pre}


1. Install the new Helm chart.

    ```sh
    helm plugin uninstall ibmc
    helm fetch --untar ibm-helm/ibm-object-storage-plugin && cd ibm-object-storage-plugin
    helm plugin install ./ibm-object-storage-plugin/helm-ibmc
    helm ibmc --help
    ```
   {: pre}