---
name: containers-ts-storage-vpc-block-monitoring
title: Debugging Block Storage for VPC metrics
description: If your storage metrics are not visible in the monitoring dashboard, you can manually view them in the CLI. Note that manual verification of your storage metrics is a temporary workaround and not a permanent monitoring solution for viewing metrics. After completing the following steps, if you are able to manually view the metrics in the CLI and not the dashboard, this indicates that there is a console issue for which you must [open a support issue](/docs/containers?topic=containers-get-help.
last-updated: 2026-05-26
---

> ## Documentation Index
> The table of contents for this documentation set is at https://cloud.ibm.com/docs/containers?format=markdown
> The index for all IBM Cloud docs is at: https://cloud.ibm.com/docs/llms.txt
> Use these files to discover more information as needed.

# Debugging Block Storage for VPC metrics 
{: #debug_monitoring}
{: troubleshoot}
{: support}

When you try to view Block Storage for VPC metrics in the monitoring dashboard, the metrics do not populate. 
{: tsSymptoms}

Metrics might fail to populate in the dashboard for one of the following reasons: 
* The PVC you want to monitor might not be mounted. Metrics are only populated for PVCs that are mounted to a pod.
* There might be a console-related issue, which can be verified by manually viewing the storage metrics in the CLI. 
{: tsCauses}

Check that the PVC is mounted. If the issue persists, manually view your metrics in the CLI to determine if the cause is related to issues with the console. 
{: tsResolve}

1. [Log in to your account. If applicable, target the appropriate resource group. Set the context for your cluster.](https://cloud.ibm.com/docs/containers?topic=containers-access_cluster&format=markdown)

2. Describe the PVC. If the **Used By** row of the output is populated with the name of a pod, then the PVC is mounted. 

    ```sh
    kubectl describe pvc <pvc_name>
    ```
    {: pre}

    Example output 

    ```sh
    Name:          my-pvc
    Namespace:     default
    StorageClass:  ibmc-vpc-block-5iops-tier
    Status:        Bound
    Volume:        pvc-a11a11a1-111a-111a-a1a1-aaa111aa1a1a 
    Labels:        <none>
    Annotations:   pv.kubernetes.io/bind-completed: yes
                   pv.kubernetes.io/bound-by-controller: yes
                   volume.beta.kubernetes.io/storage-provisioner: vpc.block.csi.ibm.io
    Finalizers:    [kubernetes.io/pvc-protection]
    Capacity:      10Gi
    Access Modes:  RWO
    VolumeMode:    Filesystem
    Used By:       my-pod-11a1a1a1a1-1a11a 
    Events:        <none>
    ```
    {: screen}

3. If the PVC is not mounted to a pod, review the steps for [setting up Block Storage for VPC](https://cloud.ibm.com/docs/containers?topic=containers-vpc-block&format=markdown#vpc-block-add) and mount the PVC to a pod. Then try to view the metrics again. 
4. If the PVC is mounted, follow the steps for [manually verifying the Block Storage for VPC metrics](#manual-monitor-metrics) then [open a support issue](https://cloud.ibm.com/docs/containers?topic=containers-get-help. The steps for manual verification in the CLI allow you to view your metrics, but are not a solution for metrics that do not populate in the console. However, if you are able to manually verify your metrics, this indicates that there is a console issue for which you must open an issue.

&format=markdown## Manually viewing storage metrics in the CLI
{: #manual-monitor-metrics}

If your storage metrics are not visible in the monitoring dashboard, you can manually view them in the CLI. Note that manual verification of your storage metrics is a temporary workaround and not a permanent monitoring solution for viewing metrics. After completing the following steps, if you are able to manually view the metrics in the CLI and not the dashboard, this indicates that there is a console issue for which you must [open a support issue](/docs/containers?topic=containers-get-help.
{: shortdesc}

After you complete the following steps, make sure to remove the resources you created while debugging. 
{: important}


1. Create and deploy a custom `clusterRole` configuration. In this example, the `clusterRole` is named `test-metrics-reader`.

    ```yaml
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
    name: test-metrics-reader
    rules:
    - nonResourceURLs:
        - "/metrics"
        verbs:
        - get
    - apiGroups:
        - ""
        resources:
        - nodes/metrics
        verbs:
        - get
    ```
    {: pre}

    ```sh
    kubectl apply -f <file_name>
    ```
    {: pre}

1. Create a service account. In this example, the service account is named `test-sa`.

    ```sh
    kubectl create sa test-sa
    ```
    {: pre}

1. Add a `clusterRoleBinding` to the `clusterRole`.

    ```sh
    kubectl create clusterrolebinding test-metrics-reader --clusterrole test-metrics-reader --serviceaccount=default:test-sa
    ```
    {: pre}

1. List your nodes and note the name and IP of the node for which you want to gather metrics. 

    ```sh
    kubectl get nodes
    ```
    {: pre}

    Example output

    ```sh
    NAME          STATUS    ROLES    AGE     VERSION              
    10.111.1.11   Ready     <none>   1d      v1.35+IKS            
    ```
    {: screen}

1. Create a yaml file to deploy a pod onto the node. Make sure to specify the service account you created and the node IP address.

    ```yaml
    apiVersion: v1
    kind: Pod
    metadata:
    name: testpod
    spec:
    nodeName: 10.111.1.111
    containers:
    - image: nginx
        name: nginx
    serviceAccountName: test-sa
    ```
    {: pre}

    ```sh
    kubectl apply -f <file_name>
    ```
    {: pre}

1. Retrieve the service account token from within the pod. 

    1. Log in to the pod.

        ```sh
        kubectl exec testpod -it -- bash
        ```
        {: pre}

    1. Run the following command to get the token. Note that there is no output. 

        ```sh
        token=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
        ```
        {: pre}

1. While you are still logged in to the pod, run the command to view the storage metrics. Make sure to specify the node IP address. 

    ```sh
    curl -k -H "authorization: bearer <token> https://<node_IP>:10250/metrics | grep kubelet_volume_stats
    ```
    {: pre}

1. View the metrics in the terminal output. You might have to wait several minutes for the metrics to output. If you are still unable to view metrics, [open a support issue](/docs/containers?topic=containers-get-help.

1. After you have finished viewing the metrics, and determined whether the issue is related to dashboard or the metrics agent, delete the configurations and resources that you created in the previous steps.

Do not skip this step. 
{: important}

1. Exit the pod.
    ```sh
    exit
    ```
    {: pre}

1. Delete the pod.
    ```sh 
    kubectl delete pod testpod
    ```
    {: pre}

1. Delete the `clusterRoleBinding`.
    ```sh
    kubectl delete clusterrolebinding test-metrics-reader
    ```
    {: pre}

1. Delete the service account. 
    ```sh
    kubectl delete sa test-sa
    ```
    {: pre}

1. Delete the cluster role.
    ```sh
    kubectl delete clusterrole test-metrics-reader
    ```
    {: pre}