Why is the status of my OpenShift Data Foundation storage cluster stuck at Failed to reconcile?

Virtual Private Cloud Classic infrastructure

When you create your OpenShift Data Foundation storage cluster and run oc describe storagecluster <storage-cluster-name>, you see an error indicating that PVC names exceed the Kubernetes character limit.

Device sets are groups of OSDs (Object Storage Daemons) that OpenShift Data Foundation uses to distribute data across your cluster. When you create a storage cluster, OpenShift Data Foundation automatically creates PVCs for these device sets.

When you run the describe command, you see an error similar to the following example.

ceph-cluster-controller: failed to reconcile. failed to reconcile cluster "ocs-storagecluster-cephcluster": failed to configure local ceph cluster: failed to create cluster: failed to start ceph osds: 3 failures encountered while running osds in namespace openshift-storage: failed to create "provision" job for node "ocs-deviceset-ibmc-vpc-block-metro-retain-10iops-tier-0-datnv6k". Job.batch "rook-ceph-osd-prepare-aaa000aaa111a1a0e10ba1a11aa1a119" is invalid: [spec.template.spec.volumes[8].name: Invalid value: "ocs-deviceset-ibmc-vpc-block-metro-retain-10iops-tier-0-aaaaa1b-bridge": must be no more than 63 characters

Kubernetes PVC names must be fewer than 63 characters. If you have a multizone VPC cluster and create your OpenShift Data Foundation storage cluster by using a retain storage class like ibmc-vpc-block-metro-retain-10iops-tier, the corresponding device set PVCs are assigned names that exceed the 63 character limit.

Create a custom storage class with a shorter name to ensure that the generated PVC names stay within the Kubernetes character limit.

  1. Get the YAML configuration of the storage class that you want to use in your ODF storage cluster and save it in a file on your local machine.

    oc get sc ibmc-vpc-block-metro-retain-10iops-tier -o yaml

  2. Edit the name of the storage class. Make sure that the name of your storage class is fewer than 30 characters to allow for the OpenShift Data Foundation storage cluster device set IDs to be under the 63 character Kubernetes limit. Create the storage class in your cluster.

    oc create -f <custom-storage-class.yaml>

  3. Clean up your OpenShift Data Foundation deployment.

  4. Create an OpenShift Data Foundation deployment that uses the storage class you created.