Cluster states and statuses
The Master Status provides details of what operation from the master state is in progress. The status includes a timestamp of how long the master has been in the same state, such as Ready (1 month ago)
. The Master State reflects the lifecycle of possible operations that can be performed on the master, such as deploying, updating, and deleting. Each state and status is described in the following documentation.
Review your cluster state in the console or by running the following commands.
ibmcloud ks cluster ls
ibmcloud ks cluster get --cluster <cluster_name_or_ID>
Aborted
You sent a delete request before the Kubernetes master deployment completed. After your cluster is deleted, it is removed from your dashboard. If your cluster is stuck in this state for a long time, open an IBM Cloud support case.
Critical
The Kubernetes master can't be reached or all worker nodes in the cluster are down. If you enabled IBM Key Protect in your cluster, the Key Protect container might fail to encrypt or decrypt your cluster secrets. If so, you can view an error
with more information when you run kubectl get secrets
.
Create failed
The cluster cannot be created. Delete the failed cluster and try to create another one. If the issue persists when creating additional clusters, open an IBM Cloud support case.
Delete failed
The Kubernetes master or at least one worker node can't be deleted. List worker nodes by running ibmcloud ks worker ls --cluster <cluster_name_or_ID>
.
IBM Support is notified and works to resolve the issue. You can't resolve the issue by trying to delete the cluster again.
Instead, check the Master Status field for more information, or wait for the cluster to delete.
- If worker nodes are listed, see Unable to create or delete worker nodes.
- If no workers are listed, open an IBM Cloud support case.
Deleted
The cluster is deleted but not yet removed from your dashboard. If your cluster is stuck in this state for a long time, open an IBM Cloud support case.
Deleting
The cluster is being deleted and cluster infrastructure is being dismantled. You can't access the cluster. You can't undo a deletion. After the cluster is deleted, you can no longer check the master state because the cluster is completely removed.
Deployed
The master is successfully deployed. Check the status to verify that the master is Ready
or to see if an update is available.
Deploy failed
The deployment of the Kubernetes master can't be completed. You can't resolve this state. Contact IBM Cloud support by opening an IBM Cloud support case.
Deploying
The Kubernetes master is not fully deployed yet. You can't access your cluster. Wait until your cluster is fully deployed to review the health of your cluster.
Error
The master is not operational. IBM is automatically notified and takes action to resolve this issue. You can continue monitoring the health until the master is normal
. You can also open an IBM Cloud support case.
Normal
All worker nodes in a cluster are up and running. You can access the cluster and deploy apps to the cluster. This state is considered healthy and does not require an action from you.
Although the worker nodes might be normal, other infrastructure resources, such as networking and storage, might still need attention. If you just created the cluster, some parts of the cluster that are used by other services such as Ingress secrets or registry image pull secrets, might still be in process.
Pending
The Kubernetes master is deployed. The worker nodes are provisioning and aren't available in the cluster yet. You can access the cluster, but you can't deploy apps to the cluster.
Requested
A request to create the cluster and order the infrastructure for the Kubernetes master and worker nodes is sent. When the deployment of the cluster starts, the cluster state changes to Deploying
. If your cluster is stuck in the
Requested
state for a long time, open an IBM Cloud support case.
Scaled down
The master resources have been scaled down to zero replicas. This is a temporary state that occurs while etcd is being restored after a backup. You cannot interact with your cluster while it is in this state. Wait for the etcd restoration to
complete and the master state to return to deployed
.
Update failed
The master update failed. IBM Support is notified and works to resolve the issue. You can continue to monitor the health of the master until the master reaches a normal state. If the master remains in this state for more than 1 day, open an IBM Cloud support case. IBM Support might identify other issues in your cluster that you must fix before the master can be updated.
Updating
The master is updating its Kubernetes version. The update might be a patch update that is automatically applied, or a minor or major version that you applied by updating the cluster. The Kubernetes API server that runs in your Kubernetes master
is being updated to a new Kubernetes API version. During the update, you can still access and change the cluster. However, you cannot initiate concurrent master operations, such as enabling API server auditing. Worker nodes, apps, and resources
that the user deployed aren't modified and continue to run. Wait for the update to complete to review the health of your cluster. During the update, your highly available master can continue processing requests, and your app workloads and
worker nodes continue to run. After the master update is complete, you can update your worker nodes. If the update is unsuccessful, the master returns to a deployed
state and continues running the previous version. IBM Support is notified and works to resolve the issue. You can check if the update failed in the Master Status field.
Update cancelled
The master update is canceled because the cluster was not in a healthy state at the time of the update. Your master remains in this state until your cluster is healthy and you manually update the master. To update the master, use the ibmcloud ks cluster master update
command. If you don't want to update the master to the default major.minor
version during the update, include the --version
option and specify the latest patch version that is available for the major.minor
version that you want, such as 1.31
. To list available versions, run ibmcloud ks versions
.
Unsupported
The Kubernetes version that the cluster runs is no longer supported. Your cluster's health is no longer actively monitored or reported. Additionally, you can't add or reload worker nodes. To continue receiving important security updates and support, you must update your cluster. Review the version update preparation actions, then update your cluster to a supported Kubernetes version.
Warning
At least one worker node in the cluster is not available, but other worker nodes are available and can take over the workload. Try to reload the unavailable worker nodes.
- Your cluster has zero worker nodes, such as if you created a cluster without any worker nodes or manually removed all the worker nodes from the cluster. Resize your worker pool to add worker nodes to recover from a
Warning
state, and then update the Calico node entries for your worker nodes. - A control plane operation for your cluster failed. View the cluster in the console or run
ibmcloud ks cluster get --cluster <cluster_name_or_ID>
to check the Master Status for further debugging.