IBM Cloud Docs
Updating vSAN clusters

Updating vSAN clusters

vSAN™ generates system baselines and baseline groups for use with VMware® Update Manager (VUM) and you can use these recommended baselines to update software, patches, and extensions for the vSphere® ESXi™ hosts in your VMware Cloud Foundation for Classic - Automated instance that uses vSAN. vSAN 6.6.1 and later generates automated build recommendations for vSAN clusters. vSAN combines information in the VMware Compatibility Guide and vSAN Release catalog with information about the installed vSphere ESXi releases.

These recommended updates provide the best available release to keep your hardware in a supported state.

  • vSAN System Baselines - vSAN build recommendations are provided through vSAN system baselines for VUM. vSAN generates one baseline group for each vSAN cluster. The baselines are listed in the Baselines pane of the Baselines and Groups tab. VUM automatically scans each vSAN cluster to check compliance against the baseline group. However, to upgrade your vSAN cluster, you must manually remediate the system baseline through VUM you can remediate vSAN system baseline on a single host or on the entire cluster.
  • vSAN Release catalog - The vSAN release catalog maintains information about available releases, preference order for releases, and critical patches needed for each release. vSAN requires internet connectivity to access the release catalog. You don't need to be enrolled in the Customer Experience Improvement Program (CEIP) for vSAN to access the release catalog.
  • Working with vSAN Build Recommendations - VUM checks the installed vSphere ESXi releases against information in the Hardware Compatibility List (HCL) in the VMware Compatibility Guide. It determines the correct upgrade path for each vSAN cluster, based on the current vSAN Release catalog. vSAN also includes the necessary drivers and patch updates for the recommended release in its system baseline. vSAN build recommendations ensure sure that each vSAN cluster remains at the current hardware compatibility status or better. If hardware in the vSAN cluster is not included on the HCL, vSAN recommends an upgrade to the latest release.

The vSAN cluster upgrade proceeds in the following sequence of tasks:

  • Enable the vSAN Online Health Workflow – This workflow enables the vSAN baselines in VUM so that updates can be reviewed and remediated. It needs to be carried out initially only to enable vSAN with VUM
  • Pre-requisites – Understand the prerequisites, the process, and restrictions
  • Upgrade the VMware vCenter® Server Appliance (VCSA) – For more information, see VCSA update and SSO-linked vCenters.
  • Upgrade the vSphere ESXi hosts – For more information, see Creating baselines and attaching to inventory objects.
  • Upgrade the vSAN disk format – Refer to Upgrade the vSAN Disk Format. Upgrading the disk format is optional, but for best results, upgrade the objects to use the latest version. The on-disk format exposes your environment to the complete feature set of vSAN.

Enable the vSAN online health workflow

Use the tasks in the following section to make the vSAN baselines available in VUM. vSAN 6.6.1 and later provides a seamless automated update process to ensure that a vSAN cluster is up to date with the best available release to keep your VCF for Classic - Automated instance in a supported state with:

  • vSAN version recommendations - automatically generated by using information from the VMware Compatibility Guide, the vSAN release catalog, and awareness of the underlying hardware configuration. This also includes the necessary drivers and patch updates for the recommended release in its system baseline.
  • vSAN build recommendations - ensures that the clusters remain at the current hardware compatibility status or better.

To see that vSAN updates in VUM the vSAN online Health workflow is followed. Therefore, vSAN Online Health needs to connect to the sites vcsa.vmware.com and vmware.com to perform these online health checks to enable the vSAN Online Health Workflow we need to:

  • Configure the VCSA to use the proxy.
  • Configure vSAN to use the proxy.
  • Enable the Customer Experience Improvement Program (CEIP).
  • Perform a test upload and validate that the upload worked.

The first step is to add your my.vmware.com credentials to the vSAN Build Recommendation Engine. After successful login, vSAN generates a baseline group of recommended updates for each vSAN cluster. vSAN system baselines are listed in the Baselines pane of the Baselines and Groups tab.

Configure the VCSA to use the proxy

  1. From your jump server web browser, connect to the VCSA Management Interface https://<vCenter ip>:5480
  2. Using the credentials from the IBM Cloud® for VMware Solutions console, log in to the VCSA Management Interface as root.
  3. In the VCSA Management Interface, click Networking and click Manage.
  4. To configure a proxy server, in the Proxy Settings pane, click Edit.
  5. Select Use a Proxy Server, enter the proxy server settings, and click OK.

There have been reports where the proxy information is only set for HTTP but not for HTTPS. To configure the proxy information also for HTTPS traffic, it must be enabled first. After you log in to the VCSA through SSH, use the proxy.get command to view the configuration and confirm that the HTTPS parameters are not set.

If the HTTPS parameters are not set, then use the following command: proxy.set --protocol https --server ``<proxy ip>`` --port 3128

Configure vSAN to use the proxy

  1. Navigate to Home > Hosts and Clusters, select the vSAN cluster in the Navigation pane and then select the Configure tab and navigate to vSAN and then General. Scroll to the Internet Connectivity section and click Edit.
  2. Enter the IP address and port number of the proxy, and click OK.

Enable Customer Experience Improvement Program (CEIP)

This is an optional step. By using the vSphere Web Client, go to Home > Administration > Customer Experience Improvement Program, and then click Join.

Complete a test upload and validate that the upload worked

  1. Using the vSphere Web Client navigate to Home > Hosts and Clusters. Select the required cluster and then select the Monitor tab and the vSAN page, then click Health. Click Enable Online Health.
  2. Click Retest and wait for the process to complete.
  3. A new check appears in Health that is called Online health connectivity, and Enable Online Health changes to Retest with Online Health.
  4. Click Retest with Online Health to start the first upload and wait for the process to complete, by reviewing the status in the Recent Tasks pane. The Test Name changes to Online health (Last check: just now).
  5. When completed, in the Health window, scroll to and expand the vSAN Build Recommendation, and click vSAN Build Recommendation Engine Health.
  6. Click Login to my.vmware.com and enter your credentials. When the process completes, the Test Result changes to a Passed state.
  7. Click the Update Manager tab, and the vSAN Cluster is added to the Baselines.

Prerequisites

Before you start the vSAN upgrade process, ensure that the following requirements are met:

  • Review the VMware Knowledge base articles and review any known compatibility issues between your current vSAN version and the wanted target vSAN version.
  • The vSphere environment is up to date:
    • The VCSA must be at an equal or higher patch level than the vSphere ESXi hosts. Update the VCSA if required.
    • All hosts must be running the same build of ESXi. If the vSphere ESXi host versions are not matched, then update.
  • All vSAN disks should be healthy:
    • No disk is failed or absent. This can be determined through the vSAN Disk Management view in the vSphere Web Client. Home > Hosts and Clusters, then select the vSAN Cluster and click the vSAN tab and then Physical Disks. Scroll through all the disks and review the vSAN Health Status.
    • No inaccessible vSAN objects. This can be verified with the vSAN Health Service by clicking Home > Hosts and Clusters, then select the vSAN Cluster. Click the Monitor tab, vSAN and then click Health. Review the Test Results.
    • No active resync at the start of the upgrade process by clicking Home > Hosts and Clusters, then select the vSAN Cluster and click the vSAN tab and then click Resync Components. The Resync components count should be 0. Some resync activity is expected during the upgrade process, as data needs to be synchronized following host restarts.
  • vSphere ESXi host preparation - When you move a host into maintenance mode in a vSAN cluster, you have three options to choose:
    • No data migration - If you select this option, vSAN does not evacuate any data from this host. If you power off or remove the host from the cluster, some virtual machines (VMs) might become inaccessible.
    • Ensure availability - If you select this option, you can move the host into maintenance mode faster than Full data migration and allows access to the VMs in the environment.
    • Full data migration
  • Exit maintenance mode and resync - When the vSphere ESXi host is upgraded and moved out of maintenance mode, a resync occurs. You can see this through the VMware vSphere Web Client. Ensure this is complete before you move onto the next host. A resync is occurring as the host that is updated can now contribute to the vSAN datastore again. It is vital to wait until this resync is complete to ensure that there is no data loss.
  • After starting a vSAN Cluster upgrade:
    • Do not attempt to upgrade a cluster by introducing new versions to the cluster and migrating workloads.
    • If introducing new hosts, ensure that they are of the same initial version and upgrade them along with the rest of the cluster.
    • If you are adding or replacing disks during an upgrade, ensure that they are formatted with the appropriate legacy on-disk format version, if applicable.
    • Therefore, certain vSAN behavior changes are controlled by the on-disk format. It is important that newer on-disk format versions are not introduced into a mixed-version cluster.

Upgrade the VCSA Appliance

For more information, see VCSA update and SSO-linked vCenters.

Upgrade the vSphere ESXi hosts

For more information, see Creating baselines and attaching to inventory objects.

Upgrade the vSAN disk format

Ruby vSphere Console (RVC) is a Ruby-based command-line interface for vSphere and can be used to manage VMware vSphere ESXi and vCenter. The vSphere inventory is presented in a tree structure to navigate and run commands against vCenter objects.

Many basic administrative tasks can be done much more efficiently than clicking through the vSphere Client. RVC is fully implemented in the VCSA and is accused by an SSH connection to the appliance.

  1. SSH to the VCSA and login by using root and the password that is provided on the ICVS Console.

  2. At the prompt, type: rvc Administrator@vsphere.local@localhost and press Enter.

  3. Enter the Administrator’s password provided on the ICVS Console. You are now at the root of the virtual file system, type ls and then press Enter. The output is: 0 / 1 localhost/

  4. Type cd 1, enter and then ls and press Enter. The output is: 0 / datacenter1 (datacenter)

  5. Type cd 0, enter and then ls and press Enter. The output is:

    0 storage/ 1 computers [host]/ 2 networks [network]/ 3 datastores [datastore]/ 4 vms [vm]/

  6. Type cd 1 and press Enter. Then, type ls and press Enter. The output is your cluster: 0 cluster1 (cluster)

  7. Use the vSAN commands against this cluster. To check the disk status type vsan.disks_stats 0 and press Enter.

  8. Make sure that the Health Status for all disks is OK. And then start the upgrade by typing vsan.ondisk_upgrade 0 and then pressing Enter.

  9. Depending on your vSAN size, this task might take some time. When completed, type vsan.objstatusreport 0 and then press Enter to verify that the object versions are upgraded to the new on-disk format.

  10. The vSAN cluster upgrade is now completed. Type exit and press Enter to leave RVC.