IBM Cloud Docs
Updating Istio

Updating Istio

Do not use istioctl to update the version of Istio that is installed by the managed add-on. When updated, the managed Istio add-on includes an update of the Istio version as well.

Updating to a minor version of the Istio add-on

IBM Cloud keeps all your Istio components up-to-date by automatically rolling out patch updates to the most recent version of Istio that is supported by IBM Cloud Kubernetes Service. Supported versions are tested by IBM Cloud and approved for the use in IBM Cloud Kubernetes Service.

Before you begin:

Keep in mind the following as you update minor versions:

  • You can only manually update the Istio add-on one version at a time. To update the Istio add-on by two or more versions, you can repeat the manual update process or you can uninstall the add-on and then install the later version.

  • You can't revert your managed Istio add-on to a previous version. To revert to an earlier minor version, you must uninstall the add-on and then install the earlier version.

  • When you update the Istio control components in the istio-system namespace to the latest minor version, you might experience disruptive changes. Review the following changes that occur during a minor version update.

    • As updates are rolled out to control plane pods, the pods are re-created. The Istio control plane is not fully available until after the update completes.
    • The Istio data plane continues to function during the update. However, some traffic to apps in the service mesh might be interrupted for a short period of time.
    • The external IP address for the istio-ingressgateway load balancer does not change during or after the update.

To update the minor version of the Istio add-on:

  1. Review the current version of your Istio add-on.

    kubectl get iop managed-istio -n ibm-operators -o jsonpath='{.metadata.annotations.version}'

  2. Review the available Istio add-on versions.

    ibmcloud ks addon-versions

  3. Review the changes that are in each version in the Istio add-on change log.

  4. If you are upgrading from version 1.11 to version 1.12 and your Istio components were provisioned at version 1.10 or earlier:

    1. Run the command to get the details of your mutating webhook configurations.

      kubectl get mutatingwebhookconfigurations

      Example output

      NAME                     WEBHOOKS   AGE
istio-sidecar-injector   5          32m

    2. In the output, find the istio-sidecar-injector and review the WEBHOOKS column. If there are 5 or more webhooks, run the following command to delete the additional webhooks.

      kubectl delete mutatingwebhookconfigurations istio-sidecar-injector && kubectl rollout restart deploy addon-istio-operator -n ibm-operators

      Example output

      mutatingwebhookconfiguration.admissionregistration.k8s.io "istio-sidecar-injector" deleted

    3. Check that the additional webhooks were deleted. Get the details of your mutating webhook configurations and verify that there are 4 istio-sidecar-injector webhooks.

      kubectl get mutatingwebhookconfigurations

      Example output

      NAME                     WEBHOOKS   AGE
istio-sidecar-injector   4          60s

    4. Run the command to get the details of your validating webhook configuration.

      kubectl get validatingwebhookconfigurations

      Example output

      NAME                           WEBHOOKS   AGE
istio-validator-istio-system   2          66s
istiod-istio-system            1          31m

    5. Review the output. If the istiod-istio-system webhook is listed, run the following command to delete it.

      kubectl delete ValidatingWebhookConfiguration istiod-istio-system

      Example output

      validatingwebhookconfiguration.admissionregistration.k8s.io "istiod-istio-system" deleted

    6. Verify that the istiod-istio-system webhook is no longer listed.

      kubectl get validatingwebhookconfigurations

      Example output

      NAME                           WEBHOOKS   AGE
istio-validator-istio-system   2          2m

  5. Update the Istio add-on.

    ibmcloud ks cluster addon update istio --version <version> -c <cluster_name_or_ID>

  6. Before you proceed, verify that the update is complete.

    The update process can take up to 20 minutes to complete.

    1. Ensure that the Istio add-on's Health State is normal and the Health Status is Addon Ready. If the state is updating, the update is not yet complete.

      ibmcloud ks cluster addon ls --cluster <cluster_name_or_ID>

    2. Ensure that the control plane component pods in the istio-system namespace have a STATUS of Running.

      kubectl get pods -n istio-system

      NAME                                                     READY   STATUS    RESTARTS   AGE
istio-system    istio-egressgateway-6d4667f999-gjh94     1/1     Running     0          61m
istio-system    istio-egressgateway-6d4667f999-txh56     1/1     Running     0          61m
istio-system    istio-ingressgateway-7bbf8d885-b9xgp     1/1     Running     0          61m
istio-system    istio-ingressgateway-7bbf8d885-xhkv6     1/1     Running     0          61m
istio-system    istiod-5b9b5bfbb7-jvcjz                  1/1     Running     0          60m
istio-system    istiod-5b9b5bfbb7-khcht                  1/1     Running     0          60m

  7. Update your istioctl client and sidecars.

Updating the istioctl client and sidecars

Whenever the Istio managed add-on is updated, update your istioctl client and the Istio sidecars for your app.

For example, the patch version of your add-on might be updated automatically by IBM Cloud Kubernetes Service, or you might update the minor version of your add-on. In either case, update your istioctl client and your app's existing Istio sidecars to match the Istio version of the add-on.

  1. Get the version of your istioctl client and the Istio add-on control plane components.

    istioctl version --short=false

    Example output

    client version: version.BuildInfo{Version:"1.11.2"}
pilot version: version.BuildInfo{Version:1.23.5}
pilot version: version.BuildInfo{Version:1.23.5}
data plane version: version.ProxyInfo{ID:"istio-egressgateway-77bf75c5c-vp97p.istio-system", IstioVersion:1.23.5}
data plane version: version.ProxyInfo{ID:"istio-egressgateway-77bf75c5c-qkhgm.istio-system", IstioVersion:1.23.5}
data plane version: version.ProxyInfo{ID:"istio-ingressgateway-6dcb67b64d-dffhq.istio-system", IstioVersion:1.23.5}
data plane version: version.ProxyInfo{ID:"httpbin-74fb669cc6-svc8x.default", IstioVersion:1.23.5}
data plane version: version.ProxyInfo{ID:"istio-ingressgateway-6dcb67b64d-cs9r9.istio-system", IstioVersion:1.23.5}
...

  2. In the output, compare the client version (istioctl) to the version of the Istio control plane components, such as the pilot version. If the client version and control plane component versions don't match:

    1. Download the istioctl client of the same version as the control plane components.

      curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.23.5 sh -

    2. Navigate to the Istio package directory.

      cd istio-1.23.5

    3. Linux and macOS users: Add the istioctl client to your PATH system variable.

      export PATH=$PWD/bin:$PATH

  3. In the output of step 1, compare the pilot version to the data plane version for each data plane pod.

    • If the pilot version and the data plane version match, no further updates are required.
    • If the pilot version and the data plane version don't match, restart your deployments for the data plane pods that run the old version. The pod name and namespace are listed in each entry as data plane version: version.ProxyInfo{ID:"<pod_name>.<namespace>", IstioVersion:"1.8.4"}.
    kubectl rollout restart deployment <deployment> -n <namespace>

Updating from an unsupported version of the Istio add-on

Update your Istio components to the latest patch version that is supported by IBM Cloud Kubernetes Service.

Migrating from the Istio add-on to community Istio

You can migrate to the community Istio instead of using the managed Istio add-on.

Before you begin

Step 1: Disabling the Istio add-on from the console

Disable the add-on from the console or CLI.

  1. In your cluster dashboard, click the name of the cluster where you want to remove the Istio add-on.

  2. Navigate to the Add-ons section.

  3. On the Managed Istio card, click the Action menu icon.

  4. Click Uninstall. The managed Istio add-on is disabled in this cluster.

  5. On the Managed Istio card, verify that the add-on you uninstalled is no longer listed.

Step 1: Disabling the Istio add-ons from the CLI

Disable the add-on and verify that no additional Istio add-ons remain.

  1. Disable the istio add-on.
    ibmcloud ks cluster addon disable istio --cluster <cluster_name_or_ID> -f
  2. Verify that all managed Istio add-ons are disabled in this cluster. No Istio add-ons are returned in the output.
    ibmcloud ks cluster addon ls --cluster <cluster_name_or_ID>
  3. Wait 10 minutes before continuing to the next step. This gives us time to unmanaged the istio operator.

Step 2: Scale down the Istio operator

Scale down the Istio operator deployment.

Run the following command:

kubectl scale deployment -n ibm-operators addon-istio-operator --replicas=0

Step 3: Saving resources

Save any resources that you created or modified in the istio-system namespace and all Kubernetes resources that were automatically generated by custom resource definitions (CRDs).

  1. Save the managed-istio-custom ConfigMap to troubleshoot a problem or to reinstall the add-on later.

    kubectl get cm -n ibm-operators managed-istio-custom -o yaml > Customizations.yaml

  2. Save all IstioOperator CRs (IOPs).

    • List the IOP resources:
      kubectl get iop -A
    • For each IOP resource listed, remove the finalizer. Example using the managed-istio IOP:
      kubectl patch -n ibm-operators istiooperator/managed-istio --type json --patch='[ { "op": "remove", "path": "/metadata/finalizers" } ]'
    • For each IOP resource listed, save each to a file:
      kubectl get iop -n <IOP_namespace> <IOP_name> -o yaml > <IOP_name>.yaml

  3. Wait 10 minutes before continuing to the next step.

Step 4: Changing the installer of the IOPs

Delete all Istio operator (IOP) resources, such as for a custom ingress gateway.

  1. Make sure your istioctl cli tool is at the necessary patch version.

    istioctl version

  2. For each IOP file that you saved in the previous step, run the upgrade command.

    istioctl upgrade -f <filename>.yaml

Step 5: Removing the Istio operator and IOPs

Delete the Istio operator deployment, service account, cluster role binding, cluster role, and all IOPs.

  1. Run the following commands to delete the istio operator deployment:

    kubectl delete deployment -n ibm-operators addon-istio-operator --ignore-not-found=true
kubectl delete serviceaccount -n ibm-operators addon-istio-operator --ignore-not-found=true
kubectl delete clusterrolebinding addon-istio-operator --ignore-not-found=true
kubectl delete clusterrole addon-istio-operator --ignore-not-found=true

  2. Delete the IOPs.

    • List the IOP resources:
      kubectl get iop -A
    • For each IOP resource listed, delete it:
      kubectl delete IstioOperator <resource_name> -n <namespace>

Step 6: Removing the ConfigMap

Because the ConfigMap was saved earlier, it can be removed.

Remove the managed-istio-custom ConfigMap.

kubectl delete cm -n ibm-operators managed-istio-custom

The removal of the add-on is complete and you can continue to use and upgrade the community Istio as needed.