Why is Submariner connectivity lost or gateway labels missing after an upgrade?
Virtual Private Cloud 4.21 and later
After upgrading your ODF Regional Disaster Recovery (RDR) environment, Submariner connectivity between clusters is lost or worker nodes are missing their gateway labels.
Worker node upgrades can remove node labels, including the Submariner gateway labels that are required for inter-cluster connectivity. When gateway labels are removed, Submariner cannot route traffic between clusters, which disrupts ODF-DR replication and failover.
Reapply the Submariner gateway labels and verify that connectivity is restored.
-
Identify which worker nodes are missing the gateway label.
oc get nodes --show-labels | grep submariner -
Reapply the gateway label to the affected worker nodes. Replace
<node_name>with the name of each affected node.oc label node <node_name> submariner.io/gateway=true -
If needed, adjust the gateway replica count from the ACM console.
- In the ACM console, go to Infrastructure > Clusters > Cluster sets and select your cluster set.
- Go to the Submariner add-ons tab.
- Select the affected cluster and adjust the gateway count.
-
Verify that Submariner gateway pods become healthy.
oc get pods -n submariner-operator -
Verify that inter-cluster connectivity is restored.
- In the ACM console, go to Infrastructure > Clusters > Cluster sets and select your cluster set.
- Go to the Submariner add-ons tab and confirm that connectivity shows as healthy for all clusters.
For more information, see Workaround for Submariner connectivity issues in the Red Hat documentation.